Is there anything I can do to speed aux scans up? It seems like it takes forever to search through items. Like doing a scan right now of 240 pages it's taking probably around 20-30min.

Is this just the way it works on vanilla? Or is there something wrong with my configuration?

/aux ignore owner will speed it up a bit but in recent versions that should already be on by default.
Other than that yeah that's vanilla.