Why does the mirror check whether the player using it is currently in a vanilla dimension?
Compatibility reasons or just to take some power off of it?

Would be cool if it also worked e.g. in the Aether by default, since the other curses like double damage also apply there

I found an intentional dimension check in the source code, like: if (dimension.isVanilla()) teleport(); (pseudo-code)
So maybe it's intentional? Although I don't know why either.

Yep, that's the check I meant
'isInVanillaDimension' definitely seems intentional