If you have it set that you don't respawn in the Nether, a toggleable enemy that is only visible to those who are intangible should rarely spawn. If you get "killed" by these enemies, you are sent to the Nether (or whichever dimension you set in the config.)

That's a good remark, we'll take that into account if we make that mob. In the mean time, a whitelist has been added of mobs that you can hit all the time and that can see and hit you when you're in ectoplasm mode (yea, that mode isn't currently accessible other than by commands). Sadly, due to the way soul mode invulnerability works, a mod would have to do explicit AI for its mobs to attack souls.

Would it be possible to make the mob spawn rate configurable as well as allowing for an option that if killed in spirit from by said mob, you can no longer spawn? This would be a great addition for those of us that do hardcore.

We had some plans for an enemy like that some time ago, which is why we made players respawn in the nether for a few versions. It will probably come some time in the future, though at the moment we are busy enough adding decent mechanics to the mod.
I will try to add the config for next version.

Only hostile / harmable to/by those who are currently in ghost mode.

And config for mobs you can fight and will fight you while in ghost mode would be good too. I'm sure there's a decent amount of mobs that would be dangerous to players, both living and dead.