Would there be a way to limit the specters we take out with the Soul Reaper, just like Nightfall has its limit of 3 Remnants?

Nightfall doesn't have any limit, and neither does the Soul Reaper. I could set up a limit line in the config that can be changed from -1 (no limit) no a number you choose (like 3).

It would be great for servers. Some have the Soul Reaper banned due to summon spam.

That makes sense, I'll add that shortly.