For some reason when I add a certain amount of mods to my folder, more structures spawn and they spawn in all dimensions, on top of the world in the nether, and floating or on land in the end. I thought it might be due to another mod not working with the exile mods, but after testing I found that no matter what mods I placed in it would still bug. However, I was able to make it work when I had around 15 mods but when I got around the number 30 it began to bug again.

So with these mods, it worked perfectly.

However with all of the Mods added it bugs, and I have tried to isolate it to just one mod but it seems to be several which leads me to believe it's an issue with the exile mods.

I even tried making a server and going into the config and found that I was able to whitelist biomes and dimensions, but even after doing just that it was still bugged. The default for dimensions is just what is whitelisted so I added another for blacklisted and added the nether and end to that and they still spawned. keep in mind I did this in the config file named world_of_exiles.json5.
Also not sure if this is intended or not but when everything was working I didn't even see one of the black towers in the overworld, but when I was running all of the mods I would see it everywhere.

This is the roof of the nether (with all the mods)

I use AoE 2.1.5 and i have no spawns in any dimension besides overworld and hell (aoe dim) no spawn in the nether nor in the end.

Also i have no problems with crazy spawn rates with the following config changes:

In world of exile config change to these settings and having them rare

	"config": { 
		"spacing": 128,
		"separation": 64,

When i do /locate these are the nearest towers i get from spawn
T = tower LT = ladder tower
From T to Spawn is 2400m

My modlist: