Playing with quark in All the mods 3 lite. Here is the log.
As little as around a dozen foxhounds loaded in at once can almost completely prevent wither skeletons and just about any other mob from spawning in nether fortresses. This problem seems to be resistant to leaving an area (128+m) and reloading again, as well as changing the difficulty to peaceful and switching back to easy/normal/hard, as both of these changes were performed to little/no effect. This might be because foxhounds spawn first in newly loaded chunks, before vanilla mobs do, as doing "/kill @e[type=!player]" and waiting for mobs to spawn in already-loaded chunks appeared to solve the problem.

Problem was discovered via use of spectator mode and applying the glowing effect to all loaded entities on a repeating command block. I could not find any other reports detailing this issue, so I'm making one myself.

This has been fixed in later quark versions.