I'm not the greatest at making working configs but following the tutorials and documentation I was under the impression that this is the way to deal with bees because they don't follow normal spawning rules. If this is my doing and not a bug then I'll gladly hear the solution!

Added my spawn.json file just in case

spawn.json.txt

This is not a bug. Just a consequence of how this works. When a beehive is ready to spawn it will spawn until it succeeds. So canceling the spawn it will have weird effects. Nothing I can do about

Sorry if this is not the place to ask but what's the best way to deal with them then?

I have absolutely no idea. Mostly because I don't exactly know what you mean by 'deal with them'. What's your goal?

yes, that might be the best solution here

Removing all mobs unless from spawners and structures but my guess is that I'll have to disable the generation of beehives, correct?

This is not a bug. Just a consequence of how this works. When a beehive is ready to spawn it will spawn until it succeeds. So canceling the spawn it will have weird effects. Nothing I can do about