I loaded in my world pretty close to a castle and Boom Boom was not in there, so I quickly teleported around the map looking for more castles and did not see him in any of them. Spawn eggs work, but he does not seem to spawn naturally? Unless I'm just moving around the world too fast and he doesn't enter the world that close to spawn?

Sounds like either a mod incompatibility issue or a certain forge version that's preventing it from spawning.
I'm currently running a bunch of other mods alongside this mod and Forge 47.2.0, and Boom Boom has been spawning in every Brick Fortress Structure for me.