I don't know why, when I deleted the module and started the server.
My background keeps spawning garbage logs.

Is there a way around this?

Looks like the structure is registered in the world before you remove the mod. I am not sure how to solve this issue but probably modifying/resetting level.dat might help.

There seems to be no good way to solve this problem