Strangely the music discs seem to randomly play in the world on loop for a few times, it doesnt seem to be when around a structure or in any particular biome, though it does dissapear and reappear when leaving and entering the area.

Note im playing in 1.20.1 on forge version 47.3.0

Sounds like you have one of those mods that adds skeletons with backpacks that play music? I forget the mods name