Context

I've been toying around with audio and speakers in CC a lot lately, and a major issue I've noticed is desync, especially if you have a large amount of speakers. Even some of the current best libraries for working with audio (like JackMac's AUKit) are riddled with issues of desync -- and I don't think it's really that easy for them to fix it. The fact that the audio is played on the clients means that syncing multiple speakers is not going to be easy when each speaker doesn't know about each other speaker and what they're currently playing.

So the issue now becomes, how do we fix this issue without making it seem unrealistic, or without having to overhaul everything and breaking back compat? If we can't solve it in software, it will need to be solved with 'hardware.'

Suggestion

Speaker networks, like the title of the feature request.

I believe Computronics really had a good route. One block acted as a 'controller' of sorts (Tape Drive), while the actual speakers themselves connected to it to play audio. No, I'm not saying to add in tape drives, but having one block control multiple speakers may be the way to go about this.

One peripheral call to play audio on all connected speakers, with a separate "audio cable" to connect speakers (or perhaps it just plays on all connected speakers on the same CC network). This would allow CC and the client to know that "this sound should be played on all these speakers at the same time." As well, this would mean only one speaker_audio_empty event (perhaps controller_audio_empty?) would need to be queued instead of the current spam of events we get when we have multiple speakers attached.

This is realistic enough as well, we'd not be stepping out of bounds into "meta-ness" or anything with this. My own surround sound system at home only connects the TV to one speaker, the rest are controlled wirelessly by that single speaker.

Pros

• Multiple speaker in-sync playback would be much easier to achieve.
• Ideally speakers would still be able to connect to computers normally, so it wouldn't break backwards compatibility.

Cons

• Wireless in-sync playback would still suffer.
  • I believe if you're trying to play audio over distances where you need a wireless system, you won't be able to hear if it's out of sync with the "main" system anyways.
• Complexity, for both the user and maintainer.
  • New users may think that, in order to use speakers, they must connect the audio cables to the computer rather than the audio controller. This could be mitigated by having the controller just play audio on the same CC network it's connected to and not having a separate cable network, but that just feels slightly weird to me.
  • This would likely require an overhaul of the current audio playback system (or at least, an addon to it), as well as the controller itself.

Footnote

Had this idea while working on some code. I know it is probably a large undertaking to make an entirely new networking system and reworking the audio, but I wanted to write it down in case there were any thoughts about it.

Unfortunately I do not believe this is possible with Minecraft's new sound system. Computronics did this by using OpenAL directly, but that's something I'd rather avoid.

One neat approach Phonos goes with is to share a sound instance across all connected speakers, and then set the position of that sound to be some weighted average¹ of nearby speakers. This definitely won't work with any of the sound physics mods, but is a pretty clever idea.

What I'm less sure on, is if we want to do audio cables at all. The main advantage of audio cables is really server/network cost - as audio is shared between speakers, you don't need to send multiple identical packets to the client (see also #1313). I don't think implementation is any easier, and it doesn't solve the fundamental issue of audio desync.

I think really the best first step is to characterise the desync people see (how often does this happen, how bad is the desync, single player or multi-player) and then get a better idea of why this happens. Everyone is aware of the desync issue, but I think the only bug report I've received about it is cc-tweaked/cc-restitched#144, which is very different issue!

I can think of a few obvious causes (mostly around audio queueing being split across multiple ticks), but none of that should introduce more than 0.1s delay. My understanding is that sometimes people see much worse desync,

I struggle to reproduce desync locally - both in single player and on a local server, which suggests this is (at least partially!) caused by networking, but I don't know for sure.

Could we detect that the speakers are getting the same audio source and then have the player only able to hear the nearer one? When they move to another speaker playing the same audio the sound should seem to jump to the now nearer speaker.
It'll be odd, but things won't be overlapping and people can still boost the range of sounds via multiple speakers.

Alternatively, I guess that we could make speakers into multi-blocks and the more blocks the more range the sound has, not necessarily louder just travels further.

Neither of these would allow people to do stereo setups but people don't seem to be attempting to do that very often.

Could we detect that the speakers are getting the same audio source and then have the player only able to hear the nearer one?

Without knowing what's causing the desync¹, I don't know how reliable such a detection would be.