Minecraft Version

1.18.x

Version

2562642

Details

I ran into this issue while trying to understand the root cause of the problem that #993 aims to fix. The behavior was seen on the commit hash I used in the version number of this report (it's not the PR code).

Reproduction steps:

1. Add logging breakpoints in DfpwmStream to monitor the size of the buffer queue on client.
2. Play a long audio stream, simple is better so the server can keep ahead of the stream. I was using this code:

local speaker = peripheral.find("speaker")

local chunkSize = (arg[1] or 1) * 48000

local t, dt = 0, 2 * math.pi * 220 / 48000
while true do
    local buffer = {}
    for i = 1, chunkSize do
        buffer[i] = math.floor(math.sin(t) * 127)
        t = (t + dt) % (math.pi * 2)
    end

    while not speaker.playAudio(buffer) do
        os.pullEvent("speaker_audio_empty")
    end
end

The issue occurs on any chunk size, but I was using 1 second (48,000 samples).

3. Pause the game, resume the game, repeat. The size of queued buffers will start growing and playback will not catch up.
4. Terminate the program and wait a long time for playback to stop.

Here's a screenshot of 16 buffers in the queue.

It took 27 seconds for playback to stop after terminating the program, though the buffers should have been 1 second of audio each. Perhaps even more data was piling up in minecraft's own buffers?

Admittedly a bit of a ramble:

This doesn't seem like a very important issue but I'm reporting it because it's specific to reproduce and I think highlights a larger issue: there is no feedback mechanism for the client to slow an audio stream when data starts to pile up. At some points in even a well behaving stream, the client can have a few seconds of data buffered with no way to let the server know to slow down, which causes latency. Granted, you only notice this latency when stopping playback on the server side (you notice the delay before playback actually stops) or if you have some sort of playback visualizer running on the server side (like Jack's tracc program).

I think #993 only ameliorates this pile up for well behaving streams by changing the CLIENT_BUFFER tuning factor in DfpwmState and possibly using smaller OpenAL buffers (which may or may not be worth having to re-create logic around volume sliders and pausing). In a badly behaving stream where the client misses some of the server's predicted playback time (as is happening in this issue), #993 won't help. Maybe there's a clever way to solve both issues at once?

I thought about this some more and realized the idea of the client being able to tell the server to slow the stream is a non-starter. There are multiple clients per server in multiplayer and tracking client playback on the server would be a disaster.

Maybe instead clients should imperceptibly speed up playback when they're behind? There is already an iteration over the data for the dfpwm decoding. Maybe samples could be dropped there without a performance impact?