Audio starts jittering the more sounds are played

[cybersoulK]

after a few hundred source nodes connect and disconnect, there are a lot of loud pops or sound jitter. / abrupt interruptions (some leak in web-audio-api or cpal usage?)

pop.mp3.zip (internal recording of the bullets sounds,

• Cpal backend is used
• Web equivalent does well
• One buffer per source node. (Cloned)
• A Node graph is created (and later disconnected) per sound: Source -> gain -> destination
• None of these nodes are connected with anything else.
• This issues happens on both Mac and windows

Render capacity shows increasing numbers:

I used kira.rs with the cpal backend, and it never did this.
I just compared them side by side with exact same sound conditions.

My minimal code for each sound that plays:

 let options = AudioBufferSourceOptions::default();
 let mut source_node = AudioBufferSourceNode::new(context, options);
source_node.set_buffer(buffer);

let mut options = GainOptions::default();
options.gain = volume;
let gain_node = GainNode::new(context, options);

source_node.connect(&gain_node);
gain_node.connect(&context.destination());

source_node.start();

Sound {
   source_node,
   gain_node,
}

and later when the sound stops:

impl Drop for Sound {
   self.source_node.disconnect();
   self.gain_node.disconnect();
}

I made sure that all of the sounds are being dropped and their nodes disconnected

[orottier]

Thanks for the detailed report.
I can confirm on my machine that we run into max load with about 500 buffersourcenodes+gain concurrently and adding 10 nodes in bursts every 5 milliseconds.
I captured a profile which indicates we spend about 75% of the CPU time ordering the audio graph, so that is definitely the area to look for improvements.
FYI From other benchmarks we know that for static audio graphs we are about a factor 2 slower than the browser implementations.

[cybersoulK]

but i don't have 500 nodes connected at a given time,
it's only 5 sounds maximum playing at a time. (1 cloned buffer + 1 source + 1 gain nodes)

After the hundreds of sounds play, everything should be disconnected and cleared,
so this made me think something is leaking and making the cpu choke...

[orottier]

Right, I misunderstood.

Changing my stress test to run less nodes concurrently but checking the load for a longer running time, I can't seem to reproduce this issue.

Are you sure the Drop method is called? Or are you holding on to the Sounds for a very long time?

In the current implementation, we don't free up audio processor resources when the control thread still holds on to the corresponding AudioNode. I believe @b-ma already mentioned that we should not do this in case the node has ended/stopped and cannot be started again but this is not implemented yet: #397

Tangentially, we should probably add an AudioContext::print_diagnostics to aid debugging in these cases. It should print some counters and the full audio graph

[b-ma]

Hey, I'm a bit surprised too with these numbers because this looks almost the same setup (maybe even less demanding) as in the granular example: AudioBufferSourceNode -> GainNode -> Destination

In this example the period between each grain is 0.01 sec and their duration is 0.2 seconds, which means we spawn a 100 sources / second with 20 sources are overlapping at every moment.

Do you have the same issue if you run this example?

[b-ma]

The only difference I can think of is the size of the AudioBuffer (even if I don't really get [edit: right now] why it would change anything), but just in case, could you tell us more about your audio buffer (sample rate, duration, etc.)?
And maybe for how long your source are generally playing, is stop is called on your sources at some point?

[b-ma]

Just another question, is your source looping?

[cybersoulK]

Just another question, is your source looping?

I have some looping, but they only play 20 times. The ones that play hundreds of times are not looping and are less than 0.5 seconds in duration.

[b-ma]

Really strange... Does it helps if you explicitly stop() the node before disconnect()?

impl Drop for Sound {
   self.source_node.stop();
   // this is not mandatory then, but doesn't hurt 
   self.source_node.disconnect();
   self.gain_node.disconnect();
}

[cybersoulK]

using stop doesn't fix the issue.
it's odd that you were not able to replicate this issue yet.
i will try to make a minimal example

[cybersoulK]

https://github.com/cybersoulK/debug_web_audio_api
i made a minimal example, can you check if after 1 minute the audio breaks?

[b-ma]

Hey, thanks for the example, I think I managed to make it work by just doing:

impl Drop for Sound {
    fn drop(&mut self) {
        // self.source_node.disconnect();
        // self.gain_node.disconnect();
    }
}

But I have no idea of the why, this is a bit problematic... If I leave any of the disconnect methods, then at some point your active_sounds list starts to grow indefinitely, just like if the onened callback was not called, @orottier any idea?

Just asking myself and maybe this is because we have only part of the code, but I don't understand why you keep this list of active nodes? You could just spawn them and forget them right away

[orottier]

Hah, now this is interesting.

It's a manifestation of the bug I just fixed in 7b048c2
This fix is not present in v0.36.1

The fix addresses the issue that disconnect disassociates the nodes from its audio params. Dynamic lifetime kicks in for the AudioBufferSourceNode which is freed, but because the AudioParams have tail time true, they are never decommissioned.

I haven't tested it yet but I'm convinced this issue does not occur on the main branch now.

Just asking myself and maybe this is because we have only part of the code, but I don't understand why you keep this list of active nodes? You could just spawn them and forget them right away

Indeed. The Web Audio API deals with this for you via dynamic lifetimes, search "fire and forget" on https://developer.mozilla.org/en-US/docs/Web/API/AudioBufferSourceNode
Meaning, just call start() and perhaps call stop(time) on it and then just drop the node. The library will free up the resources when the sound is done playing.

[cybersoulK]

i confirm that the current master solved the issue

[orottier]

Great to hear! Thanks to your bug report we fixed a logic bug and a performance issue. And we're prepping up a method to better aid users in submitting diagnostics via #401

I will publish a new release of the library this weekend.