Keyboard reacts to music

[ItsVince1234]

Would it be possible for the keyboard to react to music?

[4JX]

Originally did not have much hope for such an effect, but it seems its actually possible to capture outputted audio, so the answer is "probably yes".

That being said, do you have any preference on how it'd visualize it? It could change intensity, color, be a bar (left to right / right to left). I'll probably just end up doing a slight UI rework before doing more than one of the variations.

[HyperN00B]

I think flashing the lights in the selected colors with different intensity depending on the music could work.

[fa1riyi]

I have been always wondering if you can make 4 zones change their colors seperately (no effect of left to right or right to left). For music, can you alter the speed of color changing base on tempo of music? (not sure if we can capture tempo of music), and the higher tone of sound, the lighter of color, and the lower tone of sound, the darker of color, and this should be based on Hz of sound I think.

[aarmn]

I think I can work on implementing this feature, but Im a novice in Rust so I may make it on python, also working with music streams is hugely platform dependent and I found very few libs to support multiple OS and multiple libs in Lin (pulseaudio and ALSA), also its not a promise!!!!!

[4JX]

Hey sorry for the late response on this one, you can try based on my current approach, see:

L5P-Keyboard-RGB/src/keyboard_manager.rs

Lines 359 to 417 in 8b88c89

	Effects::Sound => {
	// Adapted from https://github.com/dheijl/swyh-rs/blob/44676eb543d4238311c87b68858589e60d0debe7/src/main.rs
	// first initialize cpal audio to prevent COM reinitialize panic on Windows
	let mut audio_output_device = get_default_audio_output_device().expect("No default audio device");
	let audio_devices = get_output_audio_devices();
	let mut source_names: Vec<String> = Vec::new();
	for adev in audio_devices {
	let devname = adev.name().unwrap();
	if devname == "pipewire" {
	audio_output_device = adev;
	}
	source_names.push(devname);
	}
	let (sender, receiver): (Sender<Vec<f32>>, Receiver<Vec<f32>>) = unbounded();
	let stream: cpal::Stream;
	match capture_output_audio(&audio_output_device, sender) {
	Some(s) => {
	stream = s;
	stream.play().unwrap();
	}
	None => {
	panic!("Unable to capture audio output");
	}
	}
	while !self.stop_signals.manager_stop_signal.load(Ordering::SeqCst) {
	let mut nsamples = 0_i64;
	let mut sum_l = 0_i64;
	let mut sum_r = 0_i64;
	if let Ok(samples) = receiver.recv() {
	for (n, sample) in samples.iter().enumerate() {
	nsamples += 1;
	let i64sample = i64::from((*sample).to_i16());
	if n & 1 == 0 {
	sum_l += i64sample * i64sample;
	} else {
	sum_r += i64sample * i64sample;
	}
	//max is 16384
	let _rms_l = (((sum_l / nsamples) as f64).sqrt()) * 255.0 / 16384.0;
	let _rms_r = (((sum_r / nsamples) as f64).sqrt()) * 255.0 / 16384.0;
	println!("{}", _rms_l);
	// TODO: Figure out why uncommenting this line makes it not work
	// self.keyboard.solid_set_colors_to([_rms_l as u8, _rms_l as u8, _rms_l as u8]);
	nsamples = 0;
	sum_l = 0;
	sum_r = 0;
	}
	}
	}
	}

Its almost there prototype wise, just haven't figured out why it sets everything to 0 when you add that line.

• Tested it on my Linux machine, you'll need to use pavucontrol and in the Recording tab, while the Sound effect is active, set the capture from to "Monitor of X".