Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor waveform aligner and add test #766

Merged

Conversation

dpw13
Copy link
Collaborator

@dpw13 dpw13 commented Feb 2, 2024

This PR adds a friend test for WaveformAligner that verifies a few internal implementation details as well as verifying proper alignment for waveforms with supported offsets.

This PR also refactors WaveformAligner for readability and fixes two bugs:

  1. The code to calculate errorSum is supposed to calculate a windowed convolution but introduced an indexing bug that resulted in repeatedly calculating the product of the two waveforms instead.
  2. The calculated "mip levels" or resampled waveforms were recalculated in the original Milkdrop code but was re-used as an optimization in the projectM code. However the re-used mip levels did not take the applied shift into account, resulting in the calculated offset being relative to the previous unshifted waveform. This bug was not present in the original Milkdrop code since the mip levels were recalculated including the applied shift.

@dpw13 dpw13 marked this pull request as draft February 2, 2024 04:51
@dpw13 dpw13 force-pushed the dev/dwagner-audio-alignment-tests branch 2 times, most recently from a2f695d to 086aa37 Compare February 2, 2024 04:57
@dpw13 dpw13 marked this pull request as ready for review February 2, 2024 05:06
{
errorSum += std::abs((newWaveformMips[octave][i + sample] - m_oldWaveformMips[octave][i + sample]) * m_aligmentWeights[octave][i]);
errorSum += std::abs((newWaveformMips[octave][i + offset] - m_oldWaveformMips[octave][i]) * m_aligmentWeights[octave][i]);
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix 1.


// Store mip levels for the next frame. Note that we need to recalculate the mips for the *shifted*
// waveform, so we can't reuse the previous mips.
ResampleOctaves(m_oldWaveformMips, newWaveform);
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix 2.

Copy link
Member

@kblaschke kblaschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please split the individual changes into separate commits: one for the reformatting, one for each fix and one for the tests. This will make it easier to review and later much easier to see what was fixed where and why, without the need to dig up this PR.

If possible, add the reasoning for each change as mentioned in the PR into the respective commit messages.

@dpw13 dpw13 force-pushed the dev/dwagner-audio-alignment-tests branch from 086aa37 to e81fe7f Compare February 2, 2024 16:38
@dpw13
Copy link
Collaborator Author

dpw13 commented Feb 2, 2024

I split the commit into several pieces that should be much easier to review. I've explained what's changing in each commit. The commit that splits Align into multiple methods is likely the more difficult to review; I suggest disabling viewing whitespace changes with that commit. I've ordered the methods so hopefully the diff looks cleaner than before (where ResampleOctaves was placed too low in the file). The last commit contains only the actual fixes so git blame should make it clear what's changed and why.

@dpw13 dpw13 requested a review from kblaschke February 2, 2024 16:44
Copy link
Member

@kblaschke kblaschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides ideally running clang-format on the changed files before each commit, it looks good to me!

src/libprojectM/Audio/WaveformAligner.cpp Outdated Show resolved Hide resolved
src/libprojectM/Audio/WaveformAligner.hpp Outdated Show resolved Hide resolved
Copy link
Member

@kblaschke kblaschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean and well tested! If you're ready, I could merge at any time.

@kblaschke kblaschke added this to the 4.1 milestone Feb 5, 2024
@kblaschke kblaschke added the bug label Feb 5, 2024
@dpw13 dpw13 force-pushed the dev/dwagner-audio-alignment-tests branch from d1f46d1 to c5ecad0 Compare February 6, 2024 13:58
@kblaschke kblaschke merged commit c5ecad0 into projectM-visualizer:master Feb 6, 2024
9 checks passed
@dpw13 dpw13 deleted the dev/dwagner-audio-alignment-tests branch February 21, 2024 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Development

Successfully merging this pull request may close these issues.

3 participants