TTM seems too arbitrary, training a small 'addon' model on titles/artists could maybe help ?

[DHG-Dav]

Hello i just tried the TTM today, the behavior seems... Random.
Maybe you could consider training a smaller network (VAE or transformer ?) on titles and "inject" it on the network to condition the generation ?
I mean a little bit like loras/dreambooth/hypernetwork techniques, you keep your main model frozen, you add a few layers that takes song titles (as tensor) and should predict the song 200 notes through the model, so a few keywords (like waltz, sonate, minor, major and such) would actually have a generalized meaning rather than just taking a more or less random sample as input.
That's just an idea, i'm not native english i hope my explanation is understandable.
Thank you for your time, dedication and project !

[asigalov61]

@DHG-Dav Thank you for trying the TTM demo and thank you for your feedback! :)

I tried using AI for TTM but it produces similar results (accuracy rate on music titles/artists is ~75%). Here is the example implementation of mine if you want to take a look:

https://github.com/asigalov61/DeBussy/blob/main/DeBussy_CLaMP.ipynb

The reason why Fuzzy text matching == AI is because music titles/artists do not contain enough information for the model to generalize precise enough. Music is a bit more complex than images or text so its not that simple as just training the model.

As far as Euterpe TTM goes, you can make it less random by disabling random.shuffle() function and also by using options of the fuzzywuzzy process() function.

Hope this makes sense and thank you again for your interest in the project and your feedback.

Alex

[DHG-Dav]

Yes thank you ! i'm sorry it was silly from me, i had classical music in mind (most titles are very obvious, like "xxx sonate in B flat"), and didn't think about most musics with just arbitrary titles... It's a description that would be needed but i doubt you'll find a dataset that includes it.

Thank you for your answer !
Dav

[asigalov61]

@DHG-Dav PS. Microsoft made the following implementation for controlled text-to-music generation but it produces about the same accuracy because music is very complicated thing and it can be very subjective regardless of precision:

https://github.com/microsoft/muzic/tree/main/musecoco

[asigalov61]

@DHG-Dav PPS I also have this:

https://github.com/asigalov61/Classi

Try it out if you want to play with MIDI classifier model.

Alex