AI music maker - Riffusion
I hope you have seen the recently released and shocking applications of AI. Whether it's creating images of the words you give him or chatting with you so smoothly that you may think it's a human, AI is all that's hot right now in the internet.
Well, it turns out AI is not done yet. I present you Riffusion. Created by Seth Forsgren and Hayk Martiros as a hobby project, they tuned a model of stable diffusion to create spectrograms, which can be then converted to music. They have a really good and easy to understand explanation on the project's about page but the important part is that you tell the model some words, it creates an image of that and then each pixel of these images is interpreted using math models, processed and converted into music.
One example they have is the following interpretation of the words rock and roll electric guitar solo. This creates a spectrogram of those words based on the trained model, which looks like this:
Then the project processes this spectrogram and turns it into the following melody:
Pretty neat, right? In my opinion, it created a really cool melody that you can easily identify as rock because of the drum tempo and the guitar playing a simple but powerful riff.
Now, the interesting part of the project is that you can interact with it and ask it to create whatever you like. You just have to go to the Riffusion website, type your words and see it create the image and interpret it.
Let me know what you type and if it surprised you at @scaarg. Rock and roll... and AI!
