I love creating responsive visuals that respond very naturally to the music. What are your favorite ways of doing this?
To start, I'll list some of the most common and effective ways:
- React to input volume, changing the size or strength of an element accordingly.
- You can also react just to certain frequency bands, like bass/mid/treble. You can also split it up further, but I'm not sure how useful that is (I just stick to bass/mid/treble).
- Beat detection: introduce or change an element on beat frames. Beat detection algorithms work best if you focus on the bass frequencies / add a low pass filter first. (This is actually the simple trick behind many of my best visuals, together with feedback.)
- Old school: directly visualize the current audio stream, either in the time domain (samples) or frequency domain (FFT bands).
I also experimented with these:
- Track actual notes that are being played. Since modern music is pretty consistently recorded and performed in A=440Hz, this works surprisingly well (though I'm still working out the details).
- Automatic tempo detection. To be honest, I haven't found a reliable way to do this (except for simple four-on-the-floor beats), but also: what's the point if you can just tap the tempo?
- Track volume variance or beat density of the music, to notice changes in intensity.
Some of these last ones are not useful to create immediate responses to music, but track changes over a longer time window. They're more useful to notice changes in intensity. For this, it's also useful to note that less intense sections (breakdowns) tend to have more mid and less bass and treble than intense sections.
Anyway, that's what I have experimented with.
Anything I'm missing?
What are your favorite ways to analyze and use live music streams?
(Feel free to link to example videos.)