Audio generated visuals

Itai, aka IIITAIII, is one of the few masters of the modular synth… I also have the privilege of calling him a friend. Total respect for him and his craft here (although I definitely like the harder, heavier stuff). His music career is starting to take off and I will probably be creating a lot more generative video based on his music.

What does that even mean? short version – all sounds have frequency. All music has (potentially) many many frequencies playing at the same time. In these videos, I used an FFT (fast fourier transform) tool inside an application called VVVV. The FFT tool analyzes an audio stream in real-time and produces a measurement of amplitude across all frequencies. I can then take the amplitudes from small slices – say I want the measurement of amplitude from a low frequency which represents a kick drum. I can grab that value as the music plays, then use that value to do something visual, and it’ll be ‘in sync’ with the kick drum. The visual effect will actually just appear to be in sync with the kick drum, but in reality. it is actually driven by the existence of the kick drum on the fly.

With the 2 experiments on this page, I used finished tracks. In other words, an mp3 of a complete song. The FFT works especially well when I can use parts of the song in an isolated fashion… like an mp3 with ONLY the kick drum, rather than existing in a complete mix with a bunch of other low frequency sounds. In the future, I’ll work from isolated sounds for higher quality.

Here is Itai’s linktree, please check him out… and stay tuned for more of my work with his music.