A couple of weeks ago, along with a few others, I joined my friends at Said.fm for a hack weekend. We threw around some suggestions before and during the event; one of the ideas that coalesced was a visualiser for Said.fm’s RadioBox events, and that’s what I ended up working on.

What we wanted to achieve was to make it easy to produce a synchronised visual accompaniment to an audio recording, the idea being that the audience can sit and listen whilst simultaneously being presented with appropriate visual information.

SoundCloud lets you attach comments to specific points in tracks, which makes it a useful interface for easily synchronising data with an audio stream. Our hack uses the SoundCloud API to get these comments and, depending on what kind of information they contain, presents that in the browser as the audio reaches the appropriate point. For example, a Flickr URL becomes (via the Flickr API) a background image. A Wikipedia link is turned into an abstract of that encyclopedia page. Plain old text is displayed as-is.

At the moment, it’s a fairly basic system: the CSS is perfunctory, and it doesn’t look particularly appealing. But it is easily extensible: all that it takes to add a new kind of information is to define a server action that decodes the URL and to add a regular expression for that URL. With a little design love, it could be rather beautiful.

The demo works in Chrome; it might work in Safari or even in Internet Explorer, though I haven’t tested those. It definitely won’t work in Firefox, unfortunately, because that browser doesn’t support MP3 audio, which is all that’s available from the SoundCloud API. It’s sad when politics and legal arguments get in the way of technology, but that’s the way it is.

If you’re on a slow connection, it will take a while to start: it loads all the images and data beforehand to ensure that everything can be shown at the right point in the audio without waiting for HTTP requests.

That said, here are a couple of examples (tip: move the mouse to the bottom of the page to get to the audio controls):

You can also see the source code on GitHub.

For comparison, have a look at the original content on Said.fm’s SoundCloud page.