In this tutorial: I am going to show you how to build a version of Firefox that can read the <audio> buffer values in JavaScript.

It is the first Sunday of the new year. Many are calling it “The new decade.” But this is not a new decade. Calendars started at “1″ as pointed out by George Adams. Still… the new year is a great time to do something revolutionary. My recent experimentation has been focused on the Processing.js community’s effort to make HTML <audio> data available in JavaScript using Firefox. And the progress has been great!

The story so far…

1. First Thomas Saunders got the ball rolling by posting an audio bug with Mozilla and created a neat demonstration of what could be possible if JavaScript could tap into the audio stream. Thomas’s blog is really interesting, if you get a few minutes in the morning over coffee it is well worth a read.
2. Then David Humprey, a teacher at Open Seneca began hacking away at Firefox to see if he could expose the audio stream to the terminal. The results did not mean much to Dave, but they looked familiar to me. Dave’s results can be seen here: http://vocamus.net/dave/?p=914
3. It was then my turn. I created a test-tone in Audacity, from which I grabbed a screen shot. This let us see what the wave form should look like. I passed the “.ogg” test-tone to Thomas who posted back his console values as a tidy JavaScript array. I rebuilt these with JavaScript in the Canvas. The images were identical, proving that Dave was right on track with his code. My results can be seen here:
   Reading the Audio Stream from Firefox 3.7
4. Dave kept things intense, with regular posts on his progress. On new year’s eve -eve, Dave sacrificed punctuality to help me test out his latest code. This was the first time I saw raw audio data streaming directly into JavaScript, a very exciting moment!! Audio Part III, Audio Part IV, Audio Part V.
5. Corban Brook, stepped onto the scene and threw some FFT magic into the mix. Corban created a simple spectrum analyser, completely JavaScript based. You can just stare at the beautiful video capture here: Firefox JavaScript <Audio> FFT.

Build Your Own Firefox

The rest of this post is going to be about building your own audio-enhanced version of Firefox. This custom build will include Dave Humphrey’s audio patch so you can read the audio buffer values in JavaScript. If you’re looking to get involved in bringing audio data to the web browser, or you just want to learn how to apply a code patch to a new build of Firefox, this is a great place to start. I have just installed Ubuntu 9 Karmic Koala this morning so this should also serve as a basic guide to getting things up and running in Linux from scratch.

Phase 1: Create The Build Environment

The first step is to follow the build instructions at the Mozilla Developer Center: Build Documentation. For the Ubuntu users out there, I will give you the exact commands I typed, so you should be able to follow this without reading any documentation what so ever.

a) Install Firefox build dependencies

$ sudo apt-get build-dep firefox
$ sudo apt-get install mercurial libasound2-dev libcurl4-openssl-dev libnotify-dev libxt-dev libiw-dev mesa-common-dev autoconf2.13

b) Check out the Firefox source

$ cd ~
$ hg clone http://hg.mozilla.org/mozilla-central

This step took a few minutes on my quad-core Intel with a 5mb connection.

c) Roll back your tree

$ cd ~/mozilla-central
$ hg update -r 683dfdc4adf0

This just rolls back the tree to the build version that Dave has been using for his experiments.

d) Download Humprey’s Audio Patch

wget -O /tmp/audio3.diff http://bocoup.com/core/code/audio3.diff

That copies Dave’s audio patch to your /tmp/ directory.

e) Set up “mozconfig” build configuration
Copy this block of text to your clip board, and paste it into a new file called “.mozconfig“.

. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/objdir-release
mk_add_options MOZ_MAKE_FLAGS="-j4"
mk_add_options AUTOCONF=autoconf2.13

Take care with the autoconf2.13. This number will depend on your system. In linux/osx you can try typing “autoconf” and pressing TAB TAB, which should list the autoconf’s available.

nano .mozconfig

Now paste the block of text you copied a second ago, into your .mozconfig file. Save your .mozconfig and exit. You may need to come back and adjust this as mentioned above.

Phase 2: Apply, Build & Test

Now that you have everything set up, it is time to apply the new source code to Firefox.

a) Patch it up

$ patch -p1 < /tmp/audio3.diff

b)Make your build

make -f client.mk build

You should see a torrent of text bursting from your terminal. If you encounter an error, log it, join Processing.js on IRC and “Humph” will probably help you fix it if you ask him really nicely. Once this is done you will need to set up a way to run your new build without interfering with your default Firefox install.

c) Create a new test profile

$ cd ~/mozilla-central/objdir-release/dist/bin
$ ./firefox -profilemanager --no-remote

1. Click “Create Profile”
2. Click “Next”
3. Type “audio3.diff”
4. Click “Finish”
5. Click “Start Minefield”

d) Test your <audio> buffer
Congratulations your custom build of Firefox is running! Now you need something cool to test it on. Try out Corban’s JavaScript Audio FFT demo: JavaScript <audio> FFT

If everything is working as it should you will see this:



Phase 3: Join The Web Browser Audio Revolution

Visit the Processing.js development team at irc.mozilla.net #processing.js and help us upgrade the WWW.