Web Audio API comes to Firefox

We have been working on implementing the Web Audio API in Firefox for a while now, and we currently have basic support for the API implemented on Firefox Nightly and Firefox Aurora. Web Audio provides a number of cool features that can be used in order to create music applications, games, and basically any application which requires advanced audio processing.

Features

Here are some examples of the features:

  • Scheduling events to happen at exact times during audio playbacks
  • Various types of audio filters to create effects such as echo, noise cancellation, etc.
  • Sound synthesis to create electronic music
  • 3D positional audio to simulate effects such as a sound source moving around the scene in a game
  • Integration for WebRTC to apply effects to sound coming in from external input (a WebRTC call, a guitar plugged in to your device, etc.) or to sound which is transmitted to the other party in a WebRTC call
  • Analysing the audio data in order to create sound visualizers, etc.

Code sample

Here is a simple example of what you can build with Web Audio. Let’s imagine that you’re working on a game, and you want to play a gunshot sound as soon as the player clicks on your game canvas. In order to make sure that you’re not affected by things like network delay, the audio decoder delay, etc., you can use Web Audio to preload the audio into a buffer as part of the loading process of your game, and schedule it precisely when you receive a click event.

In order to create a neater sound effect, we can additionally loop the sound while the mouse is pressed, and create a fade-out effect when you release the mouse. The following code sample shows how to do that:

// Load the sound file from the network
var decodedBuffer;
var ctx = new AudioContext();
var xhr = new XMLHttpRequest();
xhr.open("GET", "gunshot.ogg", true);
xhr.responseType = "arraybuffer";
xhr.send();
xhr.onload = function() {
  // At this point, xhr.response contains the encoded data for gunshot.ogg,
  // so let's decode it into an AudioBuffer first.
  ctx.decodeAudioData(xhr.response, function onDecodeSuccess(buffer) {
    decodedBuffer = buffer;
  }, function onDecodeFailure() { alert('decode error!'); });
};
 
// Set up a mousedown/mouseup handler on your game canvas
canvas.addEventListener("mousedown", function onMouseDown() {
  var src = ctx.createBufferSource();
  src.buffer = decodedBuffer;                                      // play back the decoded buffer
  src.loop = true;                                                 // set the sound to loop while the mouse is down
  var gain = ctx.createGain();                                     // create a gain node in order to create the fade-out effect when the mouse is released
  src.connect(gain);
  gain.connect(ctx.destination);
  canvas.src = src;                                                // save a reference to our nodes to use it later
  canvas.gain = gain;
  src.start(0);                                                    // start playback immediately
}, false);
canvas.addEventListener("mouseup", function onMouseUp() {
  var src = canvas.src, gain = canvas.gain;
  src.stop(ctx.currentTime + 0.2);                                 // set up playback to stop in 200ms
  gain.gain.setValueAtTime(1.0, ctx.currentTime);
  gain.gain.linearRampToValueAtTime(0.001, ctx.currentTime + 0.2); // set up the sound to fade out within 200ms
}, false);

The first WebAudio implementations and WebKit

The Web Audio API was first implemented in Google Chrome using the webkitAudioContext prefix. We have been discussing the API on the W3C Audio Working Group and have been trying to fix some of the problems in the earlier versions of the API. In some places, doing that means that we needed to break backwards compatibility of code which targets webkitAudioContext.

There is a guide on how to port those applications to the standard API. There is also the webkitAudioContext monkeypatch available which handles some of these changes automatically, which can help to make some of the code targeting webkitAudioContext to work in the standard API.

The implementation in Firefox

In Firefox, we have implemented the standard API. If you’re a web developer interested in creating advanced audio applications on the web, it would be really helpful for you to review Porting webkitAudioContext code to standards based AudioContext to get a sense of all of the non-backwards-compatible changes made to the API through the standardization process.

We are currently hoping to release Web Audio support in Firefox 24 for desktop and Android, unless something unexpected happens that would cause us to delay the release, but you can use most parts of the API on Nightly and Aurora right now.

There are still some missing bits and pieces, including MediaStreamAudioSourceNode, MediaElementAudioSourceNode, OscillatorNode and HRTF panning for PannerNode. We’ll add support for the remaining parts of the API in the coming weeks on Nightly and Firefox Aurora.

About Ehsan Akhgari

Ehsan works on various bits and pieces in Gecko.

More articles by Ehsan Akhgari…

About Robert Nyman [Editor emeritus]

Technical Evangelist & Editor of Mozilla Hacks. Gives talks & blogs about HTML5, JavaScript & the Open Web. Robert is a strong believer in HTML5 and the Open Web and has been working since 1999 with Front End development for the web - in Sweden and in New York City. He regularly also blogs at http://robertnyman.com and loves to travel and meet people.

More articles by Robert Nyman [Editor emeritus]…


16 comments

  1. Jan Krutisch

    First of all, thanks for pulling this through, I’m really happy to see this happening!

    This might not be the appropriate place to ask this (and I will gladly take pointers), but is there some sort of roadmap for bringing this to FFOS/B2G? I didn’t find any good information on how gecko/Firefox releases will be integrated into b2g, which is probably the more relevant question, right?

    I’m asking because as everyone can imagine and also my experiments show, audio performance is a much bigger issue on mobile, and with b2g the audio data api, with its limited possibilities is the only way to do complex real time audio, so I would guess this is an API many people in the growing b2g developer community would love to have. (Especialy Games, but also music apps)

    July 9th, 2013 at 14:42

    1. Robert Nyman [Editor]

      Glad you like it!
      In general, the initial Firefox OS releases aren’t in direct connection with the normal release schedule of Firefox. Over time, we hope that they will be more in sync.

      I can’t give you an exact date/version at this time for Web Audio API support in Firefox OS, but naturally we’re planning to do it.

      July 9th, 2013 at 16:41

  2. Carl Davidá

    Don’t forget to totally break its usefulness by refusing to add support for anything in the MPEG-4 family, like you did back during your H.264 video debacle.

    July 9th, 2013 at 17:52

    1. Ehsan Akhgari

      We currently support MP3 and H.264 playback on Windows Vista+ and Android, and support for other platforms is in progress.

      July 9th, 2013 at 18:01

    2. Robert Nyman [Editor]

      And if you are interested in our stance on open standards that aren’t encumbered by licenses, I recommend reading Video, Mobile, and the Open Web.

      July 10th, 2013 at 01:48

  3. phi2x

    It would be great if you could provide a polyfill to make the Web Audio API behave like the Audio Data API.

    Because what I really need is a way to push the audio data I generate in JS at a fixed 125kHz, and let the Audio API deal with it and down/upsample automatically the result to the speakers.

    July 9th, 2013 at 23:05

  4. Clayton Gulick

    Just wanted to stop by to say thanks for this – you guys are doing amazing work and really pushing the “web as a platform” forward. Combined with WebRTC, this is going to allow me to write some pretty incredible things.

    Implementing a spec of this size and complexity couldn’t have been easy, kudos to the whole team!

    July 10th, 2013 at 09:27

    1. Robert Nyman [Editor]

      Thanks Clayton! And definitely, both WebRTC and Web Audio are features that will make the Open Web much better.

      July 10th, 2013 at 11:06

  5. Nixon

    Well done in getting past the politics of audio manipulation in the browser!

    Does the current implementation permit the ability to source the audio from microphone via getUserMedia?

    Thank you

    July 10th, 2013 at 10:10

  6. Bingo

    It’s great great news, when will you push it in enable as default config? and I am looking forward to support mp3 format on Mac either.

    July 10th, 2013 at 22:55

  7. Bozho

    You mentioned synthesis. Does it use MIDI?

    July 12th, 2013 at 00:11

    1. Jan Krutisch

      While there is a WebMidi spec in the works (see http://webaudio.github.io/web-midi-api/), this is not about midi, no. Synthesis with the Web Audio API works by connecting audio generators with filters and effects. Currently, the Firefox implementation is missing one very important building block (the OscillatorNode), but that’s coming.

      If you are okay with an example that uses the old API and is currently only running in Chrome or safari, look here: http://jsfiddle.net/halfbyte/Ta8QX/

      July 12th, 2013 at 01:01

  8. Kong Ying Jenny

    Hope so

    July 12th, 2013 at 07:03

  9. Amp

    Just for curiosity …. any updates on Background Service API?

    July 12th, 2013 at 07:30

  10. pablocubico

    Hi! Do you have any idea on when the API will be available in Firefox OS and Firefox for Android?

    July 25th, 2013 at 07:34

    1. Ehsan Akhgari

      It will be available on Firefox for Android at the same time that it’s released for desktop. For Firefox OS, we’re trying our best to release it as part of version 1.2.

      July 25th, 2013 at 14:48

Comments are closed for this article.