Mozilla

HTML5 audio and audio sprites – this should be simple

As we’re having a HTML5 Audio developer derby this month, I thought it fun to play with audio again. And I found it sadly enough pretty frustrating.

One thing I proposed in a lot of talks is using the idea of CSS sprites and apply them to HTML5 audio. You’ll get the same benefits – loading one file in one HTTP request instead of many, avoiding failure as files might not get loaded and so on.

To test this out I wrote the following small demo using the awesome Music Non Stop by Kraftwerk.

Clicking the different buttons should play the part of the music file and nothing more. This works fine in Firefox, Chrome and Opera on my computer here. Safari, however, fails to preload the audio and the setting of the current time is off. The code is simple enough that this should work:

<div id="buttons"></div>
<audio preload controls>
  <source src="boing-boomchack-peng.mp3" type="audio/mp3"></source>
  <source src="boing-boomchack-peng.ogg" type="audio/ogg"></source>
</audio>
// get the audio element and the buttons container
// define a sprite object with the names and the start and end times
// of the different sounds.
var a = document.querySelector('audio'),
    buttoncontainer = document.querySelector('#buttons'),
    audiosprite = {
      'all': [ 0, 5 ],
      'boing': [ 0, 1.3 ],
      'boomtchack': [ 2, 2.5 ],
      'peng': [ 4, 5 ]
    },
    end = 0;
 
// when the audio data is loaded, create the buttons
// this way non-HTML5 browsers don't get any buttons
a.addEventListener('loadeddata', function(ev) {
  for (var i in audiosprite) {
    buttoncontainer.innerHTML += '<button onclick="play('' +
                                  i + '')">' + i + '</button>';
  }
}, false);
 
// If the time of the file playing is updated, compare it
// to the current end time and stop playing when this one
// is reached
a.addEventListener('timeupdate', function(ev) {
  if (a.currentTime > end) {
    a.pause();
  }
},false);
 
// Play the current audio sprite by setting the currentTime
function play(sound) {
  if ( audiosprite[sound] ) {
    a.currentTime = audiosprite[sound][0];
    end = audiosprite[sound][1];
    a.play();
  }
}

Now, this is nothing new, Remy Sharp wrote about audio sprites in 2010 and lamented especially the buggy support in iOS (audio won’t load at all until you activate it with a touch – something that sounds horribly like the “click to active” Flash has on IE).

Other issues are looping and latency of HTML5 audio. As reported by Robert O’Callahan there is a work-around by cloning the audio element before playing it (with an incredibly annoying test) and this fix has been used in the Gladius HTML5 game engine.

All in all it seems HTML5 audio still needs a lot of work which is why a lot of Games released lately under the banner of HTML5 use Flash audio or no audio at all. This is sad and needs fixing.

Interestingly enough there are some great projects that you could be part of. Are we playing yet? by Soundcloud and others for example is a test suite for audio support in browsers. You can write own tests on GitHub and report results to the browser makers.

The jPlayer team has a great HTML5 Media Event Inspector showing just how many of the HTML5 media events are supported in your current browser.

If you want to be safe, you can use SoundManager 2 by Scott Schiller to have an API that uses HTML5 when possible and falls back to Flash when the browser doesn’t have any support. It also fixes a few issues for you.

Speaking of Scott Schiller, he continually gives good insight on the state of audio. There is a 51 minute video of his article on 24 ways “Probably, Maybe, No: The State of HTML5 Audio“.

A shorter and more recent talk on the same subject is also available:

All in all it would be interesting to hear what you think of the state of HTML5 audio:

  • Did the companies that heralded HTML5 as the end of plugins drop the ball?
  • Is it really sensible to have an API that returns probably or maybe or ” when you ask it if the browser can play a certain type of media?
  • What could be done to work around these issues?

Let’s re-ignite the discussion on HTML5 audio, after all we need it for the future of messaging in the browser and telephony, too.

Oh and another thing. Of course there is the Audio Data API of Firefox and the web audio proposal from Webkit available but getting those running in mobile devices will be a much bigger change. If you want to know more about those and libraries to work around their differences, there is a great overview post available on Happyworm.

6 comments

Comments are now closed.

  1. BOSSoNe wrote on April 6th, 2012 at 15:57:

    Thank you for the tip !
    I’ve used it to create a virtual piano http://www.b1project.com/tests/virtualPiano.html and that does the job :)

  2. Yves Van Goethem wrote on April 11th, 2012 at 07:11:

    Chris, thanks for mentioning “AreWePlayingYet?”

    Concerning audio sprites I think it would make a good candidate for a test.
    If anyone has to time to contribute to it, feel free to do so :)

  3. K wrote on June 11th, 2012 at 08:15:

    Couldn’t you also use fragments to specify playback range, or would that cause multiple downloads? Example:

    src=”audio.webm” // all

    src=”audio.webm#t=0,1.3″ // boing

    src=”audio.webm#t=2,2.5″ // boomchack

    src=”audio.webm#t=4,5″ // Peng

    1. Niels Gregersen wrote on July 22nd, 2012 at 14:50:

      regarding:

      src=”audio.webm” // all

      src=”audio.webm#t=0,1.3″ // boing

      src=”audio.webm#t=2,2.5″ // boomchack

      src=”audio.webm#t=4,5″ // Peng

      I don’t think it works in Safari

  4. Javier Franco wrote on March 1st, 2013 at 04:48:

    Hi,
    Thanks for the author for this really usefull post.
    One doubt, how do I set up the end of the sprite? in which object is end?
    I’m coding this into a library, and the audio start OK, but the end never happend.

    In my play function I have:

    sprite.node.addEventListener(“canplaythrough”,function(){
    this.currentTime =sound.start;
    end = sound.end;
    this.play();
    },false);

    sound has the end and the start of the sound and the id of the sprite, so when I find the sound and the sprite, I play like thats.

    Thank in advanced

  5. Johnny wrote on April 2nd, 2013 at 22:00:

    So how can we alter this to play in safari? I’ve read and re-read remy’s blog about ios fixes, but I’m so new to this that I can’t figure out what to do ;(

    I’ve got your demo working with dynamically added audio tags pulling sprite info from an array, but it just does NOT work with safari as you predicted. Can anyone shed some light on this that isn’t so cryptic? ;(

Comments are closed for this article.