using HTML5 video with fallbacks to other formats

The Mozilla Support Project and support.mozilla.com (SUMO for short) is an open and volunteer powered community that helps over 3 million Firefox users a week get support and help with their favorite browser. The Firefox support community maintains a knowledge base with articles in over 30 languages and works directly with users through our support forums or live chat service. They’ve put together the following demonstration of how to use open video and Flash-based video at the same time to provide embedded videos to users regardless of their browser. This demo article was written by Laura Thomson, Cheng Wang and Eric Cooper

Note: This article shows how to add video objects to a page using JavaScript. For most pages we suggest that you read the article that contains information on how to use the video tag to provide simple markup fallbacks. Markup-based fallbacks are much more elegant than JavaScript solutions and are generally recommended for use on the web.

One of the challenges to open video adoption on the web is making sure that online video still performs well on browsers that don’t currently support open video.  Rather than asking users with these browsers to download the video file and use a separate viewer, the new <video> tag degrades gracefully to allow web developers to provide a good experience for everyone.  As Firefox 3.5 upgrades the web, users in this transitional period can be shown open video or video using an existing plugin in an entirely seamless way depending on what their browser supports.

At SUMO, we use this system to provide screencasts of problem-solving steps such as in the article on how to make Firefox the default browser.

If you visit the page using Firefox 3.5, or another browser with open video support, and click on the “Watch a video of these instructions” link, you get a screencast using an Ogg-formatted file.  If you visit with a browser that does not support <video> you get the exact same user experience using an open source .flv player and the same video encoded in the .flv format, or in some cases using the SWF Flash format.  These alternate viewing methods use the virtually ubiquitous Adobe Flash plugin which is one of the most common ways to show video on the web.

The code works as follows.

In the page which contains the screencasts, we include some JavaScript.   Excerpts from this code follow, but you can see or check out the complete listing from Mozilla SVN.

The code begins by setting up an object to represent the player:

Screencasts.Player = {
width: 640,
height: 480,
thumbnails: [],
priority: { 'ogg': 1, 'swf': 2, 'flv': 3 },
handlers: { 'swf': 'Swf', 'flv': 'Flash', 'ogg': 'Ogg' },
properNouns: { 'swf': 'Flash', 'flv': 'Flash', 'ogg': 'Ogg Theora' },
canUseVideo: false,
isThumb: false,
thumbWidth: 160,
thumbHeight: 120
};

We allocate a priority to each of the possible video formats.  You’ll notice we also have the 'canUseVideo' attribute, which defaults to false.

Later on in the code, we test the user’s browser to see if it is video-capable:

var detect = document.createElement('video');
if (typeof detect.canPlayType === 'function' &&
    detect.canPlayType('video/ogg;codecs="theora,vorbis"') == 'probably' ) {
      Screencasts.Player.canUseVideo = true;
      Screencasts.Player.priority.ogg =
          Screencasts.Player.priority.flv + 2
}

If we can create a video element and it indicates that it can play the Ogg Theora format we set canUseVideo to true and increase the priority of the Ogg file to be greater than the priority of the .flv file. (Note that you could also detect if the browser could play .mp4 files to support Safari out of the box.)

Finally, we use the priority to select which file is actually played, by iterating through the list of files to find the one that has the highest priority:

for (var x=0; x < file.length; x++) {
  if (!best ) {
    best = file[x];
  } else {
    if (this.priority[best] < this.priority[file[x]])
      best = file[x];
  }
}

With these parts in place, the browser displays only the highest priority video and in a format that it can handle.

If you want to learn more about the Mozilla Support Project or get involved in helping Firefox users, check out their guide to getting started.

Resources

* Note: To view this demo using Safari 4 on Mac OSX, you will need to add Ogg support to Quicktime using the Xiph Quicktime plugin available from http://www.xiph.org/quicktime/download.html

About Laura Thomson

More articles by Laura Thomson…


37 comments

  1. Sevenspade

    Later on in the code, we test the user’s browser to see if it is video-capable:
    if (typeof document.createElement('video').play === 'function' ) { Screencasts.Player.canUseVideo = true; Screencasts.Player.priority.ogg = Screencasts.Player.priority.flv + 2 }

    Creating and throwing away a video element—no possible reuse; the node is left floating nowhere—just to test if video elements are supported? Seriously? Why not just test for the existence of HTMLVideoElement?

    June 18th, 2009 at 21:53

  2. Kroc Camen

    But but but, … *quivery lip* I’ve already solved this — without JavaScript.

    http://camendesign.com/code/video_for_everybody

    I don’t have Flash installed on my system, _please_ Mozilla, please adopt HTML5 across all of your blogs and sites. I can’t see your announcements and it seems counter-intuiitive to the work you guys are doing making HTML5 video great!

    June 19th, 2009 at 00:36

  3. […] Using Open Video while staying compatible with old browsers: http://hacks.mozilla.org/2009/06/html5-video-fallbacks/ […]

    June 19th, 2009 at 01:38

  4. John Drinkwater

    If you want fallback without javascript being needed, have a look at http://camendesign.com/code/video_for_everybody

    June 19th, 2009 at 01:59

  5. Da Scritch

    This “solution” is completely bad, as you use another technology outside HTML and Flash to read it. You should complyonly to the smallest factor, that is HTML, and perfectly working.

    This is a bad as using swfobject.js to embed flash

    Please remove it before someone else use it

    June 19th, 2009 at 06:48

  6. CWW

    Actually the problems with the http://camendesign.com/code/video_for_everybody solution are that 1) Screencasts are uploaded by users of the site so the option needs to exist to do without the ogv (or mp4) since we lack serverside conversion capability. 2) Many people don’t have Flash 10. (Especially on support where we get users with all sorts of outdated software). So the one required file format needs to be H264 or lower. 3) The most popular screencasting software (Jing) actually makes SWF files from which the video feed can’t be extracted.

    Kroc Camen’s solution is great (dare I say better) if you have full control over your media and are able to make and host three different file formats (ogv, mp4, h264) or leave Flash 9 users in the dust. In this case, we have neither luxury but site survey shows that less than a couple hundred of the 3 million site users a week have JS turned off (and those are likely volunteers not users seeking help) so a JS solution fits. (Yes, I should have put this in the blog post itself but this is what happens when a blog post is written by someone — that’s me — who didn’t write the code and only was only shown a demo.)

    June 19th, 2009 at 07:51

  7. pd

    What a dumpster full of the usual propaganda spam this post and the several others about “open” video are.

    If you were really to “open” up video to the browser you would have finished the work done to implement support for *ANY* codec installed on a user’s system.

    DirectShow support got to the point of a working binary release several months ago. I downloaded it and was happily able to play anything from XviD to MPG. Here’s the URL:

    http://pearce.org.nz/firefox-directshow-video-2008_10_24.win32.zip

    and here’s a sample page:

    http://pearce.org.nz/player/

    and here’s the bug where Mozilla tries to justify it’s approach:

    https://bugzilla.mozilla.org/show_bug.cgi?id=435339

    Isn’t it about time that “open” meant open to anybody? Not open to Mozilla ideology?

    When will the world wide web finally become the web of the people, not the web the browser vendors choose to present to us?

    Is Mozilla’s underlying ethos one of a digital communist dictatorship parading as a ‘technocracy’ or a benevolent digital socialist as you would have us believe?

    I bet $5 the mozilla censors will not publish this comment!

    June 19th, 2009 at 08:05

  8. Derek

    Kroc your website is fubar’d. Stop spamming it till you can fix it.

    June 19th, 2009 at 10:24

  9. Justin Dolske

    One potential issue with the “video for everybody” method is that the content inside the video tag (the so-called fallback content) is still parsed and created by the browser. So, say, if your flash video plays automatically, you’d hear it and the OGG video playing.

    See https://bugzilla.mozilla.org/show_bug.cgi?id=487398

    June 19th, 2009 at 12:04

  10. David Baron

    Isn’t the HTML video element designed so that you can put the fallback for old browsers as the contents of the video element? Why is JavaScript necessary here?

    June 19th, 2009 at 16:54

  11. Kaiwai

    @Derek

    How about you stop using web browsers that suck – Kroc’s website works with both Firefox, Safari, Chrome and Opera. You’re obviously using Internet Explorer – do us all a favour and fine a dark hole to bury yourself in.

    June 19th, 2009 at 18:05

  12. John Drinkwater

    pd,
    Allowing formats outside of a narrow set can damage the Web because then with network effects, each Web device has to cater for those additional codecs, containers, etc…
    I can’t see the iPhone, the CrunchPad, the G1, the N97 etc supporting the full range of codecs that would be needed if the full gamut of encodings were encouraged on the net.
    So please, stop believing that by being Open, Mozilla is being closed-minded, as they are not.

    CWW,
    Only .MP4 and .ogv files are needed for Krocs solution (just two), and with an extension like Firefogg, users can upload Theora videos from their browser…
    Please though, can you remove this post from hacks.mozilla.org, it sets a bad precedent.
    If all you need is a demo for video, flash, and fallback, then do that – without JavaScript.

    Regards,

    June 20th, 2009 at 04:39

  13. Christopher Blizzard

    @John – I appreciate what you’re saying (and the letter you signed?) with Kroc. I’ve asked Kroc to come up with another post that we can post here that describes the non-JavaScript way to post video with fallbacks. I’ll be happy to cross-link from this post to that one if he decides to write it.

    June 20th, 2009 at 05:53

  14. Kroc Camen

    @Christopher Blizzard

    Hello Chris! Thanks for the reply. Of all the obvious places to actually link to the letter so that Mozilla engineers would get the chance to see it, I forgot about here. So if people are wondering what all the controversy is about, read here:

    http://camendesign.com/blog/letter_to_mozilla_re_video

    Chris, I’ve replied to the mail you sent me to propose what we could do about a replacement article. Hope you agree!

    June 20th, 2009 at 12:28

  15. […] livres o que é importante para combater a ditadura dos plugins neste campo) Muito interessante este exemplo com fallback para outros […]

    June 20th, 2009 at 14:30

  16. William Lacy

    The HTML5 OGG-video playback, test page on OGGTV, only works on the special Opera browser with native OGG-multimedia playback. I have to wait until all of the Mozilla javascript bugs are worked-out, to place a full-set of HTML5 OGG extensions, to the framework of the site.

    The test videos, will be changed later, with a newer updated OGGTV.

    June 20th, 2009 at 15:19

  17. CorporateUser

    Many companies do filter JavaScript or de-activate it in the settings for security reasons. So does my employer, a bank.

    So users in such environments will not be very happy with this proposed implementation – which to me also looks like moving your car on a truck instead on its own tires. But you god-like developers might know this much better than me, being only a simple minded user of the web ;-)

    So again: For all of us corporate users, please do not propagate JavaScript where it is not absolutely necessary.

    Thank you very much from a humble Firefox evangelist in a IE dominated large company.

    June 21st, 2009 at 00:00

  18. Eevee

    Augh. Please, no. I’m so tired of sites that use convoluted Javascript to accomplish things that plain HTML does simply. It doesn’t work with NoScript (and usually has no helpful fallback), it’s harder to examine, and it’s harder to change client-side. This is not what the Web is supposed to be about, Mozilla.

    June 21st, 2009 at 02:27

  19. Mike

    We intend to use Kroc Camen’s implementation on our Oracle Reference Library site (http://psoug.org) because it’s the way it SHOULD be done. Javascript is NOT required, nor should it be.

    A big “Thank You” to Kroc for posting the correct way to embed video using HTML5. For those interested in doing it the right way (instead of the FUBAR way, Kroc’s page is here: http://camendesign.com/code/video_for_everybody

    -Mike

    June 21st, 2009 at 10:35

  20. Nick

    I would prefer the following approach: Use only the video element and fallback content in your markup. Then run one JS script on page load, that converts video elements to the corresponding object elements for browsers that don’t support HTML5 video.

    The only users who wouldn’t see embedded video although they technically could are the ones that have Flash enabled, but Javascript disabled. And that’s a negligibly small part of most audiences.

    June 21st, 2009 at 11:35

  21. Lars Trieloff

    I have been working on an approach that uses the video markup directly in HTML, but will employ a Javascript wrapper library to provide a Flash fallback that still implements most of the HTML5 mediaelement API. This makes it possible to start, stop and resume video playback using one, standardized cross-browser API, even when the Flash fallback is being used. Thanks to gradual improvement the HTML still contains the video element, so that mashups and aggregation continue to work.

    There is a blog post describing my approach here: http://gettingsoftware.posterous.com/html5flash-using-html5-video-and-audio-right and the source code is available from Github.

    June 21st, 2009 at 14:06

  22. sxpert

    @PD
    you owe us all $5 ;)

    June 21st, 2009 at 23:58

  23. […] 原文地址:using HTML5 video with fallbacks to other formats 系列地址:颠覆网络35天 […]

    June 22nd, 2009 at 00:07

  24. Steven Rowat

    I agree with the prevailing sentiment in the comments so far: no, please, no Javascript! This is an important cross-roads of the web, I believe: an open, standard, easy-to-use HTML video is essential NOW to avoid another decade of corporate obfuscation. ANd AFAIK the audio tag (which is my primary interest) is in the same boat as the video, web-wide, and so I ask Mozilla to do the same with it: true work out a true HTML 5 implementation with best possible fallback; without Javascript.

    June 22nd, 2009 at 10:12

  25. Kroc Camen

    @Mike @Lars Trieloff

    What of RSS readers that don’t execute JavaScript?

    A common API is definitely what’s needed Lars, I think everybody would benefit if we could combine Video for Everybody with HTML5Flash so that JavaScript isn’t required to see the video (including Flash users), but a JavaScript API allows people to do more advanced stuff and interact with the page.

    June 22nd, 2009 at 10:15

  26. […] a previous post on this blog we talked about using JavaScript to create video elements on the fly. While that was a good use […]

    June 22nd, 2009 at 23:25

  27. Lars Trieloff

    @Steven: no Javascript for basic things like video playback – agreed. But if you want to build a non-linear-video editor on the web, create your own controls, etc. you need Javascript. This is where my solution is targeted at. I am also looking forward to incorporate Video for Everyone in future examples, so that the easy things work easily.

    @Kroc: RSS Readers is a different topic, because
    – they have shorter upgrade cycles than browsers, so they can soon implement html5 video natively
    – web-based feed readers use Javascript anyway, they could even use html5flash to implement html5 video now
    – there is MediaRSS, which is similar to a video element with multiple source elements

    From my point of view, the real concern is getting having the video element in the markup, so that user agents know what’s going on and to have a fallback for the non-FF/non-Chrome/non-Safari population. People with Javascript disabled are less common in my usecase, and disabling Javascript often correlates with disabling Flash and Plugins, which would make most fallbacks unworkable.

    June 23rd, 2009 at 08:24

  28. Steven Rowat

    @Lars
    “…no Javascript for basic things like video playback.”
    Good, thank you; it remains only to ask about what is included in ‘basic things’:

    My need, which I believe will be shared by others for a variety of reasons and uses-cases, is this: can the standard audio (and video) tag controllers that appear on-screen, without Javascript, be modified by CSS for color, transparency, size, and control features?

    I’m doing a site, for example, that has an individual web page for each of >100 poems. Each poem is unique, has a unique image background, and the page containing it is meant to be experienced as a whole: including, if an audio controller is added, different needs for the controller color, transparency, and sizes and types of controls available. Current Windows Media, QuickTime, and Linux-based controllers lack some or all of the basic color, size, transparency and controls configurability, and what remains is also different on different OS’s and browsers. WM and QT controllers appear so intrusive and out-of-place in many of the poems that I gave up attempting to use them, waiting instead for HTML5 audio/video tags, where I intend another attempt — one where I hope to be able to set a consistent controller across browsers and OSs.

    So, my question is: will this sort of configuration of the controller now be possible — and without Javascript?

    June 23rd, 2009 at 11:02

  29. Lars Trieloff

    @Steven – no, the controllers cannot by styled using CSS. This is where my library would come to play (you can have a look at the source code, it is not very to do)

    The spec says:

    The controls attribute is a boolean attribute. If present, it indicates that the author has not provided a scripted controller and would like the user agent to provide its own set of controls.

    If the attribute is present, or if scripting is disabled for the media element, then the user agent should expose a user interface to the user. This user interface should include features to begin playback, pause playback, seek to an arbitrary position in the content (if the content supports arbitrary seeking), change the volume, and show the media content in manners more suitable to the user (e.g. full-screen video or in an independent resizable window). Other controls may also be made available.

    This also lists all the “basic stuff”: play/pause/seek/volume

    June 23rd, 2009 at 12:10

  30. Steven Rowat

    @Lars
    Thank you, this is very helpful, and I’ll look at your code.

    I’m using PHP otherwise to generate the HTML code, so it would be great for me if that could cover what CSS can’t do; but I assume that’s not possible here since only the Javascript or HTML read by the browser can apply in a client-side issue like this. Is this correct?

    June 23rd, 2009 at 14:20

  31. […] Here’s some more information about the fallback mechanism. […]

    July 7th, 2009 at 20:04

  32. […] posts that demonstrate how to play flv files in a <video> […]

    July 8th, 2009 at 03:22

  33. […] open letter railing against the evils of javascript in manipulating html5 structs. Thankfully they all lived happily ever after [corrected original article], although surely a sign of the difficulties involved & […]

    July 10th, 2009 at 01:06

  34. Fred K

    I know it’s a bit late in the game, but I have two questions:
    1) Why should it be necessary for developers -big or small- to create/get two different versions of the same video file, to cover as wide an audience as possible? (And that is hoping that Microsoft doesn’t decide to push their own media format and excluding others.) At least two. Three, if Flash is added into the mix. It feels quite contrary to the spirit of HTML 5, and in my view it’ll probably hamper its uptake rate. In my view, if I publish an mp4 file (for example) using the element, that file should be viewable by users that have or decide to install the appropriate browser plugin. Same goes obviously if I publish an ogg file. What we’re seeing now is two sides pushing their separate codecs, disallowing the other side’s playable formats, and MS waiting in the wings. Not good. Mozilla/Opera and Apple/Google need to settle this thing quickly. Then MS can just jump on the train – no?

    2) Apparently the file extension .ogv makes an ogg video unplayable in Firefox. That’s weird, because when using ffmpeg2theora to convert files to ogg video (I’m on a Mac, ffmpeg2theora is the recommended converter), it automatically makes them *.ogv* files. And since this information is virtually non-existent (that .ogv files are unplayable in the main browser that requires ogg files for HTML 5 video) it takes a great deal of digging to understand what the f*** is going on. If you’re going to allow only one file format, make sure that the information is abundantly clear about this, and readily available. If for nothing else, to save me from having a(nother) bleeding ulcer… (I’ll put this to the ffmpeg2 people, but you (Mozilla) and Opera need to start making this info visible.)

    Thanks

    September 27th, 2009 at 18:36

  35. Daniel D

    Although I am more late on this, I can’t help but second the opinion mentioned in Fred’s first question:
    I too see no need to have every frickin file encoded twice. This is pure waste of drive space!
    This may seem irrelevant for the hobbyist content provider, but if you’re — say — an academic institution that’s going to provide lecture recordings in a VOD service, this adds up pretty quick.
    And what for?
    Encode H.264, pack it up in an MPEG 4 container and (through with Flash-fallback) you could serve it to virtually anyone — well, except Firefox users, that is.

    Let me get this straight:
    is intended to obfuscate Flash for web video, right?
    Then, why on earth would one implement it in a way that does not handle codec mismatches gracefully???
    If — for what ever ideological reasons — Mozilla chooses not to support MPEG 4 in the first place and goes for Ogg/Theora (what virtually no-one uses, but hey it’s open) instead, then why not at least handle MPEG 4 content gracefully?
    Like — for example — ask the (plug-in) system if anything handles that type and if not, pass on to the fallback?

    I strongly doubt, that it would have been much of an effort, to implement it that way…

    I’m serious guys:
    Battles for codec-superiority* are nothing you should fight on your users backs!

    (* then again, from the technical point of view Theora _isn’t_ superior to H.264 — if I’m not mistaken; the only advantage it offers is being open)

    Plus:
    Add me to the list of “Don’t encourage the use of JavaScript for this!”-people.

    Thanks

    November 2nd, 2009 at 03:12

  36. Roger Watson

    I’ve gone to Kroc Camen’s site just now using the iPhone and the demo video doesn’t play on the iPhone.
    Additionally, I’ve used his method with regards to getting video to play on the iPad and the video doesn’t play on the iPad either (actually the play and pause controls don’t show up (they are there but for some reason invisible) but if you press on the screen where you know the play button should be the video will play.
    Furthermore, if you run Safari on a pc one must press the pause button to play the video and press the play button to pause it (the video).
    C’mon!
    What a croc, Kroc!

    April 24th, 2010 at 10:04

  37. Kelly

    So far, all implementations of the video tag in HTML5 have taken (in net speak) ages to load before playing.

    I visit a site on Safari on a Mac and then in Firefox and do the same on Windows.

    In every case (so far) the html5 video takes anywhere from 10 to 25(!) seconds before playing. Too much time in this world.

    At least Flash and even QuickTime start playing within 5 seconds. You can just pause and wait and seem to be under more control than when you wait for 10 seconds before anything happens.

    All I care about is getting the content delivered quickly and easily to the audience and thats what 100% of all users want. No, we developers are not users, not even when we’re off the job.. it’s a different mindset.

    Unless you’re at a movie-trailer site, then videos should start playing within 5 seconds or the content has failed.

    June 17th, 2010 at 09:16

Comments are closed for this article.