video – more than just a tag

This article is written by Paul Rouget, Mozilla contributor and purveyor of extraordinary Open Web demos.

Starting with Firefox 3.5, you can embed a video in a web page like an image. This means video is now a part of the document, and finally, a first class citizen of the Open Web. Like all other elements, you can use it with CSS and JavaScript. Let’s see what this all means …

The Basics

First, you need a video to play. Firefox supports the Theora codec (see here to know all media formats supported by the audio and video elements).

Add the video to your document:

You might need to add some “fallback” code if the browser doesn’t support the video tag. Just include some HTML (which could be a warning, or even some Flash) inside the video tag.


Here’s some more information about the fallback mechanism.

HTML Attributes

You can find all the available attributes here.

Some important attributes:

  • autoplay: The video will be played just after the page loads.
  • autobuffer: By default (without this attribute), the video file is not downloaded unless you click on the play button. Adding this attribute starts downloading the video just after the page loads.
  • controls: by default (without this attribute), the video doesn’t include any controls (play/pause button, volume, etc.). Use this attribute if you want the default controls.
  • height/width: The size of the video

Example:

You don’t have to add the “true” value to some of these attributes in HTML5, but it’s neater to do so. If you’re not in an XML document, you can simply write:

JavaScript API

Like any other HTML element, you have access to the video element via the Document Object Model (DOM):

var myVideo = document.getElementById("myVideo");

Once you obtain a handle to the video element, you can use the JavaScript API for video.

Here is a short list of some useful methods and properties (and see here for more of the DOM API for audio and video elements):

  • play() / pause(): Play and pause your video.
  • currentTime: The current playback time, in seconds. You can change this to seek.
  • duration: The duration of the video.
  • muted: Is the sound muted?
  • ended: Has the video ended?
  • paused: Is the video paused?
  • volume: To determine the volume, and to change it.

Example:





Events

You know how to control a video (play/pause, seek, change the volume, etc.). You have almost everything you need to create your own controls. But you need some feedback from the video, and for that, let’s see the different events you can listen to:

  • canplay: The video is ready to play
  • canplaythrough: The video is ready to play without interruption (if the download rate doesn’t change)
  • load: The video is ready to play without interruption (the video has been downloaded entirely)
  • ended: The video just ended
  • play: The video just started playing
  • pause: The video has been paused
  • seeking: The video is seeking (it can take some seconds)
  • seeked: The seeking process just finished
  • timeupdate: While the video is playing, the currentTime is updated. Every time the currentTime is updated, timeupdate is fired.

Here’s a full list of events.

For example, you can follow the percentage of the video that has just been played:

function init()
{
  var video = document.getElementById("myVideo");
  var textbox = document.getElementById("sometext");
  video.addEventListener("timeupdate", function() {
  textbox.value = Math.round(100 * (video.currentTime / video.duration)) + "%"; }

}

Showing all this in action, here’s a nice open video player using the Video API.

Now that you’re familiar with some of the broad concepts behind the Video API, let’s really delve into the video as a part of the Open Web, introducing video to CSS, SVG, and Canvas.

CSS and SVG

A video element is an HTML element. That means you can use CSS to style it.

A simple example: using the CSS Image Border rule (a new CSS 3 feature introduced in Firefox 3.5). You can view how it works on the Mozilla Developer Wiki.

And obviously, you can use it with the video tag:


One of my demos uses this very trick.

Since Firefox 3.5 provides some new snazzy new CSS features, you can do some really fantastic things. Take a look at the infamous washing machine demo, in which I subject an esteemed colleague to some rotation.

It uses some CSS rules:

And some SVG:

Because the video element is like any other HTML element, you can add some HTML content over the video itself, like I do in this demo. As you can see, there is a <div> element on top of the video (position: absolute;).

Time for a Break

Well, we’ve just seen how far we can go with the video element, both how to control it and how to style it. That’s great, and it’s powerful. I strongly encourage you to read about the new web features available in Firefox 3.5, and to think about what you can do with such features and the video element.

You can do so much with the power of the Open Web. You can compute the pixels of the video. You can, for example, try to find some shapes in the video, follow the shapes, and draw something as an attachment to these shapes. That’s what I do here! Let’s see how it actually works.

Canvas & Video

Another HTML 5 element is canvas. With this element, you can draw bitmap data (see the canvas reference, and I strongly suggest this canvas overview). But something you might not know is that you can copy the content of an <img/> element, a <canvas/> element and a <video/> element.

That’s a really important point for the video element. It gives you a way to play with the values of the pixels of the video frames.

You can do a “screenshot” of the current frame of the video in a canvas.

function screenshot() {
 var video = document.getElementById("myVideo");
 var canvas = document.getElementById("myCanvas");
 var ctx = canvas.getContext("2d");

 ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
}

You can first apply a transformation to your canvas (see the documentation). You can also copy a thumbnail of the video.

If you draw every frame in a canvas, your canvas will look like a video element. And you can draw what you want in this canvas, after drawing the frame. That’s what I do in this demo.

Once you have a video frame in your canvas, you can compute the values of the pixels.

Some things you should know if you want to compute the pixels values of a frame:

  • you can’t use this mechanism with a video from another domain.
  • you can’t use this mechanism with a video from a file:/// URL (which would be useful during the development of your web application). But you can change this behavior for testing: in about:config, change the value of “security.fileuri.strict_origin_policy” to “false”. But be very careful! editing about:config — that’s an expert feature!
  • There are two ways to display the result of your application on the top of the video:
    • use your canvas as a video (if you draw the frame every time), and then draw directly into the canvas
    • use a transparent canvas on the top of the video
  • the canvas element can be “display: none”
  • the video element can be “display: none”

About JavaScript

For the image processing, you will need to do a lot of computation. Here are some tricks:

  • copy your frame in a small canvas. If the canvas is three times smaller than the video, it means nine times fewer pixels to compute.
  • avoid recursion. In a recursion, the script engine doesn’t use the JIT optimization.
  • if you want to do a distance between colors, use the L.A.B colorspace.
  • if you want to find the center of an object, compute its centroid. See the “computeFrame” function that I use in this JavaScript snippet for my demo.
  • if the algorithm is really heavy, you can use a Worker thread, but take into account that you will need to send the content of the canvas to the thread. It’s a big array, and objects are automatically JSONified before being sent. It can take a while.

Conclusion

As you can see, you can do powerful things with the video element, the canvas element, CSS3, SVG and the new JavaScript engine. You have everything in your hands to create a completely new way to use Video on the web. It’s up to you now — upgrade the web!

References

About Arun Ranganathan

More articles by Arun Ranganathan…


17 comments

  1. Andy

    Pretty extensive coverage.

    Thank you, Paul!

    July 7th, 2009 at 20:53

  2. Alexandre

    It looks like there is no standard way to provide a few important information such as a thumbnail (to be shown before the video is loaded/played), and another quality (e.g. switch normal/HD).
    Yes, it is possible to do that with CSS/Javascript, but I think it would be nice to have a standard, semantically valid, accessible way to do so.

    July 8th, 2009 at 02:20

  3. Laurentj

    @alexandre :

    > It looks like there is no standard way to provide a few important information such as a thumbnail

    No, there is such feature described by the html5 specification: the “poster” attribute, in which you can indicate an URL of an image. However this is not implemented in Firefox 3.5 (it will be in the next version).

    http://dev.w3.org/html5/spec/Overview.html#attr-video-poster

    >and another quality

    Again, the specification allows it, with the source element http://dev.w3.org/html5/spec/Overview.html#the-source-element

    But it is not completely implemented in Firefox 3.5.

    July 8th, 2009 at 03:01

  4. Ms2ger

    Please, the syntax for boolean attributes is |controls|, |controls=””| or |controls=”controls”|, not |controls=”true”|. (Also, you typo’d |width| as |with| twice in the last example.)

    @Alexandre: For the a thumbnail, there’s the |poster=””| attribute, but it isn’t implemented yet in Firefox. Multiple versions could probably be done with multiple |source| element (||), but that doesn’t seem to be supported either.

    July 8th, 2009 at 06:00

  5. […] More here:  video – more than just a tag at hacks.mozilla.org […]

    July 8th, 2009 at 16:51

  6. @sull

    nice article.

    btw, was just curious about the poster attribute and landed on https://bugzilla.mozilla.org/show_bug.cgi?id=449156

    any idea when this will be avail in an 3.5RC ?

    thanks.

    sull

    July 8th, 2009 at 22:31

  7. Henri Sivonen

    The /> syntax on the video element in text/html makes the rest of the page be swallowed into video fallback. There needs to be an explicit end tag in text/html.

    July 8th, 2009 at 23:40

  8. Alexandre

    Laurentj: Thanks for your response!
    Great with @poster, I missed this one when I last read the spec’ :-)
    Yes, I knew about source to provide different versions (e.g. MP4/h264, Ogg/Theora), but in the case I want to provide two or more different qualities of the same container/codec, as far as I can see the first in the list will be played (I have not read in details), but there is apparently no simple way to specify the quality and/or to switch from one quality from the other. Probably that can be addressed partially by having some quality information in the “codec” part, and switch the order of the source elements in Javascript(?).

    July 9th, 2009 at 01:43

  9. Vince

    Why can’t you just provide support for BOTH Mpeg4 and Ogg Theora!? Why do you have to make it difficult? If I want to implement this tag I have to use THREE video formats to make sure it works in all browsers! Three! I may as well just stick with Flash since everyone will have the plugin for it anyway. It’s very frustrating as a solo developer who is trying to maintain a site in his free time when a great new spec comes out and everyone want to have a pissing contest over codecs that ultimately just screws us, the developers.

    Thanks Mozilla/Apple!

    July 10th, 2009 at 09:35

  10. @sull

    Vince-

    is how you could provide 3 formats. but you can also just provide whichever format you want and ignore others.

    having a default codec does not cause a developer to have more problems. however, having a default codec does help a content creator have more assurance that a video can be played by modern browsers regardless of extra plugins (that need to be properly updated with various codecs).

    July 13th, 2009 at 15:42

  11. @sull

    my comment got stripped but regardless… the point i was making is that the video tag can be used for flv or mp4 or other formats, not just ogg media. specifying an ogg file is not required.

    July 13th, 2009 at 15:46

  12. @sull

    Vince,

    I see what you mean now. I misinterpreted.

    Lack of Supporting mp4 with the native video tag in FF has to do mostly with licensing, as has been discussed here and elsewhere.

    But honestly, as a developer i don’t think the lack of native mp4 support using the video element adds much complexity to video handling – you could use sometihng like this – http://metavid.org/wiki/Mv_Embed or do your own browser detection and output accordingly. it’s no worse than the current state of media embedding.

    And it’s not a developers problem as much as it is a content creators concern… the users of the sites/pages that you build.
    And video content creators typically are used to providing multiple formats for diff platofrms and devices. Not much changing there either. Except, I hope… more Ogg media.

    Not much of an argument unless you are the ones responsible for the costs involved with integrating h.264… and with Ogg Theora improvements… the cost is not justifued compared to any differences in quality.

    July 14th, 2009 at 11:40

  13. […] 随着 Firefox 版本升级到 3.5,正式支持 HTML5,坊间对该标准的讨论就没停过。其中最显著的,<video>标签,因为对浏览器该支持哪一个视频编码而纠结不休,很是著名。 […]

    January 21st, 2010 at 08:42

  14. Komrade Killjoy

    MPlayer is for video

    Firefox is for markup

    It is inane to use firefox in lieu of MPlayer

    :P

    July 28th, 2010 at 05:38

  15. Alfredo

    Hello!

    I have searched but I have not found this video option.

    Description of situation:
    In a web page.php I have 2 DIVs.
    Insight every DIV there is a video.
    For first time I show Div id=”one” that start in auto play.
    I know its duration, so after, for example, 10 seconds I hide its DIV and show the div id=”two”.
    I use 2 javascript functions.

    Problem:
    Some times the video in div id=”one” late to be ready to play, late to start to play, but the count down of 10 seconds is started. So it will be interrupt in (10 – (late)) seconds.
    The video may be linked from youtobe or other server in different format as flv, wmv or ogg.

    Needed:
    How I can understand when the video is start to play, so after this time to start the count down?
    Is there a javascript option that give me a status = ‘true’ where the video started, so I start the count down to change the div?

    Thanks!
    Alfredo.
    Milan.

    November 21st, 2010 at 07:08

  16. […] You can analyse and manipulate video with Canvas to inject content […]

    April 25th, 2011 at 06:19

  17. […] You can analyse and manipulate video with Canvas to inject content […]

    April 25th, 2011 at 07:12

Comments are closed for this article.