Articles for July 7th, 2009

This article is written by Paul Rouget, Mozilla contributor and purveyor of extraordinary Open Web demos.

Starting with Firefox 3.5, you can embed a video in a web page like an image. This means video is now a part of the document, and finally, a first class citizen of the Open Web. Like all other elements, you can use it with CSS and JavaScript. Let’s see what this all means …

The Basics

First, you need a video to play. Firefox supports the Theora codec (see here to know all media formats supported by the audio and video elements).

Add the video to your document:

<video id="myVideo" src="myFile.ogv"/>

You might need to add some “fallback” code if the browser doesn’t support the video tag. Just include some HTML (which could be a warning, or even some Flash) inside the video tag.

<video id="myVideo" src="myFile.ogv">
<strong>Your browser is not awesome enough!</strong>
</video>

Here’s some more information about the fallback mechanism.

HTML Attributes

You can find all the available attributes here.

Some important attributes:

• autoplay: The video will be played just after the page loads.
• autobuffer: By default (without this attribute), the video file is not downloaded unless you click on the play button. Adding this attribute starts downloading the video just after the page loads.
• controls: by default (without this attribute), the video doesn’t include any controls (play/pause button, volume, etc.). Use this attribute if you want the default controls.
• height/width: The size of the video

Example:

<video id="myVideo" src="myFile.ogv" 
   autobuffer="true" controls="true"/>

You don’t have to add the “true” value to some of these attributes in HTML5, but it’s neater to do so. If you’re not in an XML document, you can simply write:

<video id="myVideo" src="myFile.ogv" autobuffer controls/>

JavaScript API

Like any other HTML element, you have access to the video element via the Document Object Model (DOM):

var myVideo = document.getElementById("myVideo");

Once you obtain a handle to the video element, you can use the JavaScript API for video.

Here is a short list of some useful methods and properties (and see here for more of the DOM API for audio and video elements):

• play() / pause(): Play and pause your video.
• currentTime: The current playback time, in seconds. You can change this to seek.
• duration: The duration of the video.
• muted: Is the sound muted?
• ended: Has the video ended?
• paused: Is the video paused?
• volume: To determine the volume, and to change it.

Example:

<button onclick="myVideo.play()">Play</button>
<button onclick="myVideo.volume = 0.5">Set Volume</button>
<button onclick="alert(myVideo.volume)">Volume?</button>

Events

You know how to control a video (play/pause, seek, change the volume, etc.). You have almost everything you need to create your own controls. But you need some feedback from the video, and for that, let’s see the different events you can listen to:

• canplay: The video is ready to play
• canplaythrough: The video is ready to play without interruption (if the download rate doesn’t change)
• load: The video is ready to play without interruption (the video has been downloaded entirely)
• ended: The video just ended
• play: The video just started playing
• pause: The video has been paused
• seeking: The video is seeking (it can take some seconds)
• seeked: The seeking process just finished
• timeupdate: While the video is playing, the currentTime is updated. Every time the currentTime is updated, timeupdate is fired.

Here’s a full list of events.

For example, you can follow the percentage of the video that has just been played:

function init() 
{
  var video = document.getElementById("myVideo");
  var textbox = document.getElementById("sometext");
  video.addEventListener("timeupdate", function() {
  textbox.value = Math.round(100 * (video.currentTime / video.duration)) + "%"; }
 
}

<video id="myVideo" src="myFile.ogv" 
            autoplay="true" onplay="init()"/>
<input id="sometext"/>

Showing all this in action, here’s a nice open video player using the Video API.

Now that you’re familiar with some of the broad concepts behind the Video API, let’s really delve into the video as a part of the Open Web, introducing video to CSS, SVG, and Canvas.

CSS and SVG

A video element is an HTML element. That means you can use CSS to style it.

A simple example: using the CSS Image Border rule (a new CSS 3 feature introduced in Firefox 3.5). You can view how it works on the Mozilla Developer Wiki.

And obviously, you can use it with the video tag:

 
<video id="myVideo" src="myFile.ogv" 
style="-moz-border-image: 
           url(tv-border.jpg) 25 31 37 31 stretch stretch; 
           border-width: 20px;"/>

One of my demos uses this very trick.

Since Firefox 3.5 provides some new snazzy new CSS features, you can do some really fantastic things. Take a look at the infamous washing machine demo, in which I subject an esteemed colleague to some rotation.

It uses some CSS rules:

• filter, clip-path.
• css transform.

And some SVG:

• SVG & foreignobject.

Because the video element is like any other HTML element, you can add some HTML content over the video itself, like I do in this demo. As you can see, there is a <div> element on top of the video (position: absolute;).

Time for a Break

Well, we’ve just seen how far we can go with the video element, both how to control it and how to style it. That’s great, and it’s powerful. I strongly encourage you to read about the new web features available in Firefox 3.5, and to think about what you can do with such features and the video element.

You can do so much with the power of the Open Web. You can compute the pixels of the video. You can, for example, try to find some shapes in the video, follow the shapes, and draw something as an attachment to these shapes. That’s what I do here! Let’s see how it actually works.

Canvas & Video

Another HTML 5 element is canvas. With this element, you can draw bitmap data (see the canvas reference, and I strongly suggest this canvas overview). But something you might not know is that you can copy the content of an <img/> element, a <canvas/> element and a <video/> element.

That’s a really important point for the video element. It gives you a way to play with the values of the pixels of the video frames.

You can do a “screenshot” of the current frame of the video in a canvas.

function screenshot() {
 var video = document.getElementById("myVideo");
 var canvas = document.getElementById("myCanvas");
 var ctx = canvas.getContext("2d");
 
 ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
}

<video id="myVideo" src="myFile.ogv" autoplay="true" with="600" height="400"/>
<canvas id="myCanvas" with="600" height="400"/>
<button onclick="screenshot()">Copy current frame to canvas</button>

You can first apply a transformation to your canvas (see the documentation). You can also copy a thumbnail of the video.

If you draw every frame in a canvas, your canvas will look like a video element. And you can draw what you want in this canvas, after drawing the frame. That’s what I do in this demo.

Once you have a video frame in your canvas, you can compute the values of the pixels.

Some things you should know if you want to compute the pixels values of a frame:

• you can’t use this mechanism with a video from another domain.
• you can’t use this mechanism with a video from a file:/// URL (which would be useful during the development of your web application). But you can change this behavior for testing: in about:config, change the value of “security.fileuri.strict_origin_policy” to “false”. But be very careful! editing about:config — that’s an expert feature!
• There are two ways to display the result of your application on the top of the video:
  • use your canvas as a video (if you draw the frame every time), and then draw directly into the canvas
  • use a transparent canvas on the top of the video
• the canvas element can be “display: none”
• the video element can be “display: none”

About JavaScript

For the image processing, you will need to do a lot of computation. Here are some tricks:

• copy your frame in a small canvas. If the canvas is three times smaller than the video, it means nine times fewer pixels to compute.
• avoid recursion. In a recursion, the script engine doesn’t use the JIT optimization.
• if you want to do a distance between colors, use the L.A.B colorspace.
• if you want to find the center of an object, compute its centroid. See the “computeFrame” function that I use in this JavaScript snippet for my demo.
• if the algorithm is really heavy, you can use a Worker thread, but take into account that you will need to send the content of the canvas to the thread. It’s a big array, and objects are automatically JSONified before being sent. It can take a while.

Conclusion

As you can see, you can do powerful things with the video element, the canvas element, CSS3, SVG and the new JavaScript engine. You have everything in your hands to create a completely new way to use Video on the web. It’s up to you now — upgrade the web!

References

• LAB Color Space – for color space distances.
• How to use threads in your JavaScript code.
• The centroid definition – good way to compute the center of an object from its pixels.
• TraceMonkey – the new Firefox JavaScript engine
• Pixel manipulation with canvas.
• The “Dynamic Content Injection” demo.
• The canvas.drawImage API – useful to inject the frame of a video into a canvas element.
• How to do transformation in your canvas.
• A good overview of the canvas API.
• Everything you need to know about canvas.
• All the new OpenWeb features in Firefox 3.5.
• Layering a div over a video.
• Transformation with CSS 3.
• Including HTML inside SVG.
• Applying SVG effects to HTML content.
• A 100% OpenVideo player.
• The reference for the video and the audio HTML5 tags.
• The Javascript API for the video and the audio tags.
• The video HTML attributes.
• The video fallback mechanism.
• Media formats supported by the audio and video elements.
• My demos.