Removing the media element ‘load’ event.

Yesterday I checked in a patch that removes support for the ‘load’ event on <video> and <audio> elements. We simply never fire it. Also, the networkState attribute is now never NETWORK_LOADED. When we’ve read to the end of the media resource, networkState changes to NETWORK_IDLE. We plan to ship this change for Firefox 3.6.

This API has been removed based on consensus from everyone who are doing HTML5 video implementations and there are lots of other options for events that Robert goes over in his post.

Changing our CSS Gradient Syntax

We landed support for a form of CSS gradients on trunk a while ago, but we got considerable feedback that our syntax — which was an incremental improvement of Webkit’s syntax, which basically exposes a standard gradient API in the most direct possible way — sucked. A bunch of people on www-style got talking and Tab Atkins produced a much better proposal. Since we haven’t shipped our syntax anywhere yet, dropping it and implementing Tab’s syntax instead was a no-brainer. So Zack Weinberg, David Baron and I did that (using a -moz prefix of course), and today it landed on trunk. It should land on the Firefox 3.6 branch shortly. It’s unfortunate to land something new like this after the last beta, but in this case, it seems like the right thing to do instead of shipping CSS gradient syntax that we know nobody wants.

We’ve never shipped the “bad” CSS gradient syntax in a final release, but it is in our first beta. We’ll be updating it before we make our final release of 3.6. Stay turned for the new syntax on hacks.

If you download and run this beta you will get updates as we make them available – once every 1-2 weeks or so. To keep up with things as they land in the betas, please follow us on the @mozhacks twitter account.

Many add-ons have not been updated for 3.6. To help us test if add-ons are compatible consider installing the Add-on compatibility reporter. It will let you run add-ons that are listed as incompatible and gives you the chance to report if add-ons appear to be working OK. This is our first attempt at crowdsourcing compatibility and if you’re feeling adventurous you should try it out.

Here’s what’s new and notable in 3.6 Beta vs. Firefox 3.5 that might be of interest to web developers:

• We now support file-based Drag and Drop so you can Drag and Drop files from your desktop in the browser. Mixed with the File API (mentioned below) this can make for a very powerful set of features for people building photo and video uploaders or anything else that wants to import data.
• @font-face now supports the WOFF format.
• We support multiple background images for background-image.
• You can now specify the background size via -moz-background-size
• We now specify a very powerful pointer-events property that lets you control whether a particular element can receive events.
• You can now specify an algorithm to render images via the image-rendering CSS property.
• Poster images now work for video elements.
• Support for a new geolocation feature that will return an approximate address for your location.
• Support for orientation events.
• …and a lot of other great stuff. For the full list, have a look at the Firefox 3.6 for developers page which has everything on it.

Here’s the list of things that we will support by 3.6 final, but aren’t in this beta:

• Support for CSS gradients is in this beta, but we’re changing it due to negative feedback on the current syntax.
• We will support the <input type=”file” multiple> so you can upload more than one file from the same file upload control.
• We have support in 3.6 for a new version of the File API draft spec but that spec is out of date and our docs aren’t complete. There’s an IDL entry for the FileRequest object, but it’s likely to be renamed to FileReader based on feedback. We’ll try to keep people updated as we release updated betas.

Route 66 by Caveman 92223

At Mozilla we’ve been talking recently about how important the web has become to everything around us. The web – and the Internet it’s built on – has become the defining computing platform for this century. And most of that has happened because of web developers, and the freedom they have enjoyed.

Firefox is one of the most important tools for web developers. Firebug combined with Firefox’s strong support for standards means that most web developers are building for Firefox first and then porting to other browsers later.

But even with developers using Firefox for development there hasn’t been an easy way for those of us at the Mozilla project to let you know about what’s going on at Mozilla – what we’re putting in Firefox, what we’re doing to bring the web forward and what we’re doing for web developers. (Although the hacks weblog is the first attempt at that.) And conversely there isn’t an obvious way for individual web developers to give Mozilla structured feedback about what’s important to them or what issues they might be having.

That’s why we’re working on the Mozilla Developer Network. It will serve two purposes:

1. To provide you with information about what’s going on at Mozilla and around the web.
2. To give you the chance to give us structured feedback and become part of the Mozilla Community.

Our first step is to do a quick survey. If we can get 5,000 responses to the survey, we’ll release the aggregate results.

The survey itself asks you about how you work, what you think of Firefox and should only take a few minutes to complete.

Thanks! We’ll have more on the Mozilla Developer Network as things progress.

If you’ve ever been editing a page and wanted to know what a particular font looked like without having to upload files to a web server and update CSS (so tedious, that!) then this demo is for you.

He’s come up with a demo that shows what’s possible when you’ve got downloadable fonts, drag and drop and editable content. (If you want to know more about drag and drop we suggest you read his excellent overview of using drag and drop to do file uploading.)

From Ryan’s description:

Font dragr is an experimental web app that uses HTML5 & CSS3 to create a useful standalone web based application for testing custom fonts, once you visit it for the first time you don’t need to be online to use it after the initial visit. It allows you, in Firefox 3.6+, to drag and drop font files from your file system into the drop area. The browser will then create a data URL encoded copy to use in the page and render the content in the dropped font.

You can either read the full description, which contains a lot of useful technical information about how the demo works, or you can view the demo below. Either way, it’s nice to see the excellent HTML5 support in Firefox 3.6 coming together with everything else we’ve added to bring a lot of new capabilities to web developers and users.

Thanks, Ryan!

This post is from Jonathan Kew and John Daggett. He’s supplied a 5 minute video that shows some of the features on the fly. If you’re a total font nerd and you enjoy a soothing British accent, you might want to watch it.

Using @font-face allows web designers a wide palette of font choices and with commercial font vendors supporting the WOFF font format, the set of font choices should improve even more. So the next step is clearly to try and make better use of features available in these fonts.

For many years, “smart” font formats such as OpenType and AAT have provided font designers ways of including a rich set of variations in their fonts, from ligatures and swashes to small caps and tabular figures. The OpenType specification describes these features, identifying each with a unique feature tag. But these have typically only been available to those using professional publishing applications such as Adobe InDesign. Firefox currently renders using font defaults; it would be much more interesting to provide web authors with a way of controlling these font features via CSS.

Håkon Wium Lie of Opera, based on discussions with Tal Leming and Adam Twardoch, proposed extending the CSS ‘font-variant’ property to include values for font features. Mozilla is actively working on a new proposal along these lines. This is a fairly big addition to CSS, so it will most likely involve some complex discussions about how best to support these features.

Experimental Implementation

As part of this effort, Jonathan Kew has been working on porting a portion of the Pango library for use with handling complex scripts and to enable the use of font features on all supported platforms. He currently has an experimental build that uses a special CSS property to associate a list of OpenType features with specific elements in a page. This is not the proposed format, it simply provides a better way of discussing possible sets of font-variant properties and the OpenType feature settings to which they should map.

.altstyles {
  /* format: feature-tag=[0,1] with 0 to disable, 1 to enable */
  -moz-font-feature-opentype: "dlig=1,ss01=1";
}

The feature setting string above implies rendering with discretionary ligatures (dlig) and the first set of stylistic alternates (ss01). An example using Jack Usine’s MEgalopolis Extra:

This font makes extensive use of OpenType features; the homepage of the font contains a sample rendering that uses many of these features. Below is a rendering of the same sample written in HTML with OpenType features enabled:

Tabular Figures for Numerical Alignment

OpenType features also enable better control over alignment. When numbers are listed in tabular form, proportional forms often result in the misalignment of digits across rows, making the list harder to scan. With tabular alignment enabled, individual digits are rendered using a fixed width so that digits align across rows:

Automatic Fractions

Automatic fractions are also possible with OpenType features, note the inline use of fractional forms in the recipe ingredient list below:

Language Sensitivity

OpenType also includes support for features on a per-language basis. This is important for rendering text correctly in some languages. For example, Turkish uses both a dotted-i and a dotless-i, so the distinction needs to be preserved when rendering ligatures or small caps. Below is the same text in English and Turkish, with both default and language-sensitive renderings shown for the Turkish text:

Historical Text

The way text is rendered constantly evolves; glyphs once in common use often morph into other forms, making the historical forms distinct from their modern forms. Below is the text of an old Massachusetts Bay Colony law, taken from a book in the Daniel R. Coquillette Rare Book Room of the Boston College Law Library.

Original scanned image:

Below is the same text rendered in HTML using the Fell Types revival fonts by Igino Marini with OpenType features enabled. Note the ‘ct’ ligature and the contextual form of the ‘s’:

More resources

Font feature support in CSS proposal
OpenType font feature list
Fell Types revival fonts by Igino Marini

In Firefox 3.5 we included support for linking to TrueType and OpenType fonts. In Firefox 3.6 we’re including support for a new font format – the Web Open Font Format, or WOFF. This format has two main advantages over raw TrueType or OpenType fonts.

1. It is compressed, which means that you will typically see much smaller download sizes compared with raw TrueType or OpenType fonts.
2. It contains information that allows you to see where the font came from – without DRM or labeling for a specific domain – which means it has support from a large number of font creators and font foundries.

The WOFF format originated from a collabaration between the font designers Erik van Blokland and Tal Leming with help from Mozilla’s Jonathan Kew. Each had proposed their own format and WOFF represents a melding of these different proposals. The format itself is intended to be a simple repackaging of OpenType or TrueType font data, it doesn’t introduce any new behavior, alter the @font-face linking mechanism or affect the way fonts are rendered. Many font vendors have expressed support for this new format so the hope is this will open up a wider range of font options for web designers.

Details on Differences between TrueType, OpenType and WOFF

First, compression is part of the WOFF format so web authors can optimize the size of fonts used on their pages. The compression format is lossless, the uncompressed font data will match that of the original OpenType or TrueType font, so the way the font renders will be the same as the original. Similar compression can be achieved using general HTTP compression but because compression is part of the WOFF format, it’s simpler for authors to use, especially in situations where access to server configuration is not possible.

Second, the format includes optional metadata so that a font vendor can tag their fonts with information related to font usage. This metadata doesn’t affect how fonts are loaded but tools can use this information to identify the source of a given font, so that those interested in the design of a given page can track down the fonts used on that page. Fonts in WOFF format are compressed but are not encrypted, the format should not be viewed as a “secure” format by those looking for a mechanism to strictly regulate and control font use.

Note: until Firefox 3.6 ships, users can test the use of WOFF fonts with Firefox nightly builds.

Examples

Below is a simple example that shows how to construct an @font-face rule that links to a WOFF font. To support browsers that only support direct linking to OpenType and TrueType fonts, the ’src’ descriptor lists the WOFF font first along with a format hint (”woff”), followed by the TrueType version:

/* Gentium (SIL International) */
 
@font-face {
  font-family: GentiumTest;
  src: url(fonts/GenR102.woff) format("woff"),
       url(fonts/GenR102.ttf) format("truetype");
}
 
body {
  font-family: GentiumTest, Times, Times New Roman, serif;
}

Structured this way, browsers that support the WOFF format will download the WOFF file. Other browsers that support @font-face but don’t yet support the WOFF format will use the TrueType version. (Note: IE support is a bit trickier, as discussed below).　As WOFF is adopted more widely the need to include links to multiple font formats will diminish.

Other examples below demostrate the use of WOFF formatted fonts but each example has been constructed so that it will work in any browser that supports @font-face, including Internet Explorer.

A font family with multiple faces

Using a Postscript CFF font

African Language Display

Below is an example of how downloadable fonts can be used to render languages for which font support is usually lacking. The example shows the UN Declaration of Human Rights, translated into two African languages, and how these render with default browser fonts vs. with a downloadable font suited for rendering these languages.

Note that in one of these examples that the font size goes from a 3.1MB TTF to a 1MB WOFF font and in the other from a 172KB TTF to an 80KB WOFF file.

Another Postscript CFF font

An example in Japanese

Working With Other Browsers

Firefox 3.6 will be the first shipping browser to support the WOFF format so it’s important to construct @font-face rules that work with browsers lacking WOFF support. One thing that helps greatly with this is the use of format hints to indicate the format of font data before it’s downloaded; browsers that don’t recognize a given format simply skip data in a format they don’t support.

Internet Explorer, including IE8, only supports the EOT font format and only implements a subset of the @font-face rule descriptors. This makes creating cross-platform @font-face rules that work with IE especially tricky. One solution is to make different rules for IE:

@font-face {
  font-family: GentiumTest;
  src: url(fonts/GenR102.eot);  /* for IE */
}
 
@font-face {
  font-family: GentiumTest;
  /* Works only in WOFF-enabled browsers */
  src: url(fonts/GenR102.woff) format("woff"); 
}

One minor downside of this is that IE doesn’t understand format hints and doesn’t parse @font-face URL’s correctly, it treats format hints as part of the URL, so web authors using the @font-face rules above will see the following in their access logs:

GET /fonts/GenR102.eot HTTP/1.1" 200 303536
GET /fonts/GenR102.woff)%20format(%22woff%22) HTTP/1.1" 404 335

IE successfully pulls down and uses the EOT version of the font but also tries to pull down the WOFF font with the format hint included in the URL. This fails and doesn’t affecting page rendering but it wastes valuable server resources. For more discussion, see Paul Irish’s blog post for one interesting workaround.

Another problem is that IE currently tries to download all fonts on the page, whether they are used or not. That makes site-wide stylesheets containing all fonts used on site pages difficult, since IE will always try to download all fonts defined in @font-face rules, wasting lots of server bandwidth.

Further Resources

Documentation

Latest draft WOFF specification
Original blog post on using @font-face
CSS3 Fonts working draft
MDC @font-face documentation

Tools

Jonathan Kew’s sample encoding/decoding code
woffTools – tools for examining and validating WOFF files
FontTools/TTX – Python library and tool for manipulating font data
Web-based font subsetting tool

General @font-face Examples

CSS @ Ten: The Next Big Thing
Example layout using Graublau Sans
Examples of Interesting Web Typography
The Elements of Typographic Style Applied to the Web

Font Resources

Font Squirrel
10 Great Free Fonts for @font-face
40 Excellent Free Fonts by Smashing Magazine

modern carpentry rides the html5 canvas wave

Thomas says:

I was challenged at work to create something that “floats naturally”. After a while of confusion in my pursuit of “natural floating” or whatever, I came up with the idea that I needed a tool to investigate my confusion. This project is a result of my trying to build that tool.

It was initially built as a Flash/Flex application, which you can see here. Later on, however, I became interested in HTML5 and the Canvas tag and decided to port the application to JavaScript and HTML (with the help of jQuery, and also Processing.js, of course…). The transition went fairly well, and I ended up with a JS/HTML version of the application that I actually think is cooler than the original Flash version.

In the end, I came up with both something that “floats naturally” and also a serendipitous encounter with the simplicity of working with only HTML and JavaScript, which has been very rewarding.

One new feature that we’re including as part of Firefox 3.6 is support for web pages to access machine orientation information if it’s available. As you can see from the demo above you can use it to figure out if the machine is moving and what direction it’s facing.

Using the API is very simple. All you have to do is add a simple event listener:

window.addEventListener("MozOrientation", function(e) {
                        /* 3 values: e.x, e.y, e.z */
                        }, true);

Originally built as something that we would include for our upcoming mobile browser release, we’ve made it available on desktop systems as well. Many modern Macbooks and Thinkpads contain devices and drivers that expose this information. We’ve added support for Linux, Macs and some Thinkpads where drivers and devices are available. (Note, on some Macbooks detect the orientation information backwards – we’re working on that.)

You can find more information on two posts by Doug Turner and an update on documentation for orientation from Eric Shepherd.

Paul Rouget has this and some other demos up in one of his posts.

This post is targeted at web developers and gives a quick overview of the three kinds of attacks that CSP helps to mitigate and also gives some quick examples so developers can get a sense of how it will work for them.

In case you don’t know what our Content Security Policy code is – and based on anecdotal evidence a lot of people don’t – it’s a set of easy to use tools that allow a web site owner to tell the browser where it should or should not load resources from. In particular it aims to prevent three different classes of common attacks we see on the web today: cross-site scripting, clickjacking and packet sniffing attacks.

Cross-site scripting attacks are largely the result of a mistake made on backend web servers where someone fails to escape data that’s submitted by users. When that happens it’s possible to inject a tag to load JavaScript code from another web site. That code could be harmless but it could also contain something dangerous, like malware. What CSP does is make it possible for a web site author, via HTTP headers, to specify what types of scripts can be loaded and from where. For developers who are setting a policy, it adds a layer of protection where even if they make a mistake it is likely to be mitigated by this additional layer of policy.

Clickjacking attacks are where someone embeds a page into a transparent iframe and “steals” user clicks to activate something dangerous. One particular attack allows a browser to be turned into a remote surveillance device. CSP includes the ability for a page to tell the browser that it never wants to be ever included in an iframe.

And last is the ability for a web site to tell the browser that it never wants resources from that page to be loaded over unencrypted HTTP. Banking and other commerce sites will find this particularly useful.

CSP is very powerful and flexible, allowing you to specify whether or not you want to load different kinds of media, different kinds of script methods, css, can be used to set up loading only from specific other hosts and a large number of other things. It’s meant to be very easy to set up for simple cases but will scale up to pretty complex infrastructure where different resources might be spread out over a large number of machines.

Here are four examples that show common use cases. Each of these examples is a header that’s delivered as a header over HTTP and it affects how the page is rendered.

A site wants all of its content to come from its own domain:

X-Content-Security-Policy: allow 'self'

Example 2: An auction site wants to be able to load images from anywhere, plugin content from a list of trusted media providers and a CDN network and scripts only from its server hosting sanitized JavaScript:

X-Content-Security-Policy: allow 'self'; img-src *; \
                           object-src media1.com media2.com *.cdn.com; \
                           script-src trustedscripts.example.com

Example 3: Server administrators want to deny all third-party scripts for the site, and a given project group also wants to disallow media from other sites (header provided by sysadmins and header provided by project group are both present):

X-Content-Security-Policy: allow *; script-src 'self'
X-Content-Security-Policy: allow *; script-src 'self'; media-src 'self';

Example 4: An online payments site wants to ensure that all of the content in its page is loaded over SSL to prevent attackers from eavesdropping on requests for insecure content:

X-Content-Security-Policy: allow https://*:443

The implementation isn’t quite complete yet, but it’s pretty close. There’s more information on the demo page for CSP, read the overview or read the spec itself.

I’ve been spending a lot of time lately thinking about 3D in the browser. It’s to the point that I have to blog about it, and try to get it out of my head and onto the web where it belongs. Part of the reason I’m writing this tonight is that my colleague, Cathy Leung, has just put up a demo so awesome I’m compelled to share it (video here):

She put the video together after this article was written about her work to make 3D accessible to web developers via C3DL. Cathy has been working with a team of students at Seneca College for the past few years, focused on an experimental 3D technology created by Mozilla. Her work proves how valuable 3D in the browser can be. I think a lot of people misunderstand its significance: “Doom in the browser?” Sure, but it’s way more than that. There is no shortage of tea pots on the web these days, as various browser vendors start to show-off early releases of WebGL enabled builds. However, Mozilla has been at this game a long time. I know because we’ve been using their stuff as long as they’ve been building it. And it’s really reached a new level now.

As Mozilla pushes WebGL further, and Cathy continues her work on C3DL, I’m focused on another project to harness the power of 3D in the browser. Together with Al MacDonald, I’m leading a Mozilla Foundation project to complete the work begun by John Resig to port the Processing language to JavaScript (aka processing.js). The processing.js port, while unfinished, is already amazing. Imagine if we took eight students, finished it, and added 3D support from C3DL? I know, exactly.

Part of what makes me so excited about all this work is the deep intersection of things I love. I’ve long been a fan of the work of Processing creators, Ben Fry and Casey Reas, every since they were part of the Aesthetics and Computation Group. I’ve always thought that their work belonged on the web instead of on the desktop. At the same time, my own work with Mozilla and the open web has proven to me the need for richer ways of working with things like open data. The web is at an interesting point in its evolution. We now take for granted that it can do what was once only possible via desktop apps. Looking forward I can already see a time when the web will also be the proper medium of artists, designers, data visualizers, and others pushing the limits of the second and third dimensions. I’m thrilled to be a small part of this effort now.

If you’d like to get involved with any of the work discussed above, please do get in touch with me. The most exciting aspect of everything I’ve told you remains that it’s all community developed, from Mozilla to C3DL to processing to processing.js. That’s how you build the software stack of the future.

We’re running a short survey to find out what the phrase “open web” means to web developers. What does that phrase mean to you?

http://rypple.com/blizzard/web

It’s an anonymous survey and will probably take you less than a minute to respond to.

Since it’s a survey that only allows you to use about 200 characters you’ll have to be short and simple. And we don’t want to poison people by offering suggestions – we’re curious about people’s honest responses.

We’ll post a summary if we get enough results.

Thanks!

The 1.1 release is largely an improvement to the Theora encoder. This post will attempt to give people a high-level overview of the changes and what they mean to web developers and people who are thinking of deploying Theora to support HTML5 video. Theora is an important technology to web developers – it’s the only competitive codec that currently complies with the W3C patent policy.

Here’s a quick list of important things that have changed in this release. We’ll go into more detail on each of these items.

• Video quality between Theora 1.0 and Theora 1.1 has been improved.
• Rate control for live streaming now works well.
• A two-pass mode has been added to the encoder that can create rate controlled videos with very predictable bandwidth requirements.
• CPU usage during encoding is much more consistent.
• Decoder performance has been improved.

Video quality between Theora 1.0 and Theora 1.1 has been improved.

One issue that people had with the Theora 1.0 encoder was that it produced video that appeared fuzzy. The 1.1 improvements are clear in these two images provided by Monty, one of the Xiph Developers. Open each of these images in new tabs and flip between them. You can really see the difference.

This was also very visible at the edges of text. Here’s an example taken from one of our Firefox 3.5 promotional videos. The first is with the 1.0 encoder (9.0MB) and the second is with the 1.1 encoder (8.2MB). You will notice that not only are the edges more defined but there’s a lot less noise in the area around the edges of the text. Once again, if you open them in tabs and flip between them you can see the difference.

Note that the original video is nearly 17MB. That was done largely to get the text crisp. With these changes we can likely use a much lower-bandwidth version of the video, probably as small as 9.9MB. That’s a pretty big difference.

Note that we’re talking about an improvement of quality at the same video bitrate. This means that we’re either able to produce higher quality videos at the same file size or we’re able to reduce the file size and keep the same quality – either way it’s a big win.

Rate control for live streaming now works well.

Before describing this change, something important must be described. This is the difference between videos encoded with a variable bitrate (VBR) and a constant bitrate (CBR).

In variable bitrate encoding the amount of data that’s required to represent the difference between two frames in a video is allowed to grow. This happens most often when shifting from a scene where there isn’t much movement to a scene where there’s a lot of motion. You could easily go from requiring 40Kb/sec to 400Kb/sec because the entire background moves.

In constant bitrate encoding the amount of data that you’re allowed to use to represent a change from one frame to the next is pinned at some maximum value. If you’ve got a low maximum value and there’s a set of frames that requires a lot of bits to represent the changes from one to the next you will need to sacrifice something in order to stay inside of that maximum value. Very often it’s some amount of video quality or the encoder will start dropping frames in order to keep under the watermark.

This leads to a pretty simple rule: If you want the highest quality video possible, you should be using variable rate encoding. This means that when you’re encoding a video you should be using quality settings (0-99, low/medium/high, 1-10) instead of picking bitrates (60Kb/sec, 200Kb/sec.) For most use cases on the web VBR-encoded videos actually work very well because users are allowed to buffer quite a bit of video out ahead of their current position so these bursts of data don’t affect the user’s experience.

But there are some use cases where having a constant bitrate is very important. These include:

1. Live, low delay streaming over HTTP with a lot of clients.
2. Streaming large files where a large read-ahead buffer is not desired.
3. Situations where large bursts of data result in large bursts in CPU to handle them.

For live, low-delay streaming over HTTP it’s important to realize what happens when there’s a sudden burst of data to handle. HTTP runs over TCP. In TCP it takes a while for a connection to increase its bandwidth. (And by “a while” I mean “not that long” but it’s long enough to affect the low latency connection that we want for this use case. This is why many low-latency applications don’t use TCP. But we’re talking about delivering video over HTTP.) If you’ve got a big burst of data and the TCP window takes a long time to open up you start building up a big send buffer on the server. (And remember in this use case you’ve got a lot of clients connected!) That requires a lot of memory to hold the send buffers for each client. What happens then is that servers will start closing connections en masse because it needs to save memory or because it thinks that the client has become somehow unreachable. This is made worse by the fact that even if the connection scales up and then scales back down it re-settles at the low rate and the process has to be repeated. The user’s experience is that the video stream stops and restarts or just stops working altogether when the server hangs up. The solution? Using a constant rate that doesn’t require the TCP window to open up suddenly and doesn’t require large send buffers for each client.

For the use case where you’re streaming large files it might not be reasonable for the client to cache a lot of data. You also might be serving up a lot of data to a lot of clients and you might want to avoid the large send buffer problem as well, just for different reasons.

And for the last use case where you’re in a CPU-constrained environment the bursting nature of variable bitrate videos means it often takes a large bursts of CPU to handle those bursts. While CPUs do burst up faster than TCP does, you might be talking to constrained processors (think mobile) or you might be serving up files near HD-sized content, which CPUs often struggle to decode.

In any case there are a number of use cases for constant bitrate encoding. Back to the question of what’s improved in Theora 1.1.

In Theora 1.0 the rate controlled encoding mode was very very broken. This resulted in two things:

1. People trying to do live streaming ran into problems.
2. People who used rate controlled settings to compare overall Theora quality to the quality of other encoders saw worse results than the format actually represented.

The first issue is clear – it was broken, it should be fixed. And it has been. The new encoder does a pretty good job of maintaining bitrates, changes quality on the fly, drops frames and even includes a “soft-target” mode so that bitrates can fluctuate a little bit to maintain quality while occasionally breaking the bandwidth rules.

The encoder also has a wonderful new piece of functionality that people will find very useful. It’s now possible to specify a maximum rate ceiling for video encoding while also specifying a minimum quality floor. What this means is that the encoder will try and maintain very crisp video frames within rate constraints. This means that it will aggressively drop frames instead of creating frame deltas that are fuzzy or low-quality. While this might sound like a poor trade-off it’s actually very useful. If you’re showing a live video of a presentation you usually want a crisp video of the slides and having a lower frame update rate is very acceptable.

The second issue that was caused by the bad rate control in Theora 1.0 is an issue of marketing. People would often use the encoder with the fixed bitrate mode instead of the quality mode and dismiss the results as a reflection of the format instead of problems with the encoder. We hope that people find better results with the new encoder.

A two-pass mode has been added to the encoder that can create rate controlled videos with very predictable bandwidth requirements.

In addition to fixing the single pass rate controlled encoder in 1.1 a two-pass encoding option has been added. This means that if you are transcoding a file (as opposed to doing a live stream) you can create a very consistent bitrate in a file if you want. This is because the encoder can look ahead in the stream to allocate bits efficiently. Monty from Xiph made a graph that shows one example of the bitrate in a file with one pass and two pass.

Above: graph of quantizer choice (which roughly corresponds to the encoding quality) when using default two-pass bitrate management as opposed to one-pass (with –soft-target) when encoding [the Matrix movie clip] at 300kbps. Both encodes resulted in the same bitrate. The quality of a one-pass encode varies considerably as the encoder has no way to plan ahead.

CPU usage during encoding is much more consistent.

People who were doing live streaming often saw huge spikes in CPU usage during high-motion events. This has been fixed and now CPU usage is much more consistent during single pass rate constrained encoding making it much easier to live stream video.

Decoder performance has been improved.

And last but not least the decoder has been made faster during the 1.1 release. How much faster depends quite a bit on the clip, but people are reporting that the new encoder is anywhere from 1.5-2x faster than the 1.0 of release of libtheora.

Coming soon to a product near you.

This release is a library release. It’s not a product in itself, but is instead something that other products include. So over the next days and weeks we’ll see other products pick up and start using this as part of their releases.

Enjoy!

A port of Puls to WebGL:

Escher-Droste effect in WebGL:

Metatunnel by FRequency:

(via Mark Steele)

Since my post on Friday, we landed a few fixes to improve our WebGL implementation and to fix a couple of bugs we discovered on Friday. I’m looking forward to seeing what people do with WebGL, and how it can be useful on the web right now. For example, EA/Maxis recently added COLLADA export of Creature designs to their popular game Spore, and they have a Sporepedia where players can see others’ creations. Right now, those previews are just as images. With WebGL, they could be fully 3D, even animated.

Over the weekend I’ve put together this example, which uses WebGL to render an exported Spore creature, and let the user rotate the 3D model to view it from different angles. For those who want to try it out, you’ll need a recent Firefox nightly (one from today, September 21, or newer), and with one preference flipped as described in this post.

I’ll be working to update the very basic “getting started” demos from the GL ES 2 book that I ported to Canvas 3D as well, so that those who are interested in experimenting can have some good basic code to look at. They’re not updated yet, but they should be in the next day or two.

For those of you on Windows who don’t have an up to date OpenGL driver, or don’t have the possibility of getting one (e.g. many common Intel graphics cards doesn’t have OpenGL drivers), you can enable software rendering by downloading a Windows build of the Mesa software OpenGL implementation. It won’t be fast, but it should be enough for you to get an idea of what’s going on, and to play with some of the demos. To use it, download webgl-mesa-751.zip and extract it somewhere on your computer. It has a single file, OSMESA32.DLL, that you need to tell Firefox where to find: open up about:config, and set the preference webgl.osmesalib to the path of OSMESA32.DLL. If you extracted it to “C:\temp”, you would put in “C:\temp\osmesa32.dll” in the pref. Then, flip webgl.software_rendering to true, and you should be good to go.

(Mac OS X users shouldn’t need to bother with software rendering, since Apple already provides a high quality OpenGL implementation, and Linux users should be ok as long as they have recent OpenGL drivers installed.)

what’s your single favorite thing about hacks.mozilla.org?

The survey is totally anonymous and takes 20 seconds or so. It’s limited, much like twitter, to a very short response. If you have ideas on improvement there’s room for that as well so please, respond away! If we get enough good feedback we’ll post results here as well.

Thanks!

This post has also been cross-posted to Felipe’s personal blog.

Multitouch on Firefox from Felipe on Vimeo.

I’ve been anxious to demonstrate the progress on our multi-touch support for Firefox, and this video showcases some possible interactions and use cases for what web pages and webapps can do with a multi-touch device.

We’re working on exposing the multi-touch data from the system to regular web pages through DOM Events, and all of these demos are built on top of that. They are simple HTML pages that receive events for each touch point and use them to build a custom multi-touch experience.

We’re also adding CSS support to detect when you’re running on an touchscreen device. Using the pseudo-selector :-moz-system-metric(touch-enabled) you can apply specific styles for your page if it’s being viewed on a touchscreen device. That, along with physical CSS units (cm or in), makes it possible to adjust your webapp for a touchscreen experience.

Firefox 3.6 will include the CSS property, but is unlikely to include the DOM events described below.

Here is an example of what the API looks like for now. We have three new DOM events (MozTouchDown, MozTouchMove and MozTouchRelease), which are similar to mouse events, except that they have a new attribute called streamId that can uniquely identify the same finger being tracked in a series of MozTouch events. The following snippet is the code for the first demo where we move independent <div>s under the X/Y position of each touch point.

var assignedFingers = {};
var lastused = 0;
 
function touchMove(event) {
    var divId;
    if (lastused < = 4)
        return;
 
    if (assignedFingers[event.streamId]) {
        divId = assignedFingers[event.streamId];
    }
    else {
        divId = "trackingdiv" + (++lastused);
        assignedFingers[event.streamId] = divId;
    }
 
    document.getElementById(divId).style.left = event.clientX + 'px';
    document.getElementById(divId).style.top  = event.clientY + 'px';
}
 
document.addEventListener("MozTouchMove", touchMove, false);
document.addEventListener("MozTouchRelease",
                          function () { lastused--; }, false);

On the wiki page you can see code snippets for the other demos. Leave any comments regarding the demos or the API on my weblog post. We really welcome feedback and hope to start some good discussion on this area. Hopefully as touch devices (mobile and notebooks) are getting more and more popular we’ll see new and creative ways to use touch and multitouch on the web.

The demo uses jQuery and drives a simple carousel that shows the current chapter as the video plays. It also shows subtitles as the video plays.

One really great, and somewhat subtle thing about this experiment? You can select the subtitles. They are just text.

Web developers will be interested in a number of features that are new in Firefox 3.6 Alpha 1:

• The TraceMonkey JavaScript engine has continued to get faster.
• We’ve made a huge number of improvements to overall DOM and element layout performance. In some cases we’re much, much faster. We’ll cover details on those in a later post.
• The compositor landing has made it possible to fix a large number of interactions between web content, CSS and plugins. We’ll be talking about this in a later post as well.
• We now support the -moz-background-size CSS property which lets you set the size of background images.
• We now support CSS Gradients.
• We now support multiple background images.
• We now support the rem unit as a CSS unit.
• image-rendering is supported for images, background images, videos and canvases.
• We now send a reorder event to embedded frames and iframes when their document is loaded.
• We’ve removed the getBoxObjectFor() method. It was non-standard and exposed all kinds of non-standard stuff to the web.
• We now send a hashchange event to a page whenever the URI part after the # changes.
• We now have Geolocation address support for user-readable position information.
• We now support the complete attribute on document.readystate.

You can keep track of this list and other features for XUL and add-ons developers on the Firefox 3.6 for developers page on developer.mozilla.org

Unlike the year that passed between Firefox 3 and Firefox 3.5, we expect that this 3.6 release will be released in a small number of months. Our main focus for the 3.6 release will be end-user perceived performance, TraceMonkey and DOM performance and new web developer features.

Enjoy and test away!

HTML5 Canvas and Audio Experiment

Please comment on their cool experiment at their blog: HTML5 Canvas Experiment.