Mozilla

theora 1.1 is released – what you should know

Less than a year after the release of Theora 1.0, the wonderful people at Xiph have released Theora 1.1. The 1.1 release is a software-only release of the Theora encoder and decoder. It does not include any changes to the Theora format. Existing Theora videos should continue to play with the new decoder and the new Theora encoder generates bitstreams that will work in existing players that can play Theora content.

The 1.1 release is largely an improvement to the Theora encoder. This post will attempt to give people a high-level overview of the changes and what they mean to web developers and people who are thinking of deploying Theora to support HTML5 video. Theora is an important technology to web developers – it’s the only competitive codec that currently complies with the W3C patent policy.

Here’s a quick list of important things that have changed in this release. We’ll go into more detail on each of these items.

  • Video quality between Theora 1.0 and Theora 1.1 has been improved.
  • Rate control for live streaming now works well.
  • A two-pass mode has been added to the encoder that can create rate controlled videos with very predictable bandwidth requirements.
  • CPU usage during encoding is much more consistent.
  • Decoder performance has been improved.

Video quality between Theora 1.0 and Theora 1.1 has been improved.

One issue that people had with the Theora 1.0 encoder was that it produced video that appeared fuzzy. The 1.1 improvements are clear in these two images provided by Monty, one of the Xiph Developers. Open each of these images in new tabs and flip between them. You can really see the difference.

This was also very visible at the edges of text. Here’s an example taken from one of our Firefox 3.5 promotional videos. The first is with the 1.0 encoder (9.0MB) and the second is with the 1.1 encoder (8.2MB). You will notice that not only are the edges more defined but there’s a lot less noise in the area around the edges of the text. Once again, if you open them in tabs and flip between them you can see the difference.

Note that the original video is nearly 17MB. That was done largely to get the text crisp. With these changes we can likely use a much lower-bandwidth version of the video, probably as small as 9.9MB. That’s a pretty big difference.

Note that we’re talking about an improvement of quality at the same video bitrate. This means that we’re either able to produce higher quality videos at the same file size or we’re able to reduce the file size and keep the same quality – either way it’s a big win.

Rate control for live streaming now works well.

Before describing this change, something important must be described. This is the difference between videos encoded with a variable bitrate (VBR) and a constant bitrate (CBR).

In variable bitrate encoding the amount of data that’s required to represent the difference between two frames in a video is allowed to grow. This happens most often when shifting from a scene where there isn’t much movement to a scene where there’s a lot of motion. You could easily go from requiring 40Kb/sec to 400Kb/sec because the entire background moves.

In constant bitrate encoding the amount of data that you’re allowed to use to represent a change from one frame to the next is pinned at some maximum value. If you’ve got a low maximum value and there’s a set of frames that requires a lot of bits to represent the changes from one to the next you will need to sacrifice something in order to stay inside of that maximum value. Very often it’s some amount of video quality or the encoder will start dropping frames in order to keep under the watermark.

This leads to a pretty simple rule: If you want the highest quality video possible, you should be using variable rate encoding. This means that when you’re encoding a video you should be using quality settings (0-99, low/medium/high, 1-10) instead of picking bitrates (60Kb/sec, 200Kb/sec.) For most use cases on the web VBR-encoded videos actually work very well because users are allowed to buffer quite a bit of video out ahead of their current position so these bursts of data don’t affect the user’s experience.

But there are some use cases where having a constant bitrate is very important. These include:

  1. Live, low delay streaming over HTTP with a lot of clients.
  2. Streaming large files where a large read-ahead buffer is not desired.
  3. Situations where large bursts of data result in large bursts in CPU to handle them.

For live, low-delay streaming over HTTP it’s important to realize what happens when there’s a sudden burst of data to handle. HTTP runs over TCP. In TCP it takes a while for a connection to increase its bandwidth. (And by “a while” I mean “not that long” but it’s long enough to affect the low latency connection that we want for this use case. This is why many low-latency applications don’t use TCP. But we’re talking about delivering video over HTTP.) If you’ve got a big burst of data and the TCP window takes a long time to open up you start building up a big send buffer on the server. (And remember in this use case you’ve got a lot of clients connected!) That requires a lot of memory to hold the send buffers for each client. What happens then is that servers will start closing connections en masse because it needs to save memory or because it thinks that the client has become somehow unreachable. This is made worse by the fact that even if the connection scales up and then scales back down it re-settles at the low rate and the process has to be repeated. The user’s experience is that the video stream stops and restarts or just stops working altogether when the server hangs up. The solution? Using a constant rate that doesn’t require the TCP window to open up suddenly and doesn’t require large send buffers for each client.

For the use case where you’re streaming large files it might not be reasonable for the client to cache a lot of data. You also might be serving up a lot of data to a lot of clients and you might want to avoid the large send buffer problem as well, just for different reasons.

And for the last use case where you’re in a CPU-constrained environment the bursting nature of variable bitrate videos means it often takes a large bursts of CPU to handle those bursts. While CPUs do burst up faster than TCP does, you might be talking to constrained processors (think mobile) or you might be serving up files near HD-sized content, which CPUs often struggle to decode.

In any case there are a number of use cases for constant bitrate encoding. Back to the question of what’s improved in Theora 1.1.

In Theora 1.0 the rate controlled encoding mode was very very broken. This resulted in two things:

  1. People trying to do live streaming ran into problems.
  2. People who used rate controlled settings to compare overall Theora quality to the quality of other encoders saw worse results than the format actually represented.

The first issue is clear – it was broken, it should be fixed. And it has been. The new encoder does a pretty good job of maintaining bitrates, changes quality on the fly, drops frames and even includes a “soft-target” mode so that bitrates can fluctuate a little bit to maintain quality while occasionally breaking the bandwidth rules.

The encoder also has a wonderful new piece of functionality that people will find very useful. It’s now possible to specify a maximum rate ceiling for video encoding while also specifying a minimum quality floor. What this means is that the encoder will try and maintain very crisp video frames within rate constraints. This means that it will aggressively drop frames instead of creating frame deltas that are fuzzy or low-quality. While this might sound like a poor trade-off it’s actually very useful. If you’re showing a live video of a presentation you usually want a crisp video of the slides and having a lower frame update rate is very acceptable.

The second issue that was caused by the bad rate control in Theora 1.0 is an issue of marketing. People would often use the encoder with the fixed bitrate mode instead of the quality mode and dismiss the results as a reflection of the format instead of problems with the encoder. We hope that people find better results with the new encoder.

A two-pass mode has been added to the encoder that can create rate controlled videos with very predictable bandwidth requirements.

In addition to fixing the single pass rate controlled encoder in 1.1 a two-pass encoding option has been added. This means that if you are transcoding a file (as opposed to doing a live stream) you can create a very consistent bitrate in a file if you want. This is because the encoder can look ahead in the stream to allocate bits efficiently. Monty from Xiph made a graph that shows one example of the bitrate in a file with one pass and two pass.

Above: graph of quantizer choice (which roughly corresponds to the encoding quality) when using default two-pass bitrate management as opposed to one-pass (with –soft-target) when encoding [the Matrix movie clip] at 300kbps. Both encodes resulted in the same bitrate. The quality of a one-pass encode varies considerably as the encoder has no way to plan ahead.

CPU usage during encoding is much more consistent.

People who were doing live streaming often saw huge spikes in CPU usage during high-motion events. This has been fixed and now CPU usage is much more consistent during single pass rate constrained encoding making it much easier to live stream video.

Decoder performance has been improved.

And last but not least the decoder has been made faster during the 1.1 release. How much faster depends quite a bit on the clip, but people are reporting that the new encoder is anywhere from 1.5-2x faster than the 1.0 of release of libtheora.

Coming soon to a product near you.

This release is a library release. It’s not a product in itself, but is instead something that other products include. So over the next days and weeks we’ll see other products pick up and start using this as part of their releases.

Enjoy!

39 comments

Comments are now closed.

  1. Kurt (supernova_00) wrote on September 25th, 2009 at 10:53:

    What is the bug report for Firefox to upgrade to this encoder?

  2. Gerv wrote on September 25th, 2009 at 11:35:

    Will the 1.1 decoder be in Firefox 3.6? :-)

    Gerv

  3. John Dowdell wrote on September 25th, 2009 at 11:38:

    Hi Chris, has there been any thought to releasing a set of plugins so that people in other browsers can also view these videos? Or will the onus remain on content developers to multiply-encode?

    tx, jd/adobe

  4. F1LT3R wrote on September 25th, 2009 at 13:12:

    Blown away by these improvements. Better quality video at a lower rates means someone has been working very, very hard. It was clearly worth the effort.

  5. dave wrote on September 25th, 2009 at 14:01:

    @jd/abode: Firefox, Opera and Chrome already play these videos natively (well Opera only has test builds so far) and Safari works with the XiphQT Quicktime component and IE can use ChromeFrame.

    @ Christopher Blizzard: I think there’s a subtle mistake where you say “if you are transcoding a file (as opposed to doing a live stream) you can create a very consistent bitrate”. Actually the bitrate will vary, what the 2-pass mode lets you do is get constant quality at a specified total size (and therefore specific *average* bitrate), but the instantaneous bitrate will rise and fall as necessary. It’s the live streams where you want the consistent bitrates.

  6. Blair McBride wrote on September 25th, 2009 at 20:18:

    I’m continually impressed with the awesome work Xiph do with Theora – another open source success story.

  7. Drazick wrote on September 26th, 2009 at 01:50:

    When will the 1.1 Decoder incorporated into Firefox?
    Will it happen with FF 3.6?

  8. Asa Dotzler wrote on September 26th, 2009 at 11:28:

    The changes to the encoder are the biggest improvement here in terms of video quality. You don’t have to wait for Firefox to incorporate the new decoder to see huge improvements in videos that are encoded with 1.1.

  9. Steve wrote on September 26th, 2009 at 11:37:

    When will the 1.1 Decoder incorporated into Firefox?
    Will it happen with FF 3.6?

    I don’t think there are really and changes to the decoder:

    * It does not include any changes to the Theora format.
    * the new Theora encoder generates bitstreams that will work in existing players that can play Theora content.
    * The 1.1 release is largely an improvement to the Theora encoder.

  10. marius wrote on September 26th, 2009 at 11:40:

    afaik Firefox and other browsers DON’T have to uprade the decoder, because the movies encoded with the new version of theora are 100% compatible with the old ones.

    An upgraded decoder would only be useful if it somehow uses less cpu to decode or brings assembly optimizations for some architectures …that kind of stuff… otherwise the decoder already built in will have no problems decoding any movie.

  11. Matthew Gregan wrote on September 26th, 2009 at 12:57:

    I’m working on importing the Theora 1.1 decoder into trunk right now (bug 507554). I’m not sure if it’ll make Firefox 3.6.

  12. Pingback from Theora le format de compression libre sort sa version 1.1 | Philippe Scoffoni on September 26th, 2009 at 14:34:

    [...] Vous pouvez retrouver une description très détaillées de ces nouveautés sur cet article. [...]

  13. Pingback from Osss.cn 开源社区 » Theora 1.1(Thusnelda)稳定版发布 on September 26th, 2009 at 17:52:

    [...] “我们很高兴宣布新的libtheora的稳定版本发布,该版本参考和实现了Xiph.org基金会的Theora视频格式。此次发布的版本为1.1,代号“Thusnelda”,吸收了过去几年里我们所做的所有编译器改进,有些代码甚至可以追溯到2003年。相比1.0解码器,这个版本的速度提升了很多,而且有很多的改进。”更多变更信息,点击查看Mozilla的文章。 [...]

  14. Ben Waggoner wrote on September 26th, 2009 at 20:31:

    I did a couple of scenario-speciifc tests comparing Theora 1.1 with the current versions of x264 and the VC-1 Encoder SDK, testing both offline VBR and real-time CBR encoding.

    http://cid-bee3c9ac9541c85b.skydrive.live.com/browse.aspx/.Public/Theora%5E_1.1

    Theora is defintely improved, but I see a lot of basis pattern throughout these samples. Theora would be well-served by a postprocessing filter. And the 1-pass CBR looks to need more refinement before it can produce good quality with more challenging content.

  15. John S. Gook wrote on September 26th, 2009 at 21:10:

    Too bad real technologies like MPEG-4 Part 10 AVC has been able to do this since forever, at half the rates that even this “enhanced” version of Thoera requires. What’s more, it doesn’t randomly break down because AVC was designed by real engineers and not stuck up kiddies trying to “stick the man.”

  16. mmmh wrote on September 27th, 2009 at 00:54:

    That video is plain delicious. Great quality.

    And thanks for this informative post. :)

  17. Dot wrote on September 27th, 2009 at 04:36:

    To make Firefogg use 1.1 stable, can I just replace ffmpeg2theora with the trunk version? Or should I wait for a Firefogg release?

  18. Pingback from Theora 1.1 mejora la calidad y arregla problemas con el streaming | Bitelia on September 28th, 2009 at 00:04:

    [...] notas de la publicación encontramos una lista de cambios completa, aunque quizás los chicos de hacks.mozilla.org son algo más [...]

  19. Raju Bitter wrote on September 28th, 2009 at 01:03:

    Excellent, good news for longing for high-quality open video!

  20. Pingback from Theora 1.1 Released « Wikimedia Technical Blog on September 28th, 2009 at 08:10:

    [...] Blizzard from Mozilla has a detailed blog post about the improvements and what they mean for open video creators, distributors and viewers. [...]

  21. michael dale wrote on September 28th, 2009 at 08:17:

    I would mention you can give the 1.1 Theora encoder a test drive with an in-browser encoder extension at firefogg.org

    http://firefogg.org/make/index.html

  22. Sage Ross wrote on September 28th, 2009 at 10:58:

    Nice. I have a bug report and I’m not sure where to leave it.

    I was trying to do two-pass encoding yesterday, both with Firefogg and with the ffmpeg2theora command line, and it would not work. With Firefogg (installed yesterday), it simply never gets past 0% complete when two-pass is selected (works fine without it) and with ffmpeg2theora (the nightly from yesterday) it gives a message about being unable to create a temporary file.

    This was on Windows Vista.

    1. Ananda wrote on February 2nd, 2010 at 08:55:

      That sounds like a permissions issue that is indicative of Windows Vista, which by default, does not let software write to any folder outside of your profile (my documents, desktop, etc.). The appropriate place to leave bug reports would be in the ffmpeg2theora project website & firefogg’s website.

      “With Firefogg (installed yesterday), it simply never gets past 0% complete when two-pass is selected (works fine without it) and with ffmpeg2theora (the nightly from yesterday) it gives a message about being unable to create a temporary file.

      This was on Windows Vista.”

  23. Rahul Sundaram wrote on September 29th, 2009 at 03:20:

    Christopher Blizzard – Brilliant post

    Matthew Gregan – It would be very nice if it did. Alternatively you can consider integrating gstreamer atleast for Linux in a optional way.

  24. Pingback from Theora 1.1, l'alternativa open a H.264 on September 29th, 2009 at 09:11:

    [...] [...]

  25. Pingback from Ubuntu 10.10 Needs a Better Online Video Experience on September 30th, 2009 at 12:37:

    [...] Firefox 3.5 appearing in Ubuntu 9.10 and excellent improvements happening to the Theora encoder and decoder, widespread adoption of the <video> tag could be the best thing to happen to the online Linux [...]

  26. david cs wrote on September 30th, 2009 at 22:02:

    why the videos encoded with 1.1 are more blurry than v1.0?

  27. Pingback from Ogg: Theora 1.1 on October 1st, 2009 at 04:01:

    [...] Enviado por vinicius (vini_ipsmakerΘyahoo·com·br): “Foi lançada a nova versão deste codec livre de vídeo. A nova versão teve várias melhorias; acho que é só esperar pelo HTML5 que veremos o theora entre os codecs mais usados de todos. Agora falta o Dirac melhorar e o CELT tornar-se estável.” [referência: hacks.mozilla.org] [...]

  28. Robin wrote on October 2nd, 2009 at 06:52:

    There’s a typo in the section “Decoder performance has been improved” where it says “encoder” in the text instead of “decoder”.

  29. Pingback from Open Video Conference: June 19-20 on October 2nd, 2009 at 08:51:

    [...] up to par quality-wise, and though the format itself hasn’t been updated, Mozilla’s Christopher Blizzard does a great job in showing how the Theora 1.1 codec (compression-decompression software) brings huge improvements. These [...]

  30. Tack wrote on October 2nd, 2009 at 09:47:

    I do see a bit more ringing in the theora 1.1 fx35_overview video, so maybe 10MB was somewhat overly optimistic.

  31. Pingback from Firefox poderá trazer melhorias para exibição nativa de vídeos no futuro | MacMagazine on October 2nd, 2009 at 12:33:

    [...] um bom tempo até que possamos conferir essas mudanças. Chris Blizzard, um dos membros da empresa, prevê que os primeiros experimentos nessa área estejam prontos para pré-avaliação em algumas semanas. [...]

  32. Tony Lovasco wrote on October 3rd, 2009 at 09:23:

    Theora 1.1 is vastly improved from 1.0 in all of my tests. Theora is finally starting to provide some real competition for patent-encumbered codecs such as Xvid.

  33. David wrote on October 5th, 2009 at 20:05:

    John S. Gook: Theora is an enhanced version of VP3. VP3 was developed by On2, not “kiddies”. VP3 is also the basis of the Flash video codecs. And this has nothing to do with “sticking it to the man”. Mozilla cannot legally ship AVC. What would you have them do—close the source, restrict distribution, and pay royalties for every Firefox download? Please learn some facts before you troll.

  34. dave wrote on October 6th, 2009 at 08:38:

    Watching the promo video again I’ve noticed that when the text cards are shown (e.g. the logo, or Fast, Faster, Fastest text) you can still see about 1 pixels worth of Mike Beltzner’s hair bobbing around at the top.

    As well as being slightly distracting, that’s not going to help compression.

  35. Pingback from FOSS for Life » Blog Archive » Announcing Fedora 12 on November 18th, 2009 at 02:35:

    [...] Next-generation (Ogg) Theora video – For several years, Theora, the open and free format not encumbered by known patents has provided a way for freedom-loving users to share video. Fedora 12 includes the new Theora 1.1, which achieves very high quality comparable to H.264, meeting the expectations of demanding users with crisp, vibrant media in both streaming and downloadable form. Thanks to the work of the Xiph.Org Foundation’s Christopher “Monty” Montgomery, sponsored by Red Hat, other Xiph developers and the contribution of Mozilla.org, Theora videos now deliver much better quality primarily via enhancements in the encoder without any change in the format, making it available to all Theora users. Using Theora video and Vorbis audio formats, Firefox 3.5 and applications using the Gstreamer multimedia framework can deliver free media on the web out of the box even better than the previous release of Fedora. Theora is being rapidly adopted by several popular websites including Wikipedia, VideoPress and DailyMotion. Fedora Project is proud to support communities of free culture and open content as part of our mission. More details at http://hacks.mozilla.org/2009/09/theora-1-1-released/ [...]

  36. Pingback from Fedora 12 Released on November 19th, 2009 at 01:12:

    [...] Next-generation (Ogg) Theora video – For several years, Theora, the open and free format not encumbered by known patents has provided a way for freedom-loving users to share video. Fedora 12 includes the new Theora 1.1, which achieves very high quality comparable to H.264, meeting the expectations of demanding users with crisp, vibrant media in both streaming and downloadable form. Thanks to the work of the Xiph.Org Foundation’s Christopher “Monty” Montgomery, sponsored by Red Hat, other Xiph developers and the contribution of Mozilla.org, Theora videos now deliver much better quality primarily via enhancements in the encoder without any change in the format, making it available to all Theora users. Using Theora video and Vorbis audio formats, Firefox 3.5 and applications using the Gstreamer multimedia framework can deliver free media on the web out of the box even better than the previous release of Fedora. Theora is being rapidly adopted by several popular websites including Wikipedia, VideoPress and DailyMotion. Fedora Project is proud to support communities of free culture and open content as part of our mission. More details at http://hacks.mozilla.org/2009/09/theora-1-1-released/ [...]

  37. Pingback from Distribution Release: Fedora 12 - LinuxTuts | Tutorials on December 5th, 2009 at 03:04:

    [...] to support communities of free culture and open content as part of our mission. More details at theora 1.1 is released – what you should know at hacks.mozilla.org Graphics support improvements – Fedora 12 introduces experimental 3D support for AMD Radeon HD [...]

  38. Daniel Hendrycks wrote on January 24th, 2010 at 11:24:

    http://www.petitionspot.com/petitions/oggandyoutube/

    Petition for Ogg on YouTube’s HTML5 player. Please make an article about this.

Comments are closed for this article.