Mozilla

Standards Articles

Sort by:

View:

  1. Multimedia on the web and using HTML5 sensibly

    Last week I went to the London Ajax User Meetup in London, England to deliver two talks about HTML5. One was a re-run of a talk I gave at MIT about Multimedia on the web and the second was a call to arms to use HTML5 sensibly. You can go over to Skillsmatter web site to see both talks back to back – but be sure to catch the notes to the second talk in this post, too.

    In addition to my notes here there is also a great write-up of the evening by Ian Alderson available on the web.

    Multimedia on the web

    We’ve covered this talk before in the blog post here and you can read the extensive notes for the whole talk with code examples on my blog. The slides of the multimedia talk are on slideshare and here:

    You can see “Multimedia on the web” on any HTML5 enabled device here (courtesy of vid.ly) or embedded here:

    Using HTML5 sensibly

    Using HTML5 sensibly was a talk I wanted to give for a long time. Rather than just giving the facts, I wanted to raise some questions every developer should ask themselves, the HTML5 working groups and everyone who cares to listen. We are in the middle of a big change in our development environment and not many people realise that we are breaking a lot of old conventions without fallback.

    You can get the slides on Slideshare or see them here:

    You can see “Using HTML5 sensibly” on any HTML5 enabled device here (courtesy of vid.ly).

    Notes on “Using HTML5 sensibly”

    I started the talk by telling the story of the Antarctic Snow Cruiser – a marvel of technology built in 1939 to allow a crew to explore Antarctica. The job of the Cruiser was to give the crew all the neccesary creature comforts and allow them to drive through Antarctica, which is both very cold and also features massive cracks that the vehicle somehow needs to get over. The ingenious solution was to have huge wheels that can be retracted into the Cruiser to prevent them from getting brittle over night and to push the vehicle over the cracks. The Cruiser was built in a rush and tested on the way to the port to go down to Antarctica. It was a hit with the press and people delayed the whole trip even more by wanting to have their photo taken with it. Upon arrival the vehicle almost crashed into the sea as the ramp built for its delivery onto the ice was not strong enough. Even worse, once on the ice the Cruiser couldn’t get a grip and the wheels spun uselessly, almost overheating the engine. Despite all the hype, the cruiser was a massive failure.

    A few things went wrong with the cruiser:

    • It was purely engineering driven
    • It was tested while on the road
    • It was never tested in the real environment
    • There was a massive media excitement before it proved its worth

    Much like we deal with HTML5 nowadays. A lot of the information about HTML5 is marketing driven and used to sell a certain environment or browser to people. Instead of real applications and day-to-day use web sites we create demo sites to show what “HTML5 can do for you”. The term HTML5 gets washed out as it is used both for implementations of parts of the specification but also for browser-specific code that only works in a certain environment.

    This is partly because of the specifications and the different players in the HTML5 world itself. The specs do not only contain definitions of markup but also JavaScript APIs and instructions on how to write a browser that renders HTML5 content. This is great as it takes away the liberty browser vendors had in the past in “creatively” applying the HTML4 specs, but it also means that people can focus on parts of the spec and of course rant about all the other parts. You’ve probably seen one talk about the amazing cool new semantics of HTML5 markup and another about API implementations using completely non-sensical markup back to back. This is what we get right now as the specs are too big and all-encompassing for implementers.

    One of the main topics of the HTML5 work is pragmatism – XML and strict HTML just used too many things we simply don’t need. This shows itself a lot in the simplification of a plain HTML document. In HTML4.01 strict this was:

    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
    <html lang="en">
    <head>
      <meta http-equiv="Content-Type" 
            content="text/html; charset=UTF-8">  
      <title>HTML - c'est moi!</title>
      <link rel="stylesheet" type="text/css"
            href="styles.css">
      <script type="text/javascript" src="magic.js">
      </script>
    </head>
    <body>
      <h1>Heading! Everybody duck!</h1>
      <p>I am content, hear me roar!</p>
      <p class="footer">By me!</p>
    </body>
    </html>

    Neither the long DOCTYPE definition nor the http-equiv was ever used by browsers. It is not needed for browsers to do their job. In HTML5 this is a plain Vanilla document:

    <!DOCTYPE html>  
    <html lang="en">
    <head>
      <meta charset="utf-8">  
      <title>HTML5, c'est moi, ou HTML...</title>
      <link rel="stylesheet" href="styles.css">
      <script src="magic.js"></script>
    </head>
    <body>
      <h1>Heading! Everybody duck!</h1>
      <p>I am content, hear me roar!</p>
      <p class="footer">By me!</p>
    </body>
    </html>

    Much simpler and shorter. In addition to that HTML5 also includes new semantic elements:

    <!DOCTYPE html>  
    <html lang="en">
    <head>
      <title>HTML5, c'est moi, ou HTML...</title>
      <link rel="stylesheet" href="styles.css">
      <script src="magic.js"></script>
    </head>
    <body>
      <header><h1>Heading! Everybody duck!</h1></header>
      <section>
        <p>I am content, hear me roar!</p>
      </section>
      <footer><p>By me!</p></footer>
    </body>
    </html>

    This gives us handles for styling and tells the browser what the different parts are. The simplification goes further though. As people never seemed to have bothered to write valid XML in HTML it is now also completely OK to mix upper and lower case and omit quotation marks around the attributes (if the attributes have one value, multi value attributes like classes need quotes around them once you add more than one value):

    <!DOCTYPE html>  
    <html lang=en>
      <TITLE>HTML5, c'est moi, ou HTML...</title>
      <LINK rel=stylesheet href=styles.css>
      <script src=magic.js></SCRIPT>
    <body>
      <HEADER><h1>Heading! Everybody duck!</h1></header>
      <section>
        <p>I am content, hear me roar!</p>
      </SECTION>
      <footer><p>By me!</p></FOOTER>
    </body>
    </HTML>

    Browsers are forgiving enough to fix these inconsistencies for us. For example the generated code in Firefox is:

    <!DOCTYPE html>  
    <html lang="en"><head>  
      <title>HTML5, c'est moi, ou HTML...</title>
      <link rel="stylesheet" href="styles.css">
      <script src="magic.js"></script>
      </head><body>
      <header><h1>Heading! Everybody duck!</h1></header>
      <section>
        <p>I am content, hear me roar!</p>
     
      </section>
      <footer><p>By me!</p></footer>
    </body></html>

    However, if we were to omit the closing </p> tag in the <section> the generated code looks different:

    <!DOCTYPE html>
    <html lang="en"><head>  
      <title>HTML5, c'est moi, ou HTML...</title>
      <link rel="stylesheet" href="styles.css">
      <script src="magic.js"></script>
    </head><body>
      <header><h1>Heading! Everybody duck!</h1></header>
      <section>
        <p>I am content, hear me roar!
          <footer></footer>
        </p>
        <p>By me!</p>
      </section>
    </body></html>

    As you can see, the <footer> got moved into the paragraph inside the section. Why? Who knows? The issue is that we ask a browser to make an educated guess for us and as a browser doesn’t understand semantics, this is what we get.

    Which leads me to a few questions we need to ask and find the answer to:

    • Can innovation be based on “people never did this correctly anyways”?
    • Is it HTML or BML? (HyperText Markup Language or Browser Markup Language)
    • Should HTML be there only for browsers? What about conversion Services? Search bots? Content scrapers?

    The next issue I wanted to discuss is legacy browsers which seems to be a euphemism for IE6. The issue with IE6 is that when you use the new semantic HTML elements exclusively then they can’t be styled in this browser. By using the correct new syntax that should bring us a lot of benefits you condemn older browser users to see unstyled documents.

    There are a few ways around this. The first one requires JavaScript – if you create the element using the DOM first you can style it in IE:

    <!DOCTYPE html>
    <html lang="en">
    <head>
      <meta charset="utf-8">
      <style>header{color:#fff;background:#369;}</style>
      <script>document.createElement('header');</script>
    </head>
    <body>
      <header><h1>Hello!</h1></header>
    </body>
    </html>

    This technique has been wrapped up nicely in a script called HTML5 shiv by Remy Sharp. Of course, this means that you make markup dependent on JavaScript to work which is not what markup is about, according to purists of the web. You can work around that dependency in a few ways. You can write HTML5 as XHTML, you can use conditional comments or a namespace or you can use both the new semantic elements and DIVs with a class around them to please all browsers:

    <!DOCTYPE html>
    <html lang="en">
      <head><meta charset="utf-8">
        <title>HTML5, c'est moi, ou HTML...</title>
        <link rel="stylesheet" href="styles.css">
        <script src="magic.js"></script>
        <style type="text/css" media="screen">
          .header,header,
          .footer,footer,
          .aside,aside,
          .section,section{
            display:block;
          }
        </style>
      </head>
      <body>
      <div class="header"><header>
        <h1>Heading! Everybody duck!</h1>
      </header></div>
      <div class="section"><section>
        <p>I am content, hear me roar!</p>
      </section></div>
      <div class="footer"><footer>
        <p>By me!</p>
      </footer></div>
    </body>
    </html>

    This seems to be the most pragmatic solution but is a lot of extra effort.

    The main issue is that not only legacy browsers are failing to support HTML5 properly which is why people tend to use a library like Modernizr to check for HTML5 support before applying it. Modernizr adds classes to the HTML element after checking what the current browser supports and thus gives you handles to use in progressive enhancement. All the hacks and fixes are also bundled in HTML5 Boilerplate, including server settings and ways to make legacy browsers behave. In essence, the much smaller Vanilla HTML5 markup becomes a lot of code again if you test it in the real world.

    If you also want to use all the cool new JavaScript APIs you can use Polyfills to make legacy browsers do something with the code you write.

    Which leads to more questions we need to answer:

    • Should we shoe-horn new technology into legacy browsers?
    • Do patches add complexity as we need to test their performance? (there is no point in giving an old browser functionality that simply looks bad or grinds it down to a halt)
    • How about moving IE fixes to the server side? Padding with DIVs with classes in PHP/Ruby/Python after checking the browser and no JS for IE?

    The last feature of HTML5 I want to talk about is Video and Audio. Again, there is much more complexity than meets the eye:

    Tecnically, embedding an HTML5 video should be as simple as this:

    <video src="interview.ogv" controls>
      <a href="interview.ogv">Download the video</a>
    </video>

    Once you consider all the different formats needed for different browsers though it becomes a lot more work:

    <video controls>
      <source src="interview.mp4" type="video/mp4">
      </source>
      <source src="interview.webm" type="video/webm">
      </source>
      <source src="interview.ogv" type="video/ogg">
      </source>
      <p>Download the 
        <a href="interview.mp4">video in MP4 format</a>.
      </p>
    </video>

    The reason is intellectual property, legal codec battles and implementation of video in different browsers and platforms. In essence, you need to provide the video in three formats. Of course when there is a need then some company will come up with a solution. Vid.ly is a service by encoding.com that provides you with a single URL that sends the video in the right format to the device you use. All in all they create 28 different formats for you to cater for all kind of mobiles and browsers.

    The other big issue with HTML5 video is that there is no protection from downloading the videos. This is as intended but a big problem when it comes to premium content on the web. As discussed in the comments of this post by Adobe publishers want to have a way to stop people from downloading and reusing their videos but instead just want their audience to watch them in a closed environment.

    All in all there are a few questions to answer when it comes to HTML5 video:

    • Can we expect content creators to create video in many formats to support an open technology?
    • Can a service like vid.ly be trusted for content creation and storage?
    • Is HTML5 not applicable for premium content?

    In the end, I want every developer and designer out there to take a stand and demand answers to these questions. There is no point in just saying “that’s how it is”. When we keep our technologies closed and when we rely on scripting to make web sites work we do break the web in the long run. The current debate about hashbangs for links shows this.

    Everybody can take part in the HTML5 working group and add documentation to MDN – time to get to work.

  2. People of HTML5 – Remy Sharp

    HTML5 needs spokespeople to work. There are a lot of people out there who took on this role, and here at Mozilla we thought it is a good idea to introduce some of them to you with a series of interviews and short videos. The format is simple – we send the experts 10 questions to answer and then do a quick video interview to let them introduce themselves and ask for more detail on some of their answers.

    Leggi la traduzione in italiano

    Remy SharpToday we are featuring Remy Sharp co-author of Introducing HTML5 and organiser of the Full Frontal conference in Brighton, England.

    Remy is one of those ubiquitous people of HTML5. Whenever something needed fixing, there is probably something on GitHub that Remy wrote that helps you. He is also very English and doesn’t mince his words much.

    You can find Remy on Twitter as @rem.

    The video interview

    Watch the video on YouTube or Download it from Archive.org as MP4 (98 MB), OGG (70 MB) or WebM (68MB)

    Ten questions about HTML5 for Remy Sharp

    1) Reading “Introducing HTML5″ it seems to me that you were more of the API – focused person and Bruce the markup guy. Is that a fair assumption? What is your background and passion?

    That’s spot on. Bruce asked me to join the project as the “JavaScript guy” – which is the slogan I wear under my clothes and frequently reveal in a superman ‘spinning around’ fashion (often at the surprise of clients).

    My background has always been coding – even from a young age, my dad had me copying out listings from old spectrum magazines only to result in hours of typing and some random error that I could never debug.

    As I got older I graduated to coding in C but those were the days the SDKs were 10Mb downloaded over a 14kb modem, and compile in to some really odd environment. Suffice to say I didn’t get very far.

    Then along came JavaScript. A programming language that didn’t require any special development environment. I could write the code in Notepad on my dodgy Window 95 box, and every machine came with the runtime: the browser. Score!

    From that point on the idea of instant gratification from the browser meant that I was converted – JavaScript was the way for me.

    Since then I’ve worked on backend environments too (yep, I’m a Perl guy, sorry!), but always worked and played in the front end in some way or another. However, since started on my own in 2006, it’s allowed me to move focus almost entirely on the front end, and specialise in JavaScript. Basically, work-wise: I’m a pig in shit [Ed: for our non-native English readers, he means "happy")].

    2) From a programmer’s point of view, what are the most exciting bits about the HTML5 standard? What would you say is something every aspiring developer should get their head around first?

    For me, the most exciting aspects of HTML5 is the depth of the JavaScript APIs. It’s pretty tricky to explain to Joe Bloggs that actually this newly spec’ed version of HTML isn’t mostly HTML; it’s mostly JavaScript.

    I couldn’t put my finger on one single part of the spec, only because it’s like saying which is your favourite part of CSS (the :target selector – okay, so I can, but that’s not the point!). What’s most exciting to me is that HTML5 is saying that the browser is the platform that we can deliver real applications – take this technology seriously.

    If an aspiring developer wanted something quick and impressive, I’d say play around with the video API – by no means is this the best API, just an easy one.

    If they really wanted to blow people away with something amazing using HTML5, I’d say learn JavaScript (I’m assuming they’re already happy with HTML and CSS). Get a book like JavaScript: The Good Parts and then get JavaScript Patterns and master the language. Maybe, just maybe, then go buy Introducing HTML5, it’s written by two /really/ good looking (naked) guys: http://flic.kr/p/8iyQTE and http://flic.kr/p/8iy6Z1 [Ed: maybe NSFW, definitely disturbing].

    3) In your book you wrote a nice step-by-step video player for HTML5 video. What do you think works well with the Video APIs and what are still problems that need solving?

    The media API is dirt simple, so it means working with video and audio is a doddle. For me, most of it works really well (so long as you understand the loading process and the events).

    Otherwise what’s really quite neat, is the fact I can capture the video frames and mess with them in a canvas element – there’s lots of fun that can be had there (see some of Paul Rouget’s demos for that!).

    What sucks, and sucks hard, is the spec asks vendors, ie. browser makers, *not* to implement full screen mode. It uses security concerns as the reason (which I can understand), but Flash solved this long ago – so why not follow their lead on this particular problem? If native video won’t go full screen, it will never be a competitive alternative to Flash for video.

    That all said, I do like that the folks behind WebKit went and ignored the spec, and implemented full screen. The specs are just guidelines, and personally, I think browsers should be adding this feature.

    4) Let’s talk a bit about non-HTML5 standards, like Geolocation. I understand you did some work with that and found that some parts of the spec work well whilst others less so. Can you give us some insight?

    On top of HTML5 specification there’s a bunch more specs that make the browser really, really exciting. If we focus on the browser being released today (IE9 included) there’s a massive amount that can be done that we couldn’t do 10 years ago.

    There’s the “non-HTML5″ specs that actually were part of HTML5, but split out for good reason (so they can be better managed), like web storage, 2D canvas API and Web Sockets, but there’s also the /really/ “nothing-to-do-with-HTML5″ APIs (NTDWH5API!) like querySelector, XHR2 and the Device APIs. I’m super keen to try all of these out even if they’re not fully there in all the browsers.

    Geolocation is a great example of cherry picking technology. Playing against the idea that the technology isn’t fully implemented. Something I find myself ranting on and on about when it comes to the question of whether a developer should use HTML5. Only 50% of Geolocation is implemented in the browsers supporting it, in that they don’t have altitude, heading or speed – all of which are part of the spec. Does that stop mainstream apps like Google Maps from using the API? (clue: no).

    The guys writting the specs have done a pretty amazing job, and in particular there are few cases where the specs have been retrospectively written. XHR is one of these and now we’ve got a stable API being added in new browsers (i.e. IE6 sucks, yes, we all know that). Which leads us to drag and drop. The black sheep of the APIs. In theory a really powerful API that could make our applications rip, but the technical implementation is a shambles. PPK (Peter-Paul Koch) tore the spec a bit of a ‘new one’. It’s usable, but it’s confusing, and lacking.

    Generally, I’ve found the “non-HTML5″ specs to be a bit of mixed bag. Some are well supported in new browsers, some not at all. SVG is an oldie and now really viable with the help of JavaScript libraries such as Raphaël.js or SVGWeb (a Flash based solution). All in all, there’s lots of options available in JavaScript API nowadays compared to back in the dark ages.

    5) Let’s talk Canvas vs. SVG for a bit. Isn’t Canvas just having another pixel-based rectangle in the page much like Java Applets used to be? SVG, on the other hand is Vector based and thus would be a more natural tool to do something with open standards that we do in Flash now. When would you pick SVG instead of Canvas and vice versa?

    Canvas, in a lot of ways is just like the Flash drawing APIs. It’s not accessible and a total black box. The thing is, in the West, there’s a lot of businesses, rightly or wrongly, that want their fancy animation thingy to work on iOS. Since Flash doesn’t work there, canvas is a superb solution.

    However, you must, MUST, decide which technology to use. Don’t just use canvas because you saw a Mario Kart demo using it. Look at the pros and cons of each. SVG and the canvas API are not competitive technologies, they’re specially shaped hammers for specific jobs.

    Brad Neuberg did a superb job of summarising the pros and cons of each, and I’m constantly referring people to it (here’s the video).

    So it really boils down to:

    • Canvas: pixel manipulation, non-interactive and high animation
    • SVG: interactive, vector based

    Choose wisely young padawan!

    6) What about performance? Aren’t large Canvas solutions very slow, especially on mobile devices? Isn’t that a problem for gaming? What can be done to work around that?

    Well…yes and no. I’m finishing a project that has a large canvas animation going on, and it’s not slow on mobile devices (not that it was designed for those). The reason it’s not slow is because of the way the canvas animates. It doesn’t need to be constantly updating at 60fps.

    Performance depends on your application. Evaluate the environment, the technologies and make a good decision. I personally don’t think using a canvas for a really high performance game on a mobile is quite ready. I don’t think the devices have the ommph to get the job done – but there’s a hidden problem – the browser in the device isn’t quite up to it. Hardware acceleration is going to help, a lot, but today, right now, I don’t think we’ll see games like Angry Birds written in JavaScript.

    That said… I’ve seriously considered how you could replicate a game like Canabalt using a mix of canvas, DIVs and CSS. I think it might be doable ::throws gauntlet::

    I think our community could actually learn a lot from the Flash community. They’ve been through all of this already. Trying to make old versions of Flash from years back do things that were pushing the boundaries. People like Seb Lee-Delisle (@seb_ly / http://seb.ly) are doing an amazing job of teaching both the Flash and JavaScript community.

    7) A feature that used to be HTML5 and is now an own spec is LocalStorage and its derivatives Session Storage or the full-fledged WebSQL and IndexedDB. Another thing is offline storage. There seems to be a huge discussion in developer circles about what to use when and if NoSQL solutions client side are the future or not. What are your thoughts? When can you use what and what are the drawbacks?

    Personally I love seeing server-less applications. Looking at the storage solutions I often find it difficult to see why you wouldn’t use WebStorage every time.

    In a way it acts like (in my limited experience of) NoSQL, in that you lookup a key and get a result.

    Equally, I think SQL in the browser is over the top. Like you’re trying to use the storage method *you* understand and forcing it into the browser. Seems like too much work for too little win.

    Offline Apps, API-wise, ie. the application cache is /really/ sexy. Like sexy with chocolate on top sexy. The idea that our applications can run without the web, or switch when it detects it’s offline is really powerful. The only problem is that the events are screwed. The event to say your app is now offline requires the user to intervene via the browser menu, telling the browser to “work in offline mode”. A total failure of experience. What’s worse is that, as far as I know, there’s no plan to make offline event fire properly :-(

    That all said, cookies are definitely dead for me. I’ve yet to find a real solution for cookies since I found the Web Storage API – and there’s a good decent number of polyfills for Web Storage – so there’s really no fear of using the API.

    8) Changing the track a bit, you’ve built the HTML5shiv to make HTML5elements styleable in IE. This idea sparked quite a lot of other solutions to make IE6 work with the new technologies (or actually simulate them). Where does this end? Do you think it is worth while to write much more code just to have full IE6 support?

    There’s two things here:

    1. Supporting IE6 (clue: don’t)
    2. Polyfills

    IE6, seriously, and for the umpteenth time, look at your users. Seriously. I know the project manager is going to say they don’t know what the figures are, in that case: find out! Then, once you’ve got the usage stats in hand, you know your audience and you know what technology they support.

    If they’re mostly IE6 users, then adding native video with spinning and dancing canvas effect isn’t going to work – not even with shims and polyfills. IE6 is an old dog that just isn’t up to doing the mileage he used to be able to do back in his prime. But enough on this subject – the old ‘do I, or don’t I developer for IE6′ is long in the tooth.

    Polyfills – that’s a different matter. They’re not there to support IE6, they’re there to bring browsers up to your expectations as a developer. However, I’d ask that you carefully consider them before pulling them in. The point of these scripts is they plug missing APIs in those older browsers. “Old browsers” doesn’t particularly mean IE6. For example, the Web Sockets API has a polyfill by way of Flash. If native Web Sockets aren’t there, Flash fills the gap, but the API is exposed in exactly the same manner, meaning that you don’t have to fork your code.

    I don’t think people should be pulling in scripts just for the hell of it. You should consider what you’re trying to achieve and decide whether X technology is the right fit. If it is, and you know (or expect) your users have browsers that don’t support X technology – should you plug it with JavaScript or perhaps should you consider a different technology?

    This exact same argument rings true for when someone adds jQuery just to add or remove a class from an element. It’s simply not worth it – but clearly that particular developer didn’t really understand what they needed to do. So is education the solution? I should hope so.

    9) Where would you send people if they want to learn about HTML5? What are tutorials that taught you a lot? Where should interested people hang out?

    HTML5 Doctor – fo sho’. :)

    I tend to also direct people to my http://html5demos.com simply to encourage viewing source, and hacking away.

    Regarding what tutorials taught me – if I’m totally honest, the place I’ve learnt the most from is actually HTML5Doctor.com. There’s some pretty good JavaScript / API tutorials coming from the chaps at http://bocoup.com. Otherwise, I actually spend a lot of time just snooping through the specifications, looking for bits that I’ve not seen before and generally poking them with a stick.

    10) You have announced that you are concentrating on building a framework to make Websockets easy to work with. How is that getting on and what do you see Websockets being used for in the future? In other words, why the fascination?

    Concentrating is a strong word ;-) but it is true, I’ve started working on a product that abstracts Web Sockets to a service. Not the API alone, since it’s so damn simple, but the server setup: creating sessions, user control flow, waiting for users and more.

    The service is called Förbind. Swedish for “connect”, ie. to connect your users. It’s still early days, but I hope to release alpha access to forbind.net this month.

    I used to work in finance web sites and real-time was the golden egg: to get that data as soon as it was published. So now that it’s available in a native form in the browser, I’m all over it!

    What’s more, I love the idea of anonymous users. I created a bunch of demos where the user can contribute to something without ever really revealing themselves, and when the users come, you start to see how creative people are without really trying. Sure, you get a lot of cocks being drawn, but you also see some impressive ideas – my business 404 page for example allows people to leave a drawing, one of the most impressive is a Super Mario in all his glory. Anonymous users really interest me because as grey as things can seem sometimes, a stranger can easily inspire you.

    Do you know anyone I should interview for “People of HTML5″? Tell me on Twitter: @codepo8

  3. A call for quality HTML5 demo markup

    HTML5 is a necessary evolution to make the web better. Before the HTML5 specs were created we used (and still use) a hacked together bunch of systems meant for describing and linking documents to create applications. We use generic elements to simulate rich interaction modules used in desktop development and we make assumptions as to what the user agent (browser) can do for the end user.

    The good thing about this mess is that it taught us over the last few years to be paranoid in our development approach – we realised that in order to deliver working, accessible, maintainable and scalable systems we have to be professional and intelligent in our decisions and especially in our planning and architecting.

    The trouble is that with the excitement around the cool new HTML5 things to play with a lot of these principles get lost and discarded as outdated. They aren’t – part of the mess that the web is in is that in 1999 a lot of people discarded everything else and instead worked only on the new hotness that was Internet Explorer 6 and DHTML. Let’s not repeat that mistake.

    The two faces of HTML5 enthusiasm

    Right now there are two things to get really excited about in HTML5: the richer, more meaningful semantics of new elements and the APIs that give us deep access into the workings of the browser and the operating system. The former allows us to build much richer interfaces and the latter allows us to build much more complex applications.

    All of this comes with the awesome of view source (or check in development tools) for debugging. You can advocate HTML5 by writing clean and useful markup. You can kill it by treating the markup as a second class citizen which is only there to serve the script.

    The markup enthusiasts are very happy about HTML5 and make it easy for people to upgrade – lots of cool new blog templates and boilerplate HTML templates are being built and polyfills created to allow using the new features and not leave behind outdated browsers.

    On the app development side of things it looks much different and that is just sad. My only possible explanation is that people who come from desktop environments now tackle the HTML5 APIs without ever having to care about markup. The pragmatism of HTML5 allows a much more forgiving syntax than XHTML but it shouldn’t mean that we can just re-apply all the bad mistakes that held us back when it comes to maintenance for years.

    During my career as a web developer I realised a few things make a lot of sense when building web apps:

    • If there is an element for a certain task – use that one. It is very likely that the element comes with accessibility and interaction features for free that you would otherwise have to simulate.
    • Separate CSS, JavaScript and HTML – which means it is easy to refactor your code without having to change all of them. It also means you can work in parallel with others instead of breaking each other’s code
    • Never rely on markup or content – as sooner or later some editing tool will come into place that messes with everything you created

    This means a lot of things:

    • For starters it means that inline styles are simply evil as they override any settings you have in your style sheets. Only use them when this is exactly what you need to do or when you calculate them dynamically.
    • The same applies to inline scripting. If you have an onclick="foo()" somewhere in your HTML and foo() changes to bar() you have to rename it in every HTML document (of course nowadays it is one template, but it still means hunting down a reference you might miss)
    • If instead of using a native HTML element for a certain job you use SPANs and DIVs you’ll have to add classes to make them look and work – and simulate the keyboard accessibility, too.
    • You can’t rely on the text value of any element. A <button>edit</button> using the “edit” as the trigger for certain functionality would have to have the JS localised, too when you create the German <button>bearbeiten</button>.

    Bla bla bla… C’mon Chris, it isn’t that bad!

    The above best practices have been mentioned for years and a lot of people get sick of seeing them repeated. After all, this is intelligent development and standard practice in backend technologies. I came across a lot of “creative” uses lately though when doing a view-source on HTML5 demos – specifically the ones in the HTML5 advent calendar. And here is my yield of what people do.

    Simulating a navigation list

    One of the first things I encountered was a painting tool that had a gallery of created graphics as a background. Now, to me this would be a simple list:

    <ul>
      <li>
        <a href="{link}">
          <img src="{src}" alt="{title}">
        </a>
      </li>
      [...]
    </ul>

    The markup I found though was this:

    <div id="sky">
      <div class="skyTile" style="{calculated styles};display:block">
        <img class="skyThumb" src="{src}" style="{generated styles}">
        <img src="images/thumb_dropShadow.png" 
             class="skyThumbShadow" 
             style="{generated styles}">
      </div>
      [...]
    </div>

    This, of course is generated by the backend. My first gripe is the dropshadow image, after all this is an HTML5 showcase – just use CSS3. We also have the three instances of generated styles and double classes. Granted, an extra class gives you a handle to all images instead of tiles, so why not. But as there is no real link around the image, the click handler has to read the url from somewhere. There is a lot of unnecessary back and forth between DOM and scripting here which does slow down the whole demo. Seeing that this is also the main navigation from the entry page to the editor this could be a list inside a nav element. A list groups these items together, a bunch of DIVs doesn’t (screen readers for example tell you how many items there are in a list).

    Another list I found was supposed to be links to refresh the app and have a hierarchy but instead was a flat list with classes to define hierarchy and group:

    <ul id="mitems">
      <li class="hedtop">Simple</li>
      <li class="mol">{name}</li>
      [... repeated ...]
      <li class="hed">{parent name}</li>
      <li class="mol">{name}</li>
      [... repeated ...]
    </ul>

    This could be OK, granted you also shift keyboard focus, but why not:

    <nav>
    <ul id="mitems">
      <li>{parent name}
        <ul>
          <li><a href="{url}">{name}</a></li>
          [... repeated ...]
        </ul>
      </li>
      [... repeated ...]
    </ul>
    </nav>

    This would give you styling hooks and functionality for free. Links and buttons are great to trigger functionality – but it seems that is too easy.

    Click (probably) here for functionality

    If I build a button for an application to trigger a certain functionality this is the markup:

    <button type="button" class="edit">Edit</button>

    Buttons are what trigger functionality – both a backend script or a JavaScript. They are keyboard accessible, have a disabled state and sensible out-of-the-box styling that nowadays can be completely overwritten. The class can be the definition of what the button should do – instead of the text which will change. You could use an ID but a class allows to repeat buttons (for example on the top and the bottom of a results list).

    The buttons I found though were slightly different:

    <div id="homeButtonPanel">
      <div class="homeButton" id="drawHomeButton" 
           style="display: block;">
        <p>Start Drawing</p>
      </div>
      <div class="homeButton" id="viewHomeButton" 
           style="display: block;">
        <p>View the Mural</p>
      </div>
    </div>
     
    <div id="controlPanel" style="display: block;">
    <div class="titleButton"><p>{title}</p></div>
      <div class="mainMenu">
        <div class="mainButton mainSelected" id="drawButton">
          <p>Draw</p>
          <div id="inkDisplay">
            <div id="inkMeter" style="width: 94px;">
            </div>
          </div>
        </div>
        <div class="menuContainer">
          <div class="menuButton drawMenuButton" id="behaviorsButton"
               style="display: block; opacity: 1;">
            <p>Gesture<br>Behaviors</p>
          </div>
        </div>
        <div class="menuContainer">
          <div class="menuButton drawMenuButton" id="artworkButton" 
               style="display: block; opacity: 1;">
            <p>Artwork</p>
          </div>
        </div>
        <div class="menuContainer">
          <div class="menuButton drawMenuButton" id="resetButton" 
               style="display: block; opacity: 1;">
            <p>Reset</p>
          </div>
        </div>
        <div class="menuContainer">
          <div class="menuButton drawMenuButton" id="undoButton" 
               style="display: block; opacity: 1;">
            <p>Undo</p>
          </div>
        </div>
        <div class="menuContainer">
          <div class="menuButton drawMenuButton" id="saveButton" 
               style="display: block; opacity: 1;">
            <p>Save</p>
          </div>
        </div>
      </div>
      <div class="mainMenu">
        <div class="mainButton" id="viewButton"><p>View</p></div>
      </div>
      <div class="mainMenu">
        <div class="secondaryButton" id="helpButton"><p>Help</p></div>
      </div>
      <div class="mainMenu">
        <div class="mainShare shareButtonSmall" id="mainTwitterButton">
          <img alt="twitter" src="images/twitter_icon.png" 
               style="opacity: 0.6;">
        </div>
        <div class="mainShare shareButtonLarge" id="mainFacebookButton">
          <img alt="facebook" src="images/facebook_icon.png">
        </div>
        <div class="mainShare shareButtonSmall" id="mainEmailButton">
          <img alt="email" src="images/email_icon.png" style="opacity: 0.6;">
        </div>
      </div>
    </div>

    So instead of using a nested list with classes for each button and the hierarchy in the nesting we have a lot of classes and a hand-rolled DIV construct. Instead of making buttons really disabled we rely on opacity and there is quite a mix of inline images and background images (if all were background images, they could be one sprite!). Keyboard navigation will have to be written for this and if you were to add a button you’d have to come up with another ID.

    HTML5 actually has a construct for this. There is a menu element with command child elements. These are in use in Chrome’s side bar for example and should be what we use. If you want to make it work for everyone, a nested list with button elements is what to go for.

    The overly complex DIV construct is quite used though – this was another example:

    <div class="uiScreen" id="startScreen">
      <div class="panelBackground" id="mainResumeButton" style="display: none;">
        <div class="largeUIButton" id="resumeButton">Resume Game</div>
      </div>
      <div class="panelBackground" id="mainStartButton">
        <div class="largeUIButton" id="startButton">Start Game</div>
      </div>		
      <div class="panelBackground" id="mainHelpButton">
        <div class="largeUIButton" id="helpButton">Help</div>
      </div>
      <div class="panelBackground" id="mainHighscoreButton">
        <div class="largeUIButton" id="highscoreButton">High Scores</div>
      </div>
    </div>

    When in doubt – add an ID and class to everything.

    Other buttons I encountered were actually links pointing to javascript:// using an inline style to indicate hierarchy:

    <ul class="navlist">
      <li><a id="play" href="javascript://" 
             style="display: inline;">Play</a></li>    
      <li><a id="pause" href="javascript://">Pause</a></li>
      <li style="padding-left: 2px;">Effects:</li>
      <li><a id="bw" href="javascript://">Bw</a></li>
      <li><a id="outline" href="javascript://">Outline</a></li>
      <li><a id="color" href="javascript://">Color</a></li>
      <li><a id="invert" href="javascript://">Invert</a></li>
      <li><a id="matrix" href="javascript://">Matrix</a></li>
      <li><a id="old" href="javascript://">Old</a></li>
    </ul>

    Talking of inline – here’s a great example of a tool generating a lot of code that could be replaced by a single event handler and event delegation:

    <div id="tools">
      <span onmouseout="buttonOut(1)" onmouseover="buttonOver(1)"
            onclick="buttonClicked(1)" id="button1" class="button">
        <img alt="" src="image/button/1.png">
      </span>
      <span onmouseout="buttonOut(2)" onmouseover="buttonOver(2)"
            onclick="buttonClicked(2)" id="button2" class="button">
            <img alt="" src="image/button/2.png">
      </span>
     
      [...repeated 20 times...]
     
    <div id="toolsSeparator">&nbsp;</div>
      <a title="" id="toolbarButtonSave" class="toolbarButton"
         href="javascript:saveCanvas()">
        <img alt="" src="image/save.png">
      </a>
      <a title="New" id="toolbarButtonNew" class="toolbarButton"
         href="javascript:newCanvas()">
        <img alt="New" src="image/new.png"></a>
      <!--[if !IE]><![IGNORE[--><!--[IGNORE[]]-->
      <a id="toolbarButtonMenu" class="toolbarButton"
         onmouseout="closeMenuDelay()" onmouseover="showMenuHover()"
         href="javascript:showMenu()">
        <img alt="&gt;" src="image/menu.png">
      </a>
      <div onmouseout="closeMenuDelay()" onmouseover="overMenu()" id="menu">
        <a class="saveMenu" onmouseout="closeMenuDelay()" 
           onmouseover="overMenu()" href="javascript:saveCanvas()">
          save normal
        </a>
        <a class="saveMenu" onmouseout="closeMenuDelay()" 
           onmouseover="overMenu()" href="javascript:saveCanvasHi()">
          save high quality
          <span class="footnote">&nbsp; (rename to *.png)</span>
        </a>
        <a onmouseout="closeMenuDelay()" onmouseover="overMenu()"
           href="javascript:showAbout()">
          about...
        </a>
        <a class="lastMenu" target="_top" onmouseout="closeMenuDelay()"
           onmouseover="overMenu()" 
           href="{url}">
          <span class="footnote">
            <em>elsewhere:&nbsp;</em>
          </span>
          <em>a sound memory game</em>
        </a>
      </div>
      <!--<![endif]-->
    </div>

    Notice that if the images for the button couldn’t be loaded for one reason or another (or you can’t see them at all) this application is very confusing indeed – no alternative text for the images and no text content to fall back to. I am also very much sure that the in and out handlers trigger visual effects CSS could handle better.

    Reasons and effects

    I know there are probably good reasons for all of this, and I am sure I will also do things wrongly when I am rushed or want to get things out quickly. What we have to understand though is that right now we have a responsibility to show the best of breed demos we can.

    We cannot preach the open web and technologies and view-source over closed systems and at the same time forget the things we learnt in the last decade. Some of these things I found look like code Frontpage or Dreamweaver could have produced in the 90ies and resulted in a lot of badly performing, hard to maintain products that either still annoy people who have to use them or get replaced every 2 years.

    We have a mandate to educate the new developers coming to the web. Unlearning something is much harder than learning it – so let’s not start with bloat and quick fixes that work but forget to advocate clean code and thinking about the impact your product has on the end users (thinking accessibility) and the poor sods that will have to maintain your product when you are gone. We are not here to advocate effects and products, we are here to promote the tools that allow anyone to easily build something cool that is also easy to understand.

    HTML5 is about evolving the web as a platform – we also need to evolve with it and take more responsibility. We have app and we have markup enthusiasts. Let’s make them work together to build things that are great functionality and clean semantics.

  4. Delivering the good message of local storage

    As you might know there are an incredible amount of advent calendar blogs out at the moment each delivering one cool article for each day of December until Christmas.

    Today (6/12/11) two calendar blogs delivered an article of mine talking about the benefits of using local storage in browsers and how to implement it.

    You can find the English version on the 24ways.org calendar and a German translation at the Webkrauts calendar.

    In essence, the article explains why it is a good idea to use local storage, explains how to use it, how to work around the issue that local storage can only store strings and gives example code on how to speed up web service use by caching information client-side much like you would do it on a server.

    Have a read and go speed up your own solutions by using what browsers provide you these days.

  5. Firefox 4: HTML5 Forms

    Firefox 4 will come with better support for HTML5 forms. In the latest beta we are experimenting with a set of new features: more inputs types (email, url, tel, search), new attributes (placeholder, autofocus, list), decoupled forms and different validation mechanisms. This is thanks mostly to the hard work of Mounir Lamouri.

    Some examples will work in other browsers, but you’ll need Firefox 4 beta to see all of them.

    New input field types

    In the same fashion as new HTML5 elements, we have new field types to better express what kind of input we want. The look and feel of many of these elements is the same as a text field, but they carry a different semantic meaning. This means that browsers can optimize the experience for users. For example, a mobile browser can provide a specific keyboard for a field. Or the browser can pre-fill phone number fields based on people in your address book. And extensions may even provide some features based on those types.

    In this beta, Firefox comes with four new kind of inputs :

    <input type="search">
    <input type="tel">
    <input type="url">
    <input type="email">

    Of the four new input types, url and email will also validate their content. We’ll talk about that later.

    We also have support for a new kind of field:

    <output for="i1 i2">

    You can use this element to represent an area of the page that reacts to the interaction with a form. Think about the total price in a cart after you’ve changed the quantity of items or the shipping options. It won’t compute anything, you’ll need to do that with JavaScript, but it will give hints to accessibility technologies. The for attribute is a list of IDs of the fields that intervene into the calculation.

    Text fields have been improved with <datalist> support. You can easily provide a list of suggestions for a field as the user is typing. You bind the field and the datalist by using the list attribute. It will use every option element inside datalist to populate the list. And for browsers without datalist support, they will display the content of the datalist element. So be sure to provide some valid markup to get a nice fallback.


    or

    <label>Enter a city: <input list="cities"></label>
    <datalist id="cities">
      or
      <label>choose one
        <select>
          <option value="Paris">Paris</option>
          <option value="London">London</option>
          <option value="Berlin">Berlin</option>
          <option value="New York">New York</option>
          <option value="Tokyo">Tokyo</option>
          <option value="Sydney">Sydney</option>
          <option value="Johannesburg">Johannesburg</option>
        </select>
      </label>
    </datalist>

    New input attributes

    Autofocus

    When you add this to a field, it will receive focus as soon as possible. The direct advantage for the user is that all sites will have the same algorithm for autofocus instead of relying on different JavaScript code. And browsers or extensions could disable this behavior if a user is not interested in it.

    <input autofocus>

    Placeholder

    The value of this attribute will be displayed inside the form when it is empty and not focused. You can put an example of the kind of expected value.



    <label>Telephone: <input placeholder="1-800-555-1234"></label>
    <label>Comments: <textarea placeholder="Tell us what you think about our service…"></textarea>

    Decoupled forms

    You have more options to define the interaction between fields and forms.

    form attribute

    <input> elements don’t need to be children of a <form> element anymore. You can define them anywhere you want and bind them to a form using the new form attribute. It takes the ID of the form it should bind to.

    Here’s an example. Let’s say you’re working on a search engine for some blogging software. You want a very simple form for the general use case and some advanced options if the user needs more control.

    At the top of the page you could put:

    <input type="search" name="search_field" form="search_form">

    And at the bottom:

    <form id="search_form" action="search.php" method="post">
      <fieldset>
        <legend>Advanced options</legend>
        <input type="checkbox">Private posts
        <!-- Other options -->
      </fieldset>
    </form>

    This will behave as if the search field was part of the form. And you get the freedom of placing it wherever you want in your HTML.

    Form options on fields

    All the options that can be defined at the form level can be overridden at the field element. All submit fields (<button> and input type=”submit”) accept four new attributes : formenctype, formaction, formmethod and formtarget.

    One use case could be a form with a preview and post buttons. Each one needs all the fields of the form but they perform a different action.

    <form action="new_post.php" method="post">
    <label>Title: <input type="text"></label>
    <label>Content: <textarea></textarea></label>
    <input type="submit" formaction="preview.php" formmethod="get" value="Preview">
    <input type="submit" value="Post">
    </form>

    When the user clicks on the Preview button, its attributes will override the form’s attributes. In this case, instead of a POST request to new_post.php, the whole form will be sent to the preview.php script with a GET method.

    Validation mechanisms

    One of the big area of improvements for forms is validation. To give the best experience, we need to give feedback as soon as possible to the user. So people have written a lot of JavaScript code to do that. Wouldn’t it be nicer if browsers handled that?

    required

    By adding this attribute, you’ll mark this field as required. For text fields, it means that it shouldn’t be left empty. For checkbox buttons, it means it should be checked. And for radio buttons, it means one of the button for a group should be selected.

    Try each of the examples below and you’ll see that they change color when you interact with them.

    <input type="text" required>


    <input type="checkbox" required>


    <input type="radio" name="radiogroup" required>
    <input type="radio" name="radiogroup" required>
    <input type="radio" name="radiogroup" required>


    url

    URL fields are automatically validated.

    <input type="url" value="mozilla">
    <input type="url" value="http://mozilla.org">



    email

    Emails are also automatically validated. By passing the multiple attribute (also valid on type=”file”), you can also validate a list of mails separated by commas.

    <input type="email" value="foo">
    <input type="email" value="foo@bar.org">



    <input type="email" multiple value="foo@bar.org, spongebob">
    <input type="email" multiple value="foo@bar.org, spongebob@squarepants.org">



    pattern

    Urls and emails will not be the only type of data you’ll want to validate. Therefore, the pattern attribute will allow you to provide a JavaScript regular expression. This will be matched against the value of the field to determine if it’s valid. You should also provide a title attribute explaining the pattern to the user.

    In the example below, try hovering over the text field. You should see a popup that tells you how to fill out the form.

    <input pattern="[0-9][A-Z]{3}" title="A part number is a digit followed by three uppercase letters.">


    The constraint validation API

    If you need even more control over the validation, you can use the setCustomValidity method. If you provide an empty string to this method, the element will be considered valid. Otherwise, it will be marked as invalid and the string will be used as a tooltip to help your user understand the problem.

    <label>Password: <input type="password" id="password1" oninput="checkPasswords()"></label>
    <label>Confirm password: <input type="password" id="password2" oninput="checkPasswords()"></label>
    <script>
    function checkPasswords() {
      var password1 = document.getElementById('password1');
      var password2 = document.getElementById('password2');
      if (password1.value != password2.value) {
        password2.setCustomValidity('Passwords should match');
      } else {
        password2.setCustomValidity('');
      }
    }
    </script>



    If one of the field of a form is not valid, then submitting the form will be blocked and the first invalid field will be focused with a message explaining the problem. If you want to override this behaviour and send the form anyway, you can add a novalidate attribute on the form or the formnovalidate attribute on the appropriate submit button.

    If you want more details on the validation mechanisms, check out Mounir’s blog post.

    New CSS selectors

    And to go with all this goodness are a few new CSS selectors.

    :required, :optional

    All fields are marked as :optional by default. If they have the required attribute, they’ll match the :required pseudo-class instead.

    :valid, :invalid

    These pseudo-classes represent the state of the field regarding validation. You can use :invalid to override the default styling that Firefox 4 provides.

    Here’s an example of a text box where the default style has been overridden.



    :-moz-placeholder

    This pseudo-class targets input fields displaying a placeholder. This is not yet part of CSS, so you’ll need to use a pseudo-element for WebKit based browsers.

    <style>
    #selectors2 :-moz-placeholder {
      font-style: italic;
    }
    #selectors2 ::-webkit-placeholder {
      font-style: italic;
    }
    </style>
    <form id="selectors2">
      <input placeholder="Style me">
    </form>

    Conclusion

    Form features in HTML5 are very new, and there’s still a wide difference between browsers. Opera implemented part of the spec (it was called Webforms2 at that time) so it has decent support for HTML5 Forms, along with some quirks since the spec evolved since that implementation. WebKit-based browsers are currently implementing some parts of the spec so you’ll also find some early support there as well.

    We will not be adding more form features in Firefox 4, and there is clearly still some work to get full support for HTML5 forms. New field types (numbers, colors, dates), new attributes (step, min, max), new events (onforminput, onformchange) and so on. We’ll be adding support for more of HTML5 forms in later releases.

    This was just a rough introduction. To get all the details, you should go to the documentation on Mozilla Developer Network.

  6. HTML5 Video preload attribute supported in Firefox 4, autobuffer attribute removed

    This is a re-post from Chris Pearce’s blog. To comply with the HTML5 specification, we replaced the autobuffer attribute with the tri-state preload attribute. We encourage you to update your code. See the documention on MDC.

    Late last week I landed support on Firefox trunk for the HTML5 video ‘preload’ attribute. This replaces the ‘autobuffer’ attribute, which we previously supported. If you were previously using the autobuffer attribute on your HTLM5 videos, you need to update to using the preload attribute as well.

    The preload attribute provides a hint to the browser as to how much downloading is sensible for a given media. The preload attribute can have three values:

    1. none” – suggests to the browser that it doesn’t need to load this media at all until the user plays the resource. The browser will delay any network traffic required to load the media until the users tries to play the resource, or explicitly loads the resource. I suggest using this preload value, along with the poster attribute, when it’s unlikely that the user will play the resource. This is probably most useful in a mobile environment, where data can be expensive.
    2. metadata” – suggests to the browser that it isn’t necessary to load the entire resource in advance. The browser will suspend the load after loading metadata, displaying the first video frame (if there’s no poster image), and ensuring it can play the media. This is the default behaviour, and prevents excess network traffic when the web developer isn’t certain the video will definitely be played. If you don’t specify a preload value, Firefox will automatically do this. This was also the default behaviour in Firefox 3.5 and 3.6 in the absence of the autobuffer attribute. This default behaviour is a suitable compromise between bandwidth saving and user convenience.
    3. auto” – suggests to the browser that it should load as much of the resource as possible. As long as the browser’s own media cache isn’t full, it will keep on downloading. I suggest this is most useful in the “YouTube” case, when you’ve got a media which the user is almost certainly going to watch, and so having the user download the media is not likely to be wasting server bandwidth. This behaviour is the same as using the autobuffer attribute in Firefox 3.5 and 3.6

    Since the autobuffer attribute is no longer present in Firefox 4, and the preload attribute is not present in Firefox 3.5 and 3.6, if you want a media to download completely, you should include both preload=”auto” and autobuffer in the video element, e.g.:

    <video autobuffer preload=”auto” src=”video.ogg”></video>

    If you want the video to only be downloaded if the user actually plays the video, you should not include either the preload or the autobuffer attribute. The default behaviour in Firefox 3.5, 3.6, and the upcoming Firefox 4 is to only load up to the first frame and then suspend the download.

  7. Firefox 4: An early walk-through of IndexedDB

    Web developers already have localStorage, which is used for client side storage of simple key-value pairs. This alone doesn’t address the needs of many web applications for structured storage and indexed data. Mozilla is working on a structured storage API with indexing support called IndexedDB, and we will have some test builds in the next few weeks. This can be compared to the WebDatabase API implemented by several browsers that uses a subset of the allowable language of SQLite. Mozilla has chosen to not implement WebDatabase for various reasons discussed in this post.

    In order to compare IndexedDB and WebDatabase, we are going to show four examples that use most parts of the asynchronous APIs of each specification. The differences between SQL storage with tables (WebDatabase) and JavaScript object storage with indexes (IndexedDB) becomes pretty clear after reading the examples. The synchronous versions of these APIs are only available on worker threads. Since not all browsers currently implement worker threads, the synchronous APIs will not be discussed at this time. The IndexedDB code is based off a proposal that Mozilla has submitted to the W3C WebApps working group that has gotten positive feedback so far. The code for both APIs does not include any error handling (for brevity), but production code should always have it!

    These examples are for the storage of a candy store’s sale of candy to customers, which we’ll refer to as kids. Each entry in candySales represents a sale of a specified amount of candy to a kid, specified by an entry in candy and kids respectively.

    Example 1 – Opening and Setting Up a Database

    This first example demonstrates how to open a database connection and create the tables or object stores if the version number is not correct. Upon opening the database, both examples check the version and create the necessary tables or object stores and then set the correct version number. WebDatabase is a bit stricter in how it handles versions by giving an error if the database version is not what the caller expects (this is specified by the second argument to openDatabase). IndexedDB simply lets the caller handle versioning as they see fit. Note that there is active discussion about how IndexedDB should handle version changes in the working group.

    WebDatabase

    var db = window.openDatabase("CandyDB", "",
                                 "My candy store database",
                                 1024);
    if (db.version != "1") {
      db.changeVersion(db.version, "1", function(tx) {
        // User's first visit.  Initialize database.
        var tables = [
          { name: "kids", columns: ["id INTEGER PRIMARY KEY",
                                    "name TEXT"]},
          { name: "candy", columns: ["id INTEGER PRIMARY KEY",
                                     "name TEXT"]},
          { name: "candySales", columns: ["kidId INTEGER",
                                          "candyId INTEGER",
                                          "date TEXT"]}
        ];
     
        for (var index = 0; index < tables.length; index++) {
          var table = tables[index];
          tx.executeSql("CREATE TABLE " + table.name + "(" +
                        table.columns.join(", ") + ");");
        }
      }, null, function() { loadData(db); });
    }
    else {
      // User has been here before, no initialization required.
      loadData(db);
    }

    IndexedDB

    var request = window.indexedDB.open("CandyDB",
                                        "My candy store database");
    request.onsuccess = function(event) {
      var db = event.result;
      if (db.version != "1") {
        // User's first visit, initialize database.
        var createdObjectStoreCount = 0;
        var objectStores = [
          { name: "kids", keyPath: "id", autoIncrement: true },
          { name: "candy", keyPath: "id", autoIncrement: true },
          { name: "candySales", keyPath: "", autoIncrement: true }
        ];
     
        function objectStoreCreated(event) {
          if (++createdObjectStoreCount == objectStores.length) {
            db.setVersion("1").onsuccess = function(event) {
              loadData(db);
            };
          }
        }
     
        for (var index = 0; index < objectStores.length; index++) {
          var params = objectStores[index];
          request = db.createObjectStore(params.name, params.keyPath,
                                         params.autoIncrement);
          request.onsuccess = objectStoreCreated;
        }
      }
      else {
        // User has been here before, no initialization required.
        loadData(db);
      }
    };

    Example 2 – Storing Kids in the Database

    This example stores several kids into the appropriate table or object store. This example demonstrates one of the risks that have to be dealt with when using WebDatabase: SQL injection attacks. In WebDatabase explicit transactions must be used, but in IndexedDB a transaction is provided automatically if only one object store is accessed. Transaction locking is per-object store in IndexedDB. Additionally, IndexedDB takes a JavaScript object to insert, whereas with WebDatabase callers must bind specific columns. In both cases you get the insertion id in the callback.

    WebDatabase

    var kids = [
      { name: "Anna" },
      { name: "Betty" },
      { name: "Christine" }
    ];
     
    var db = window.openDatabase("CandyDB", "1",
                                 "My candy store database",
                                 1024);
    db.transaction(function(tx) {
      for (var index = 0; index < kids.length; index++) {
        var kid = kids[index];
        tx.executeSql("INSERT INTO kids (name) VALUES (:name);", [kid],
                      function(tx, results) {
          document.getElementById("display").textContent =
              "Saved record for " + kid.name +
              " with id " + results.insertId;
        });
      }
    });

    IndexedDB

    var kids = [
      { name: "Anna" },
      { name: "Betty" },
      { name: "Christine" }
    ];
     
    var request = window.indexedDB.open("CandyDB",
                                        "My candy store database");
    request.onsuccess = function(event) {
      var objectStore = event.result.objectStore("kids");
      for (var index = 0; index < kids.length; index++) {
        var kid = kids[index];
        objectStore.add(kid).onsuccess = function(event) {
          document.getElementById("display").textContent =
            "Saved record for " + kid.name + " with id " + event.result;
        };
      }
    };

    Example 3 – List All Kids

    This example lists all of the kids stored in the kids table or the kids object store. WebDatabase uses a result set object which will be passed to the callback method provided after all rows have been retrieved. IndexedDB, on the other hand, passes a cursor to the event handler as results are retrieved. Results should come back faster, as a result. While not shown in this example, you can also stop iterating data with IndexedDB by simply not calling cursor.continue().

    WebDatabase

    var db = window.openDatabase("CandyDB", "1",
                                 "My candy store database",
                                 1024);
    db.readTransaction(function(tx) {
      // Enumerate the entire table.
      tx.executeSql("SELECT * FROM kids", function(tx, results) {
        var rows = results.rows;
        for (var index = 0; index < rows.length; index++) {
          var item = rows.item(index);
          var element = document.createElement("div");
          element.textContent = item.name;
          document.getElementById("kidList").appendChild(element);
        }
      });
    });

    IndexedDB

    var request = window.indexedDB.open("CandyDB",
                                        "My candy store database");
    request.onsuccess = function(event) {
      // Enumerate the entire object store.
      request = event.result.objectStore("kids").openCursor();
      request.onsuccess = function(event) {
        var cursor = event.result;
        // If cursor is null then we've completed the enumeration.
        if (!cursor) {
          return;
        }
        var element = document.createElement("div");
        element.textContent = cursor.value.name;
        document.getElementById("kidList").appendChild(element);
        cursor.continue();
      };
    };

    Example 4 – List Kids Who Bought Candy

    This example lists all the kids, and how much candy each kid purchased. WebDatabase simply uses a LEFT JOIN query which makes this example very simple. IndexedDB does not currently have an API specified for doing a join between different object stores. As a result, the example opens a cursor to the kids object store and an object cursor on the kidId index on the candySales object store and performs the join manually.

    WebDatabase

    var db = window.openDatabase("CandyDB", "1",
                                 "My candy store database",
                                 1024);
    db.readTransaction(function(tx) {
      tx.executeSql("SELECT name, COUNT(candySales.kidId) " +
                    "FROM kids " +
                    "LEFT JOIN candySales " +
                    "ON kids.id = candySales.kidId " +
                    "GROUP BY kids.id;",
                    function(tx, results) {
        var display = document.getElementById("purchaseList");
        var rows = results.rows;
        for (var index = 0; index < rows.length; index++) {
          var item = rows.item(index);
          display.textContent += ", " + item.name + "bought " +
                                 item.count + "pieces";
        }
      });
    });

    IndexedDB

    candyEaters = [];
    function displayCandyEaters(event) {
      var display = document.getElementById("purchaseList");
      for (var i in candyEaters) {
        display.textContent += ", " + candyEaters[i].name + "bought " +
                               candyEaters[i].count + "pieces";
      }
    };
     
    var request = window.indexedDB.open("CandyDB",
                                        "My candy store database");
    request.onsuccess = function(event) {
      var db = event.result;
      var transaction = db.transaction(["kids", "candySales"]);
      transaction.oncomplete = displayCandyEaters;
     
      var kidCursor;
      var saleCursor;
      var salesLoaded = false;
      var count;
     
      var kidsStore = transaction.objectStore("kids");
      kidsStore.openCursor().onsuccess = function(event) {
        kidCursor = event.result;
        count = 0;
        attemptWalk();
      }
      var salesStore = transaction.objectStore("candySales");
      var kidIndex = salesStore.index("kidId");
      kidIndex.openObjectCursor().onsuccess = function(event) {
        saleCursor = event.result;
        salesLoaded = true;
        attemptWalk();
      }
      function attemptWalk() {
        if (!kidCursor || !salesLoaded)
          return;
     
        if (saleCursor && kidCursor.value.id == saleCursor.kidId) {
          count++;
          saleCursor.continue();
        }
        else {
          candyEaters.push({ name: kidCursor.value.name, count: count });
          kidCursor.continue();
        }
      }
    }

    IndexedDB generally simplifies the programming model for interacting with databases, and allows for a wide number of use cases. The working group is designing this API so it could be wrapped by JavaScript libraries; for instance, there’s plenty of room for a CouchDB-style API on top of our IndexedDB implementation. It would also be very possible to build a SQL-based API on top of IndexedDB (such as WebDatabase). Mozilla is eager to get developer feedback about IndexedDB, particularly since the specification has not been finalized yet. Feel free to leave a comment here expressing your thoughts or leave anonymous feedback through Rypple.

  8. Beyond HTML5: Database APIs and the Road to IndexedDB

    IndexedDB is an evolving web standard for the storage of significant amounts of structured data in the browser and for high performance searches on this data using indexes. Mozilla has submitted substantial technical feedback on the specification, and we plan to implement it in Firefox 4. We spoke to prominent web developers about evolving an elegant structured storage API for the web. While versions of Safari, Chrome, and Opera support a technology called Web SQL Database, which uses SQL statements as string arguments passed to a JavaScript API, we think developer aesthetics are an important consideration, and that this is a particularly inelegant solution for client-side web applications. We brought developer feedback to the editor of the IndexedDB specification, and also spoke with Microsoft, who agree with us that IndexedDB is a good option for the web. With additional implementations from the Chrome team in the offing, we think it is worth explaining our design choices, and why we think IndexedDB is a better solution for the web than Web SQL Database.

    Web applications can already take advantage of localStorage and sessionStorage in IE 8+, Safari 4+, Chrome 4+, Opera 10.5+ and Firefox 2+ to store key-value pairs with a simple JavaScript API. The Web Storage standard (encompassing localStorage and sessionStorage), now widely implemented, is useful for storing smaller amounts of data, but less useful for storing larger amounts of structured data. While many server-side databases use SQL to programmatically store structured data and to meaningfully query it, on the client-side, the use of SQL in a JavaScript API has been contentious.

    SQL? Which SQL?

    Many web developers certainly are familiar with SQL, since many developers touch just as much server-side code (e.g. PHP and database operations) as client-side code (e.g. JavaScript, CSS, and markup). However, despite the ubiquity that SQL enjoys, there isn’t a single normative SQL standard that defines the technology. In particular, SQLite supports most of SQL-92, with some notable omissions, and is what the WebDatabase API is based on. But SQLite itself isn’t a specification — it’s a release-ready technology! And the best definition of what constitutes the supported subset of SQL that SQLite uses is the SQLite manual. In order to really get Web SQL Database right, we’d have to first start with defining a meaningful subset of SQL for web applications. Why define a whole other language, when more elegant solutions exist within JavaScript itself?

    The Benefits and Pitfalls of SQLite

    We think SQLite is an extremely useful technology for applications, and make it available for Firefox extensions and trusted code. We don’t think it is the right basis for an API exposed to general web content, not least of all because there isn’t a credible, widely accepted standard that subsets SQL in a useful way. Additionally, we don’t want changes to SQLite to affect the web later, and don’t think harnessing major browser releases (and a web standard) to SQLite is prudent. IndexedDB does not have this problem; even though our underlying implementation of IndexedDB may be based on SQLite, we keep developers insulated from changes to SQLite by exposing an API that isn’t based on SQLite’s supported syntax.

    Aesthetics and Web Developers

    Last year, we held a summit at the Mozilla campus to discuss storage on the web. We invited web developers to speak to us about a desirable structured storage API on the web. Many did express resigned acceptance of a SQLite-based API, since they had already experimented with releases of Web SQL Database in some browsers, and claimed that something in circulation was better than a collection of ideas. Yet, all voiced enthusiasm for better design choices, and how a simpler model would make life easier for them. We watched as developers whiteboarded a simple BTree API that addressed their application storage needs, and this galvanized us to consider other options. We were resolved that using strings representing SQL commands lacked the elegance of a “web native” JavaScript API, and started looking at alternatives. Along with Microsoft, we sent feedback about the IndexedDB proposal and actively became involved in the standardization effort.

    In another article, we compare IndexedDB with Web SQL Database, and note that the former provides much syntactic simplicity over the latter. IndexedDB leaves room for a third-party JavaScript library to straddle the underlying primitives with a BTree API, and we look forward to seeing initiatives like BrowserCouch built on top of IndexedDB. Intrepid web developers can even build a SQL API on top of IndexedDB. We’d particularly welcome an implementation of the Web SQL Database API on top of IndexedDB, since we think that this is technically feasible. Starting with a SQL-based API for use with browser primitives wasn’t the right first step, but certainly there’s room for SQL-based APIs on top of IndexedDB.

    We want to continue the discussion with web developers about storage on the web, since that helps us structure our thoughts about product features and future web standards. We look forward to seeing the next generation of web applications with support for high performance searches on indexed data, and to seeing web applications work even more robustly in “airplane mode.”

    Links

  9. upcoming changes to the viewport meta tag for firefox mobile

    This is a guest post by Matt Brubeck who works on the Firefox Mobile team.

    The upcoming release of Mobile Firefox (Fennec) 1.1 features improved
    support for the <meta name="viewport"> tag. Previous version of Fennec supported the width, height, and initial-scale viewport properties, but had problems with some sites designed for iPhone and Android browsers. We now support the same properties Mobile Safari does, and we also changed Fennec to render mobile sites more consistently on screens of different sizes and resolutions.

    touch.facebook.com before:

    touch.facebook.com after:

    You can see these changes for yourself in the latest Fennec 1.1 and trunk nightly builds for Maemo, Android, Windows, Mac, or Linux.

    Background

    Mobile browers like Fennec render pages in a virtual “window” (the viewport), usually wider than the screen, so they don’t need to squeeze every page layout into a tiny window (which would break many non-mobile-optimized sites). Users can pan and zoom to see different areas of the page.

    Mobile Safari introduced the “viewport meta tag” to let web developers control the viewport’s size and scale. Many other mobile browsers now support this tag, although it is not part of any web standard. Apple’s documentation does a good job explaining how web developers can use this tag, but we had to do some detective work to figure out exactly how to implement it in Fennec. For example, Safari’s documentation says the content is a “comma-delimited list,” but existing browsers and web pages use any mix of commas, semicolons, and spaces as separators.

    Viewport basics

    A typical mobile-optimized site contains something like the following:

    <meta name="viewport"
     content="width=device-width, initial-scale=1, maximum-scale=1"/>

    The width property controls the size of the viewport. It can be set to a specific number of pixels like width=600 or to the special value device-width value which is the width of the screen in CSS pixels at a scale of 100%. (There are corresponding height and device-height values, which may be useful for pages with elements that change size or position based on the viewport height.)

    The initial-scale property controls the zoom level when the page is first loaded. The maximum-scale, minimum-scale, and user-scalable properties control how users are allowed to zoom the page in or out.

    A pixel is not a pixel

    The iPhone and many popular Android phones have 3- to 4-inch (7–10 cm) screens with 320×480 pixels (~160 dpi). Firefox for Maemo runs on the Nokia N900, which has the same physical size but 480×800 pixels (~240 dpi). Because of this, the last version of Fennec displayed many pages about one third smaller (in actual, physical size) than iPhone or Android. This caused usability and readability problems on many touch-optimized web sites. Peter-Paul Koch wrote about this problem in A pixel is not a pixel.

    Fennec 1.1 for Maemo will use 1.5 hardware pixels for each CSS “pixel”, following the lead of Android’s WebKit-based browser. This means a page with initial-scale=1 will render at close to the same physical size in Fennec for Maemo, Mobile Safari for iPhone, and the Android Browser on both HDPI and MDPI phones. This is consistent with the CSS 2.1 specification, which says:

    If the pixel density of the output device is very different from that of a typical computer display, the user agent should rescale pixel values. It is recommended that the pixel unit refer to the whole number of device pixels that best approximates the reference pixel. It is recommended that the reference pixel be the visual angle of one pixel on a device with a pixel density of 96dpi and a distance from the reader of an arm’s length.

    For web developers, this means that 320px be full width in portrait mode at scale=1, on all of the above-mentioned handheld devices, and they may size their layouts and images accordingly. But remember that not all mobile devices are the same width; you should also make sure that your pages work well in landscape mode, and on larger devices like the iPad and Android tablets.

    On 240-dpi screens, pages with initial-scale=1 will effectively be zoomed to 150% by both Fennec and Android WebKit. Their text will be smooth and crisp, but their bitmap images will probably not take advantage of the full screen resolution. To get sharper images on these screens, web developers may want to design images – or whole layouts – at 150% of their final size (or 200%, to support the rumored 320-dpi iPhone) and then scale them down using CSS or viewport properties.

    Right now Fennec uses the same default ratio of 1.5 on all devices. (It’s a hidden preference that can be changed in about:config or by an add-on.) Later we’ll need to change this – as well as many other parts of Fennec’s user interface – to work correctly on screens with different pixel densities. Note that the default ratio of 1.5 is true only when the viewport scale equals 1. Otherwise, the relationship between CSS pixels and device pixels depends on the current zoom level.

    Viewport width and screen width

    Many sites set their viewport to "width=320, initial-scale=1" to fit precisely onto the iPhone display in portrait mode. As mentioned above, this caused problems when Fennec 1.0 endered these sites, especially in landscape mode. To fix this, Fennec 1.1 will expand the viewport width if necessary to fill the screen at the requested scale. This matches the behavior of Android and Mobile Safari, and is especially useful on large-screen devices like the iPad. (Allen Pike’s Choosing a viewport for iPad sites has a good explanation for web developers.)

    For pages that set an initial or maximum scale, this means the width property actually translates into a minimum viewport width. For example, if your layout needs at least 500 pixels of width then you can use the following markup. When the screen is more than 500 pixels wide, the browser will expand the viewport (rather than zoom in) to fit the screen:

    <meta name="viewport" content="width=500, initial-scale=1"/>

    Fennec 1.1 also adds support for minimum-scale, maximum-scale, and user-scalable, with defaults and limits similar to Safari’s. These properties affect the initial scale and width, as well as limiting changes in zoom level.

    Mobile browsers handle orientation changes slightly differently. For example, Mobile Safari often just zooms the page when changing from portrait to landscape, instead of laying out the page as it would if originally loaded in landscape. If web developers want their scale settings to remain consistent when switching orientations on the iPhone, they must add a maximum-scale value to prevent this zooming, which has the sometimes-unwanted side effect of preventing users from zooming in:

    <meta name="viewport" content="initial-scale=1, maximum-scale=1">

    This is not necessary in Fennec; when the device changes orientation, Fennec updates the viewport size, the page layout, and JavaScript/CSS properties like device-width, based on its new “window” dimensions.

    Standards

    There is clearly demand for the viewport meta tag, since it is supported by most popular mobile browsers and used by thousands of web sites. It would be good to have a true standard for web pages to control viewport properties. According to the HTML5 spec, extensions to the meta element should first be registered on the WHATWG wiki and then go through the W3C standards process. If this happens, then we at Mozilla will work to make sure we can implement any changes made during standardization.

  10. Firefox, YouTube and WebM

    Five important items of note today relating to Mozilla’s support for the VP8 codec:

    1. Google will be releasing VP8 under an open source and royalty-free basis. VP8 is a high-quality video codec that Google acquired when they purchased the company On2. The VP8 codec represents a vast improvement in quality-per-bit over Theora and is comparable in quality to H.264.

    2. The VP8 codec will be combined with the Vorbis audio codec and a subset of the Matroska container format to build a new standard for Open Video on the web called WebM. You can find out more about the project at its new site: http://www.webmproject.org/.

    3. We will include support for WebM in Firefox. You can get super-early WebM builds of Firefox 4 pre-alpha today. WebM will also be included in Google Chrome and Opera.

    4. Every video on YouTube will be transcoded into WebM. They have about 1.2 million videos available today and will be working through their back catalog over time. But they have committed to supporting everything.

    5. This is something that is supported by many partners, not just Google and others. Content providers like Brightcove have signed up to support WebM as part of a full HTML5 video solution. Hardware companies, encoding providers and other parts of the video stack are all part of the list of companies backing WebM. Even Adobe will be supporting WebM in Flash. Firefox, with its market share and principled leadership and YouTube, with its video reach are the most important partners in this solution, but we are only a small part of the larger ecosystem of video.

    We’re extremely excited to see Google joining us to support Open Video. They are making technology available on terms consistent with the Open Web and the W3C Royalty-Free licensing terms. And – most importantly – they are committing to support a full open video stack on the world’s largest video site. This changes the landscape for video and moves the baseline for what other sites have to do to maintain parity and keep up with upcoming advances in video technology, not to mention compatibility with the set of browsers that are growing their userbase and advancing technology on the web.

    At Mozilla, we’ve wanted video on the web to move as fast as the rest of the web. That has required a baseline of open technology to build on. Theora was a good start, but VP8 is better. Expect us to start pushing on video innovation with vigor. We’ll innovate like the web has, moving from the edges in, with dozens of small revolutions that add up to something larger than the sum of those parts. VP8 is one of those pieces, HTML5 is another. If you watch this weblog, you can start to see those other pieces starting to emerge as well. The web is creeping into more and more technologies, with Firefox leading the way. We intend to keep leading the web beyond HTML5 to the next place it needs to be.

    Today is a day of great change. Tomorrow will be another.