Mozilla

HTML5 Articles

Sort by:

View:

  1. HTML5 audio and audio sprites – this should be simple

    As we’re having a HTML5 Audio developer derby this month, I thought it fun to play with audio again. And I found it sadly enough pretty frustrating.

    One thing I proposed in a lot of talks is using the idea of CSS sprites and apply them to HTML5 audio. You’ll get the same benefits – loading one file in one HTTP request instead of many, avoiding failure as files might not get loaded and so on.

    To test this out I wrote the following small demo using the awesome Music Non Stop by Kraftwerk.

    Clicking the different buttons should play the part of the music file and nothing more. This works fine in Firefox, Chrome and Opera on my computer here. Safari, however, fails to preload the audio and the setting of the current time is off. The code is simple enough that this should work:

    <div id="buttons"></div>
    <audio preload controls>
      <source src="boing-boomchack-peng.mp3" type="audio/mp3"></source>
      <source src="boing-boomchack-peng.ogg" type="audio/ogg"></source>
    </audio>
    // get the audio element and the buttons container
    // define a sprite object with the names and the start and end times
    // of the different sounds.
    var a = document.querySelector('audio'),
        buttoncontainer = document.querySelector('#buttons'),
        audiosprite = {
          'all': [ 0, 5 ],
          'boing': [ 0, 1.3 ],
          'boomtchack': [ 2, 2.5 ],
          'peng': [ 4, 5 ]
        },
        end = 0;
     
    // when the audio data is loaded, create the buttons
    // this way non-HTML5 browsers don't get any buttons
    a.addEventListener('loadeddata', function(ev) {
      for (var i in audiosprite) {
        buttoncontainer.innerHTML += '<button onclick="play('' +
                                      i + '')">' + i + '</button>';
      }
    }, false);
     
    // If the time of the file playing is updated, compare it
    // to the current end time and stop playing when this one
    // is reached
    a.addEventListener('timeupdate', function(ev) {
      if (a.currentTime > end) {
        a.pause();
      }
    },false);
     
    // Play the current audio sprite by setting the currentTime
    function play(sound) {
      if ( audiosprite[sound] ) {
        a.currentTime = audiosprite[sound][0];
        end = audiosprite[sound][1];
        a.play();
      }
    }

    Now, this is nothing new, Remy Sharp wrote about audio sprites in 2010 and lamented especially the buggy support in iOS (audio won’t load at all until you activate it with a touch – something that sounds horribly like the “click to active” Flash has on IE).

    Other issues are looping and latency of HTML5 audio. As reported by Robert O’Callahan there is a work-around by cloning the audio element before playing it (with an incredibly annoying test) and this fix has been used in the Gladius HTML5 game engine.

    All in all it seems HTML5 audio still needs a lot of work which is why a lot of Games released lately under the banner of HTML5 use Flash audio or no audio at all. This is sad and needs fixing.

    Interestingly enough there are some great projects that you could be part of. Are we playing yet? by Soundcloud and others for example is a test suite for audio support in browsers. You can write own tests on GitHub and report results to the browser makers.

    The jPlayer team has a great HTML5 Media Event Inspector showing just how many of the HTML5 media events are supported in your current browser.

    If you want to be safe, you can use SoundManager 2 by Scott Schiller to have an API that uses HTML5 when possible and falls back to Flash when the browser doesn’t have any support. It also fixes a few issues for you.

    Speaking of Scott Schiller, he continually gives good insight on the state of audio. There is a 51 minute video of his article on 24 ways “Probably, Maybe, No: The State of HTML5 Audio“.

    A shorter and more recent talk on the same subject is also available:

    All in all it would be interesting to hear what you think of the state of HTML5 audio:

    • Did the companies that heralded HTML5 as the end of plugins drop the ball?
    • Is it really sensible to have an API that returns probably or maybe or ” when you ask it if the browser can play a certain type of media?
    • What could be done to work around these issues?

    Let’s re-ignite the discussion on HTML5 audio, after all we need it for the future of messaging in the browser and telephony, too.

    Oh and another thing. Of course there is the Audio Data API of Firefox and the web audio proposal from Webkit available but getting those running in mobile devices will be a much bigger change. If you want to know more about those and libraries to work around their differences, there is a great overview post available on Happyworm.

  2. People of HTML5: Andrew Betts on building the FT.com HTML5 app

    HTML5 needs spokespeople to work. There are a lot of people out there who took on this role, and here at Mozilla we thought it is a good idea to introduce some of them to you with a series of interviews and short videos. The format is simple – we send the experts 10 questions to answer and then do a quick video interview to let them introduce themselves and ask for more detail on some of their answers.

    Andrew Betts Today we are featuring Andrew Betts, director of Assanka.

    We came across Andrew because of his talk “We’ve Got a Website for That – The FT Web App and the Future of the Mobile Web” at the BlackBerry Devcon in Amsterdam where he did a great job allowing us to peek under the hood of the Financial Times web app. You can reach Andrew on Twitter as @triblondon.

    The video interview

    You can see the video on any HTML5 enabled device here (courtesy of vid.ly).

    10 Questions for Andrew Betts

    1) You worked on the HTML5 application for the financial times. What made you go HTML5 and not native? How many people were involved at all?

    The FT has an agnostic approach when it comes to the formats and technologies we can use to display our content. The choice has to suit the content, the reader, and our business model. Before we launched the HTML5 app there was already an FT native app on iOS, and it received an Apple design award, but HTML5 offered several significant benefits, the most important of which was to allow us to maintain a direct relationship with the reader. Secondary benefits include a faster update process, an investment in future cross-platform compatibility (HTML5 is yet to fully realise the dream of write-once, run everywhere), and freedom from restrictions and rules that are (or may be in future) imposed by app store operators.

    2) What would you say was the easiest part and what was the thing that you had to spend the most time on to make it work?

    Interactions are the hardest thing to crack. Matching behaviour to user expectations is frighteningly complex, especially when you realise that instinctive expectations of swipe behaviour is actually quite varied and inconsistent. One example of this complexity is swiping horizontally from the bottom of a section page, which takes you to the top of the next one. If you then swipe back, do you end up where you were, or at the top? So dealing with interactions is both technically hard and architecturally challenging.

    Offline behaviour is also very tough. We get a lot of “User x can’t use the app offline” style support requests, and those are very hard to debug. We log every report but the priority is to figure out the scale of the problem.

    The easiest thing to deal with is probably layout. We send all legacy browsers to m.ft.com so we only have to cope with the most recent major versions, and that means that generally we get good standards support for CSS.

    3) How did you deal with the differences in interfaces? A mobile has much less space than a tablet and again less than a desktop. Are you using mediaqueries to give different experiences?

    We group devices into one of four sizes based on media queries, and we vary which resources we use slightly based on the reported DPI, but most of our responsive layout logic is done in JavaScript. We make a few assumptions right now to make development easier, such as that the user cannot change the size of the browser window, except by rotating the device, so we only need to change the layout on orientation change, not in response to arbitrary resizing of the window. We also currently assume the user has no mouse cursor, so there are no hover effects.

    4) How does input get into it? Is it simple enough to build touch interfaces in HTML5 or did you have to use some trickery?

    Quite a lot of trickery is involved, unfortunately. The browser has to tread a fine line between on the one hand offering a touch API to web developers and getting out of their way to let them use it, and on the other, dealing with the fact that almost all websites are still built for keyboard and mouse, so the browser needs to help the user deal with that. We developed a polyfill called Fastclick which makes click events fire as fast as touch events, and that certainly helps us provide an experience that is as snappy as using a native app.

    We also deal with swiping in a few quite distinctly different ways. Swiping between sections is an endless carousel, and we implement the tracking and sliding ourselves, by maintaining hidden page containers to the left and right of the current view and applying CSS 3D transforms. But paging through an article requires different scroll mechanics – it has a fixed length, and we have to lay out the whole article in one go to know how many pages there are. Pages that scroll vertically are a different challenge again, and there we use a combination of leaving the browser to do scrolling natively, and simulating it where we need slightly different behaviour.

    You’ll also notice that the pages don’t ‘bounce’ when you hit the top and bottom of the page, despite that being the normal behaviour in the iOS browser. We’ve gone to quite some lengths to ensure that the user experience mirrors an app and doesn’t feel like the device ‘coping’ with a site that hasn’t been designed to be viewed using a device with touch input.

    5) You said in your talk at Blackberry Devcon that you are using localStorage to cache your JavaScript libraries and some of the images. Why not use WebSQL or IndexDB for that? Isn’t the synchronous nature of localStorage slowing things down?

    No, actually localStorage is considerably faster than WebSQL (we are yet to use IndexedDB because it’s not supported on some of our target platforms). The time required for a lookup by key in localStorage is many times less than the equivalent SELECT statement on a WebSQL table. It’s true that it’s synchronous, but if the critical aspect affecting perceived performance is the time required to fetch data, which it often is, then localStorage offers a much faster response. it’s also typically reliable and relatively stable compared to the behaviour of WebSQL and particularly the HTML5 app cache.

    6) You also said that localStorage uses UTF-16 instead of UTF-8. This means you can store two ASCII characters in one UTF-16 one and thus double the storage capacity, right? Can you share some code? This would be incredible to use for everybody out there.

    Both localStorage and WebSQL use UTF-16 in webkit browsers, it’s one of many aspects of the implementation of these technologies that don’t make a lot of sense! We’re experimenting with some algorithms for compressing our data so we can make more efficient use of that storage, and one of those is to combine two characters into a single UTF-16 one. We’re not using that in production yet, but when we do we’re certainly hoping to open source it.

    There are simpler things to be done as well. A cursory poke and prod of WebSQL tables suggest that column names are stored along with the value in every cell, so some attempts to increase efficiency are as simple as using single character column names. Clearly we ought not to have to do all this, but it’s the price you pay for dealing with bleeding edge features in browsers.

    7) What would you consider the best start for someone when they want to build an HTML5 app? What are the first things to get sorted?

    Decide ahead of time what you’re trying to achieve. I could be a pedant and say that making your site HTML5 is really just a matter of changing the DOCTYPE, so an ‘HTML5 app’ is really whatever you define it to be, and having a clear idea of that is a good place to begin. Are you building something just for touch, or for keyboard and mouse as well? Does it need to work offline? Will it be crawlable by search spiders? Will it work with JavaScript disabled?

    There are some great resources like MDN, but we often find ourselves reading the HTML5 specs, and sometimes even the webkit source code to find out how something is actually implemented.

    8) Can native and web apps be the same thing?

    Technically, I guess not – strictly speaking a native app is complied code running directly on a platform, but the hybrid model seems to be getting ever more popular, and I often see apps which are thinly veiled web browsers. If you look at it from the user perspective and say, can a web app feel exactly like a native app, I don’t see why not. The challenges are greater because of the diversity of uses that web technologies are designed for, and the diversity of platforms on which they are used. And for uses such as gaming, native code is always likely to run faster and have deeper access to the OS.

    But for applications like publishing, which after all was the original purpose of the web, web technologies do provide support for most of the user experience concepts that we need.

    9) What about testing? How did you approach this? Do you have an array of hardware to play with?

    It’s a nightmare. We had a local electrician built us a charging station where we could store all our devices and keep them charged all the time. We have around 45 individual devices in our team (that’s around 4 times more devices than we have people). We’re constantly looking at ways of improving our testing process, and we keep revisiting automated, on device testing, but we’re not using it in our build cycle as yet. Right now we have a team of very dedicated testers who poke and prod devices all day.

    10) As someone who went through the process of building a big HTML5 app, what would you like to have from browser vendors to make your life easier?

    A blithe answer is ‘How long do you have’, but in practice we have to accept we’re using freshly minted technologies and there might be teething troubles. The main things on our shopping list are: more frequent and aggressive browser updates (especially on Android, where the browser is the problem child of the mobile web world), a better and more reliable app cache, and hardware acceleration of CSS transforms.

  3. HTML5: The difference between an App and a Page.

    HTML5 is only one part of the”Stack”

    HTML5 is really more than one thing. In the strictest sense, HTML5 is fifth major revision of the W3C specification of the markup language that is used to create web pages.

    But in a practical sense, HTML5 is far more than that. For developers, HTML is a wave of technologies that are evolving and gaining browser implementations along roughly the same time-line as the markup language itself. Many of the related technologies are being spearheaded by the Web Hypertext Application Technology Working Group (whatwg.org), but still other technologies in this genre are driven from elsewhere. For example WebGL, the 3D rendering engine for Web Apps, is driven at Khronos, and Mozilla’s WebAPI team is leading a collection of relevant API work including a comprehensive array of device-specific APIs. And the W3C was a Device APIs Working Group

    Mozilla is also investing heavily in an initiative to facilitate the use of standards-based Web technologies to build applications that can be installed and used in the same way a “native” applications would be.

    There are many reasons that doing so might be desirable, not the least of which is the ability to build much or all of your application for use on many different devices and platforms using a single set of development technologies.

    Developers can already build HTML Applications (with help from the HTML5 Apps documentation ) and submit their applications to the Mozilla Marketplace.

    App versus Page ?

    When people start talking about building apps with HTML5, one of the early questions that comes to the conversation is “What’s the difference between an app and a page?”

    This is bit of a difficult question to answer because of the very wide range of answers which can all be true, but which may be completely different.

    While your HTML5 app should be more than a simple HTML5 page, technically speaking it doesn’t have to be. Using the steps outlined in the guide “Getting Started with making (HTML5) apps”, you could simply create an apps manifest file and submit your HTML page as an app.

    When an Open Web app runs, the Mozilla Web Runtime installs local copies of the HTML5 markup and all of the other web assets that are specified using an HTML5 Cache Manifest ( read here ).

    If a Web page is dynamically generated, as in an ASP.NET, PHP, etc., application, any cached version represents a snapshot of a single point in time. When submitted as an app, the page actually “becomes” an app, getting a launcher item similar to native apps on whatever the host operating system is. When the app is launched, the local assets are used if there is no Internet connection or if the caching mechanism determines that the assets have not changed since they were last downloaded.

    By doing the above we have not added any additional functionality to our application. All we have done by declaring our page to be an app is to get the run-time to copy the bits locally and to run those local bits if the device is off line or if the bits haven’t changed.

    Does this get us anything ?

    Is there value in just this simple step ?

    Yes, it gets us three valuable improvements:

    • Users can see our page even if they are not connected because the run-time renders the cached version if there is no Internet connection.
    • Processor cycles and bandwidth consumption from our infrastructure or hosting are greatly reduced because new bits are only fetched when they are needed.
    • Performance improves because resources are loaded locally by default.

    Many organizations are starting with this approach because very little technical effort is necessary and it can be done very quickly. In most cases all you need to do is to create an app manifest file ( more info here ) and to submit  the app to the marketplace. Developers can submit apps now at the Mozilla marketplace Developer Submission site so that they will be available to consumers when the “buy side” of the marketplace opens later this year.)

    But if that’s all you do with your app, you are missing the opportunity.

    Take a look at http://caniuse.com/ for a snapshot view of technologies that are available to developers using the modern web runtime.

    Whether we are building a real Open Web App from scratch or we are converting an HTML page to an app and then adding modern features, HTML5 era technologies let us really start to developer client / server applications that utilize the browser and its host resources, thereby reducing our server requirements while improving the end user’s experience.

    Developers can do things like :

    • Read and write data locally and sync with a server when appropriate.
    • Run multiple threads of execution at the same time.
    • Exchange content with other applications, or share logic with them.
    • Interact with the device on which their app is running.
      • Touch user interface
      • GPS
      • Camera
      • Richly integrate multimedia.
      • And much more…..

    Since apps’ source code is deployed to a single URL and is updated on the device on which the app runs only when updates are necessary, versioning is low cost and friction free. This makes it easy to start by making a page and then iteratively adding features one after another.

    Lets do it !

    Since this model makes it so easy to get started, why not give at a try?

    Follow the guidance here : https://developer.mozilla.org/en/Apps/Getting_Started

    And submit your app here : https://marketplace.mozilla.org/

  4. BrowserQuest – a massively multiplayer HTML5 (WebSocket + Canvas) game experiment

    It’s time for some gaming action with a new HTML5 game demo: BrowserQuest, a massively multiplayer adventure game created by Little Workshop (@glecollinet & @whatthefranck) and Mozilla.

    Play the game: browserquest.mozilla.org
    BrowserQuest

    BrowserQuest is a tribute to classic video-games with a multiplayer twist. You play as a young warrior driven by the thrill of adventure. No princess to save here, just a dangerous world filled with treasures to discover. And it’s all done in glorious HTML5 and JavaScript.

    Even better, it’s open-source so be sure to check out the source code on GitHub!

    Watch a screencast:

    A multiplayer experience

    BrowserQuest screenshot

    BrowserQuest can be played by thousands of simultaneous players, distributed across different instances of the in-game world. Click on the population counter at any time to know exactly how many total players are currently online.

    Players can see and interact with each other by using an in-game chat system. They can also team up and fight enemies together.

    BrowserQuest is a game of exploration: the more dangerous the places you go, the better the rewards.

    Powered by WebSockets

    WebSockets are a new technology enabling bi-directional communication between a browser and a server on the web.

    BrowserQuest is a demo of how this technology can be used today to create a real-time multiplayer game in a single webpage. When you start to play, your browser opens up a WebSocket connection to one of several load-balanced game servers. Each server hosts multiple world instances and handles the player synchronization and game logic within all instances. Because the server code is running on Node.js, both the server and client codebases share a small portion of the same JavaScript source code.

    Server code is available on Github.

    BrowserQuest screenshot

    Built on the Web platform

    BrowserQuest makes extensive use of different web technologies, such as:

    • HTML5 Canvas, which powers the 2D tile-based graphics engine.
    • Web workers, allowing to initialize the large world map without slowing down the homepage UI.
    • localStorage, in which the progress of your character is continually saved.
    • CSS3 Media Queries, so that the game can resize itself and adapt to many devices.
    • HTML5 audio, so you can hear that rat or skeleton die!

    Available everywhere

    Since BrowserQuest is written in HTML5/JavaScript, it is available across a lot of different browsers and platforms. The game can be played in Firefox, Chrome and Safari. With WebSockets enabled, it’s also playable in Opera. Moreover, it’s compatible with iOS devices, as well as tablets and phones running Firefox for Android.

    BrowserQuest screenshot

    The mobile versions are more experimental than the desktop experience, which has richer features and performance, but it’s an early glimpse of what kind of games will be coming to the mobile Web in the future. Give it a try with your favorite mobile device!

    Join the adventure

    Want to be part of BrowserQuest? Create your own character and venture into the world. Fight enemies by yourself or with friends to get your hands on new equipment and items. You might even stumble upon a couple of surprises along the way…

  5. There is no simple solution for local storage

    TL;DR: we have to stop advocating localStorage as a great opportunity for storing data as it performs badly. Sadly enough the alternatives are not nearly as supported or simple to implement.

    When it comes to web development you will always encounter things that sound too good to be true. Sometimes they are good, and all that stops us from using them is our notion of being conspicuous about *everything* as developers. In a lot of cases, however, they really are not as good as they seem but we only find out after using them for a while that we are actually “doing it wrong”.

    One such case is local storage. There is a storage specification (falsely attributed to HTML5 in a lot of examples) with an incredibly simple API that was heralded as the cookie killer when it came out. All you have to do to store content on the user’s machine is to access the navigator.localStorage (or sessionStorage if you don’t need the data to be stored longer than the current browser session):

    localStorage.setItem( 'outofsight', 'my data' );
    console.log( localStorage.getItem( 'outofsight' ) ); // -> 'my data'

    This local storage solution has a few very tempting features for web developers:

    • It is dead simple
    • It uses strings for storage instead of complex databases (and you can store more complex data using JSON encoding)
    • It is well supported by browsers
    • It is endorsed by a lot of companies (and was heralded as amazing when iPhones came out)

    A few known issues with it are that there is no clean way to detect when you reach the limit of local storage and there is no cross-browser way to ask for more space. There are also more obscure issues around sessions and HTTPS, but that is just the tip of the iceberg.

    The main issue: terrible performance

    LocalStorage also has a lot of drawbacks that aren’t quite documented and certainly not covered as much in “HTML5 tutorials”. Especially performance oriented developers are very much against its use.

    When we covered localStorage a few weeks ago using it to store images and files in localStorage it kicked off a massive thread of comments and an even longer internal mailing list thread about the evils of localStorage. The main issues are:

    • localStorage is synchronous in nature, meaning when it loads it can block the main document from rendering
    • localStorage does file I/O meaning it writes to your hard drive, which can take long depending on what your system does (indexing, virus scanning…)
    • On a developer machine these issues can look deceptively minor as the operating system cached these requests – for an end user on the web they could mean a few seconds of waiting during which the web site stalls
    • In order to appear snappy, web browsers load the data into memory on the first request – which could mean a lot of memory use if lots of tabs do it
    • localStorage is persistent. If you don’t use a service or never visit a web site again, the data is still loaded when you start the browser

    This is covered in detail in a follow-up blog post by Taras Glek of the Mozilla performance team and also by Andrea Giammarchi of Nokia.

    In essence this means that a lot of articles saying you can use localStorage for better performance are just wrong.

    Alternatives

    Of course, browsers always offered ways to store local data, some you probably never heard of as shown by evercookie (I think my fave when it comes to the “evil genius with no real-world use” factor is the force-cached PNG image to be read out in canvas). In the internal discussions there was a massive thrust towards advocating IndexedDB for your solutions instead of localStorage. We then published an article how to store images and files in IndexedDB and found a few issues – most actually related to ease-of-use and user interaction:

    • IndexedDB is a full-fledged DB that requires all the steps a SQL DB needs to read and write data – there is no simple key/value layer like localStorage available
    • IndexedDB asks the user for permission to store data which can spook them
    • The browser support is not at all the same as localStorage, right now IndexedDB is supported in IE10, Firefox and Chrome and there are differences in their implementations
    • Safari, Opera, iOS, Opera Mobile, Android Browser favour WebSQL instead (which is yet another standard that has been officially deprecated by the W3C)

    As always when there are differences in implementation someone will come up with an abstraction layer to work around that. Parashuram Narasimhan does a great job with that – even providing a jQuery plugin. It feels wrong though that we as implementers have to use these. It is the HTML5 video debate of WebM vs. H264 all over again.

    Now what?

    There is no doubt that the real database solutions and their asynchronous nature are the better option in terms of performance. They are also more matured and don’t have the “shortcut hack” feeling of localStorage. On the other hand they are hard to use in comparison, we already have a lot of solutions out there using localStorage and asking the user to give us access to storing local files is unacceptable for some implementations in terms of UX.

    The answer is that there is no simple solution for storing data on the end users’ machines and we should stop advocating localStorage as a performance boost. What we have to find is a solution that makes everybody happy and doesn’t break the current implementations. This might prove hard to work around. Here are some ideas:

    • Build a polyfill library that overrides the localStorage API and stores the content in IndexedDB/WebSQL instead? This is dirty and doesn’t work around the issue of the user being asked for permission
    • Implement localStorage in an asynchronous fashion in browsers – actively disregarding the spec? (this could set a dangerous precedent though)
    • Change the localStorage spec to store asynchronously instead of synchronously? We could also extend it to have a proper getStorageSpace interface and allow for native JSON support
    • Define a new standard that allows browser vendors to map the new API to the existing supported API that matches the best for the use case?

    We need to fix this as it doesn’t make sense to store things locally and sacrifice performance at the same time. This is a great example of how new web standards give us much more power but also make us face issues we didn’t have to deal with before. With more access to the OS, we also have to tread more carefully.

  6. Presentation: HTML5 and friends at Mobile World Congress 2012

    As part of the WIP Jam at the Mobile World Congress in Barcelona, Spain we were asked to give and introduction to HTML5 so we sent Chris Heilmann and Joe Stagner to set the record straight in terms of what HTML5 is and what you can do with it.

    The presentation was on the last day of the conference and both of us had spent long hours explaining all Mozilla technologies on our booth and really not enough hours sleeping in between. In addition to that the room was a boiling 30 degrees and packed with lots of developers. It seems we did a good job regardless as nobody left and we had a lot of great questions afterwards.

    The slides (well, the HTML5 document) are available for you to see:

    Slides for HTML5 and friends

    However, they make much more sense when you see them in context, which is why we also provide you with a screencast of the talk:

    All in all it was a lot of fun talking to people at MWC and Boot to Gecko showed that HTML5 is the future of mobile as much as it is already the present technology of choice on the desktop.

  7. Mozilla's Boot to Gecko – The Web is the Platform

    Mozilla’s Boot to Gecko (B2G) is about building a complete, standalone operating system for the open web. It aims at making web technologies the number one choice for applications on desktop and mobile, and we believe it can displace proprietary, single-vendor stacks for application development. And we have made some exciting progress that we want to share with you!

    Continued…