Mozilla

DOM Articles

Sort by:

View:

  1. getUserMedia is ready to roll!

    We blogged about some of our WebRTC efforts back in April. Today we have an exciting update for you on that front: getUserMedia has landed on mozilla-central! This means you will be able to use the API on the latest Nightly versions of Firefox, and it will eventually make its way to a release build.

    getUserMedia is a DOM API that allows web pages to obtain video and audio input, for instance, from a webcam or microphone. We hope this will open the possibility of building a whole new class of web pages and applications. This DOM API is one component of the WebRTC project, which also includes APIs for peer-to-peer communication channels that will enable exchange of video steams, audio streams and arbitrary data.

    We’re still working on the PeerConnection API, but getUserMedia is a great first step in the progression towards full WebRTC support in Firefox! We’ve certainly come a long way since the first image from a webcam appeared on a web page via a DOM API. (Not to mention audio recording support in Jetpack before that.)

    We’ve implemented a prefixed version of the “Media Capture and Streams” standard being developed at the W3C. Not all portions of the specification have been implemented yet; most notably, we do not support the Constraints API (which allows the caller to request certain types of audio and video based on various parameters).

    We have also implemented a Mozilla specific extension to the API: the first argument to mozGetUserMedia is a dictionary that will also accept the property {picture: true} in addition to {video: true} or {audio: true}. The picture API is an experiment to see if there is interest in a dedicated mechanism to obtain a single picture from the user’s camera, without having to set up a video stream. This could be useful in a profile picture upload page, or a photo sharing application, for example.

    Without further ado, let’s start with a simple example! Make sure to create a pref named “media.navigator.enabled” and set it to true via about:config first. We’ve put the pref in place because we haven’t implemented a permissions model or any UI for prompting the user to authorize access to the camera or microphone. This release of the API is aimed at developers, and we’ll enable the pref by default after we have a permission model and UI that we’re happy with.

    There’s also a demo page where you can test the audio, video and picture capabilities of the API. Give it a whirl, and let us know what you think! We’re especially interested in feedback from the web developer community about the API and whether it will meet your use cases. You can leave comments on this post, or on the dev-media mailing list or newsgroup.

    We encourage you to get involved with the project – there’s a lot of information about our ongoing efforts on the project wiki page. Posting on the mailing list with your questions, comments and suggestions is great way to get started. We also hang out on the #media IRC channel, feel free to drop in for an informal chat.

    Happy hacking!

  2. DOM MutationObserver – reacting to DOM changes without killing browser performance.

    DOM Mutation Events seemed like a great idea at the time – as web developers create a more dynamic web it seems natural that we would welcome the ability to listen for changes in the DOM and react to them. In practice however DOM Mutation Events were a major performance and stability issue and have been deprecated for over a year.

    The original idea behind DOM Mutation Events is still appealing, however, and so in September 2011 a group of Google and Mozilla engineers announced a new proposal that would offer similar functionality with improved performance: DOM MutationObserver. This new DOM Api is available in Firefox and Webkit nightly builds, as well as Chrome 18.

    At it’s simplest, a MutationObserver implementation looks like this:

    // select the target node
    var target = document.querySelector('#some-id');
     
    // create an observer instance
    var observer = new MutationObserver(function(mutations) {
        mutations.forEach(function(mutation) {
            console.log(mutation.type);
        });    
    });
     
    // configuration of the observer:
    var config = { attributes: true, childList: true, characterData: true }
     
    // pass in the target node, as well as the observer options
    observer.observe(target, config);
     
    // later, you can stop observing
    observer.disconnect();

    The key advantage to this new specification over the deprecated DOM Mutation Events spec is one of efficiency. If you are observing a node for changes, your callback will not be fired until the DOM has finished changing. When the callback is triggered, it is supplied a list of the changes to the DOM, which you can then loop through and choose to react to.

    This also means that any code you write will need to process the observer results in order to react to the changes you are looking for. Here is a compact example of an observer that listens for changes in an editable ordered list:

    <!DOCTYPE html>
    <ol contenteditable oninput="">
      <li>Press enter</li>
    </ol>
    <script>
      var MutationObserver = window.MutationObserver || window.WebKitMutationObserver || window.MozMutationObserver;
      var list = document.querySelector('ol');
     
      var observer = new MutationObserver(function(mutations) {  
        mutations.forEach(function(mutation) {
          if (mutation.type === 'childList') {
            var list_values = [].slice.call(list.children)
                .map( function(node) { return node.innerHTML; })
                .filter( function(s) {
                  if (s === '<br>') {
                    return false;
                  }
                  else {
                    return true;
                  }
            });
            console.log(list_values);
          }
        });
      });
     
      observer.observe(list, {
      	attributes: true, 
      	childList: true, 
      	characterData: true 
       });
    </script>

    If you want to see this code running, I’ve put it up on jsbin here:

    http://jsbin.com/ivamoh/53/edit

    If you play with the live example, you’ll notice some quirks in behaviour, in particular that the callback is triggered when you press enter in each li, in particular when the user action results in a node being added or removed from the DOM. This is an important distinction to be made from other techniques such as binding events to key presses or more common events like ‘click’. MutationObservers work differently from these techniques because they are triggered by changes in the DOM itself, not by events generated either via JS or user interaction.

    So what are these good for?

    I don’t expect most JS hackers are going to run out right now and start adding mutation observers to their code. Probably the biggest audience for this new api are the people that write JS frameworks, mainly to solve problems and create interactions they could not have done previously, or at least not with reasonable performance. Another use case would be situations where you are using frameworks that manipulate the DOM and need to react to these modifications efficiently ( and without setTimeout hacks! ).

    Another common use of the Dom Mutation Events api is in browser extensions, and in the next week or so I’m going to publish a follow-up post on how MutationObservers are particularly useful when interacting with web content in a Firefox Add-on.

    Resources

  3. Announcing Firefox Aurora 10

    We’re happy to announce the availability of Aurora 10.
    (Download and Test Aurora 10)

    In additional to the normal improvements that you’ve come to expect like performance, security and bug fixes, Aurora 10 focuses in HTML5 enhancements.

    New additions

    Developer Tools

    Aurora 10 also implements incremental enhancements like IndexedDB setVersion API changes. Ongoing detailed attention to evolving specifications help to keep Firefox at the front of the Web revolution. (Read more about IndexedDB on MDN.)

    DOM

    • We now fire a “load” event on stylesheet linking when the sheet load finishes or “error” if the load fails.
    • We turn the POSTDATA prompt into an information page (when navigating in session history).
    • We only forward event attributes on body/frameset to the window if we also forward the corresponding on* property.
    • We no longer allow more than one call to window.open() when we allow popups.
    • We fixed a bug where a success callback never fired when a position update is triggered after getCurrentPosition().
    • We removed replaceWholeText().
    • We fixed an error with createPattern(zero-size canvas).
    • We now handle putImageData(nonfinite) correctly.
    • We now throw INVALID_STATE_ERR when dispatching uninitialized events.
    • We’ve made Document.documentURI readonly.
    • We fixed document.importNode to comply with optional argument omitted.

    Web workers

    • We now allow data URLs.
    • We implemented event.stopImmediatePropagation in workers.
    • We made XHR2 response/responseType work in Web Workers.

    Graphics

    • We implement the WebGL OES_standard_derivatives extension.
    • We implement minimal-capabilities WebGL mode.

    JavaScript

    • The function caller property no longer skips over eval frames.
    • We fixed E4X syntax so that it is not accepted in ES5 strict mode.
    • weakmap.set no longer returns itself instead of undefined.
    • We implemented the battery API.

    Offline: IndexedDB enhancements

    • IndexedDB setVersion API changes
    • Added support for IDBObjectStore/IDBIndex.count
    • Various methods accept both keys and KeyRanges.
    • Added support for IDBCursor.advance.
    • Implemented deleteDatabase.
    • objectStoreNames are no longer updated on closed databases when another connection adds or removes object stores
    • IDBObjectStore.delete and IDBCursor.delete now return undefined.
    • No longer throws an error if there are unknown properties in the options objects to createObjectStore/createIndex.
    • We now the errorCode to “ABORT_ERR” for all pending requests when IDBTransaction.abort() is called.
    • Fixed the sort order for indexes.

    Layout

    • We have updated the current rule for handling malformed media queries.
    • We now support the HTML5 <bdi> element and CSS property unicode-bidi: isolate.
    • The CSS3 implementation now supports unicode-bidi: plaintext.

    Media

    • Implemented Document.mozFullScreenEnabled.
    • Enabled the DOM full-screen API on desktop Firefox by default.
  4. Tagging docs for sprint at JSConf.eu October 1-2

    We’re very excited to announce that Mozilla is sponsoring the Hacker Lounge at JSConf.eu and we will be holding a doc sprint at and during the conference. The focus of this doc sprint will naturally be docs for JavaScript and DOM. We hope to encourage attendees at the conference to contribute at least a little to improving the JS and DOM docs on MDN. I and a handful of MDN community members will be there to show them how.

    To that end, we want to tag as many JS and DOM articles as possible, with tags indicating what work needs to be done on those articles. That way, anybody dropping in during the sprint (or anytime later) can look for tagged articles and find something to work on.

    Here are some of the tags we use to indicate that an article needs help:

    • NeedsTechnicalReview: needs someone to verify that the technical information is complete and correct.
    • NeedsExample: needs one or more illustrative code examples of the item documented.
    • NeedsContent: the item is incomplete and needs to be filled out.
    • NeedsJSVersion: needs information about the version of JavaScript and EcmaScript this item first appears in.
    • NeedsBrowserCompatibility: needs a browser compatibility table or needs the table filled out.
    • MakeBrowserAgnostic: the article is written with a focus on Gecko, when it is actually about a standard function or feature, which should be rewritten to be generic.

    Please help by tagging articles in MDN that need work with the appropriate tag. You can use the Talk page for each article to elaborate on what needs to be done, if the tag is not descriptive enough. To modify tags on an article, login to MDN and click Edit Tags at the bottom of the page.

    The wiki page for the doc sprint has links to queries for some of these tags.

    If you will be at JSConf.eu, I look forward to seeing you there! If you will be participating remotely, I’ll see you online!

  5. Animating with javascript: from setInterval to requestAnimationFrame

    Animating DOM elements[1] or the content of a canvas is a classical use case for setInterval. But the interval is not as reliable as it seems, and a more suitable API is now available…

    Animating with setInterval

    To animate an element moving 400 pixels on the right with javascript, the basic thing to do is to move it 10 pixels at a time on a regular interval.

    JSFiddle demo.

    An HTML5 game based on this logic would normally run at ~60fps[2], but if the animations were too complex or running on a low-spec. device (a mobile phone for instance) and processing a frame were taking more than 16ms, then the game would run at a lower framerate: when processing 1 frame takes 33ms, the game runs at 30fps and game elements move twice as slowly as they should. Animations would still look smooth enough, but the game experience would be altered.

    Animating at constant speed

    To animate at constant speed, we need to calculate the time delta since the last frame and move the element proportionally.

    Animating with requestAnimationFrame

    Since the interval parameter is irrelevant in complex animations, as there’s no guarantee that it will be honored, a new API has been designed: requestAnimationFrame. It’s simply a way to tell the browser “before drawing the next frame on the screen, execute this game logic/animation processing”. The browser is in charge of choosing the best moment to execute the code, which results in a more efficient use of resources[3].

    Here’s how an animation with requestAnimationFrame would be written.
    Note: Following code snippets don’t include feature detections and workarounds necessary to work in current browsers. Should you want to play with them, you should try the ready-to-use animLoop.js.

    Dealing with inactive tabs

    requestAnimationFrame was built with another benefit in mind: letting the browser choose the best frame interval allows to have a long interval in inactive tabs. Users could play a CPU intensive game, then open a new tab or minimize the window, and the game would pause[4], leaving resources available for other tasks.
    Note: the potential impact of such behavior on resource and battery usage is so positive that browser vendors decided to adopt it for setTimeout and setInterval as well[5].

    This behavior also means that the calculated time delta might be really high when switching back to a tab containing an animation. This will result in animation appearing to jump or creating “wormholes[6], as illustrated here.

    Wormholes can be fixed by clamping the time delta to a maximum value, or not rendering a frame when the time delta is too high.

    JSFiddle demo.

    Problems with animation queues

    Libraries such as jQuery queue animations on elements to execute them one after the other. This queue is generally only used for animations that are purposefully consecutive.
    But if animations are triggered by a timer, the queue might grow without bound in inactive tabs, as paused animations stack up in the queue. When switching back to affected tabs, a user will see a large number of animations playing consecutively when only one should happen on a regular interval:

    JSFiddle demo.

    This problem is visible in some auto-playing slideshows such as mb.gallery. To work around it, developers can empty animation queues before triggering new animations[7].
    JSFiddle demo.

    Conclusion

    The delays of setTimeout and setInterval and of course requestAnimationFrame are unpredictable and much longer in inactive tabs. These facts should be taken into account not only when writing animation logic, but in fps counters, time countdowns, and everywhere time measurement is crucial.

    [1] The DOM can now be animated with CSS3 Transitions and CSS3 Animations.
    [2] 1 frame every 16ms is 62.5 frames per second.
    [3] See the illustration of this fact on msdn.
    [4] The behavior of requestAnimationFrame in inactive tabs is still being worked on at the w3c, and might differ in other browsers.
    [5] See related Firefox bug and related chromium bug.
    [6] This term was first coined by Seth Ladd in his “Intro to HTML5 Game Development” talk.
    [7] See documentation of your js library, such as effects and stop() for jQuery.

  6. A call for quality HTML5 demo markup

    HTML5 is a necessary evolution to make the web better. Before the HTML5 specs were created we used (and still use) a hacked together bunch of systems meant for describing and linking documents to create applications. We use generic elements to simulate rich interaction modules used in desktop development and we make assumptions as to what the user agent (browser) can do for the end user.

    The good thing about this mess is that it taught us over the last few years to be paranoid in our development approach – we realised that in order to deliver working, accessible, maintainable and scalable systems we have to be professional and intelligent in our decisions and especially in our planning and architecting.

    The trouble is that with the excitement around the cool new HTML5 things to play with a lot of these principles get lost and discarded as outdated. They aren’t – part of the mess that the web is in is that in 1999 a lot of people discarded everything else and instead worked only on the new hotness that was Internet Explorer 6 and DHTML. Let’s not repeat that mistake.

    The two faces of HTML5 enthusiasm

    Right now there are two things to get really excited about in HTML5: the richer, more meaningful semantics of new elements and the APIs that give us deep access into the workings of the browser and the operating system. The former allows us to build much richer interfaces and the latter allows us to build much more complex applications.

    All of this comes with the awesome of view source (or check in development tools) for debugging. You can advocate HTML5 by writing clean and useful markup. You can kill it by treating the markup as a second class citizen which is only there to serve the script.

    The markup enthusiasts are very happy about HTML5 and make it easy for people to upgrade – lots of cool new blog templates and boilerplate HTML templates are being built and polyfills created to allow using the new features and not leave behind outdated browsers.

    On the app development side of things it looks much different and that is just sad. My only possible explanation is that people who come from desktop environments now tackle the HTML5 APIs without ever having to care about markup. The pragmatism of HTML5 allows a much more forgiving syntax than XHTML but it shouldn’t mean that we can just re-apply all the bad mistakes that held us back when it comes to maintenance for years.

    During my career as a web developer I realised a few things make a lot of sense when building web apps:

    • If there is an element for a certain task – use that one. It is very likely that the element comes with accessibility and interaction features for free that you would otherwise have to simulate.
    • Separate CSS, JavaScript and HTML – which means it is easy to refactor your code without having to change all of them. It also means you can work in parallel with others instead of breaking each other’s code
    • Never rely on markup or content – as sooner or later some editing tool will come into place that messes with everything you created

    This means a lot of things:

    • For starters it means that inline styles are simply evil as they override any settings you have in your style sheets. Only use them when this is exactly what you need to do or when you calculate them dynamically.
    • The same applies to inline scripting. If you have an onclick="foo()" somewhere in your HTML and foo() changes to bar() you have to rename it in every HTML document (of course nowadays it is one template, but it still means hunting down a reference you might miss)
    • If instead of using a native HTML element for a certain job you use SPANs and DIVs you’ll have to add classes to make them look and work – and simulate the keyboard accessibility, too.
    • You can’t rely on the text value of any element. A <button>edit</button> using the “edit” as the trigger for certain functionality would have to have the JS localised, too when you create the German <button>bearbeiten</button>.

    Bla bla bla… C’mon Chris, it isn’t that bad!

    The above best practices have been mentioned for years and a lot of people get sick of seeing them repeated. After all, this is intelligent development and standard practice in backend technologies. I came across a lot of “creative” uses lately though when doing a view-source on HTML5 demos – specifically the ones in the HTML5 advent calendar. And here is my yield of what people do.

    Simulating a navigation list

    One of the first things I encountered was a painting tool that had a gallery of created graphics as a background. Now, to me this would be a simple list:

    <ul>
      <li>
        <a href="{link}">
          <img src="{src}" alt="{title}">
        </a>
      </li>
      [...]
    </ul>

    The markup I found though was this:

    <div id="sky">
      <div class="skyTile" style="{calculated styles};display:block">
        <img class="skyThumb" src="{src}" style="{generated styles}">
        <img src="images/thumb_dropShadow.png" 
             class="skyThumbShadow" 
             style="{generated styles}">
      </div>
      [...]
    </div>

    This, of course is generated by the backend. My first gripe is the dropshadow image, after all this is an HTML5 showcase – just use CSS3. We also have the three instances of generated styles and double classes. Granted, an extra class gives you a handle to all images instead of tiles, so why not. But as there is no real link around the image, the click handler has to read the url from somewhere. There is a lot of unnecessary back and forth between DOM and scripting here which does slow down the whole demo. Seeing that this is also the main navigation from the entry page to the editor this could be a list inside a nav element. A list groups these items together, a bunch of DIVs doesn’t (screen readers for example tell you how many items there are in a list).

    Another list I found was supposed to be links to refresh the app and have a hierarchy but instead was a flat list with classes to define hierarchy and group:

    <ul id="mitems">
      <li class="hedtop">Simple</li>
      <li class="mol">{name}</li>
      [... repeated ...]
      <li class="hed">{parent name}</li>
      <li class="mol">{name}</li>
      [... repeated ...]
    </ul>

    This could be OK, granted you also shift keyboard focus, but why not:

    <nav>
    <ul id="mitems">
      <li>{parent name}
        <ul>
          <li><a href="{url}">{name}</a></li>
          [... repeated ...]
        </ul>
      </li>
      [... repeated ...]
    </ul>
    </nav>

    This would give you styling hooks and functionality for free. Links and buttons are great to trigger functionality – but it seems that is too easy.

    Click (probably) here for functionality

    If I build a button for an application to trigger a certain functionality this is the markup:

    <button type="button" class="edit">Edit</button>

    Buttons are what trigger functionality – both a backend script or a JavaScript. They are keyboard accessible, have a disabled state and sensible out-of-the-box styling that nowadays can be completely overwritten. The class can be the definition of what the button should do – instead of the text which will change. You could use an ID but a class allows to repeat buttons (for example on the top and the bottom of a results list).

    The buttons I found though were slightly different:

    <div id="homeButtonPanel">
      <div class="homeButton" id="drawHomeButton" 
           style="display: block;">
        <p>Start Drawing</p>
      </div>
      <div class="homeButton" id="viewHomeButton" 
           style="display: block;">
        <p>View the Mural</p>
      </div>
    </div>
     
    <div id="controlPanel" style="display: block;">
    <div class="titleButton"><p>{title}</p></div>
      <div class="mainMenu">
        <div class="mainButton mainSelected" id="drawButton">
          <p>Draw</p>
          <div id="inkDisplay">
            <div id="inkMeter" style="width: 94px;">
            </div>
          </div>
        </div>
        <div class="menuContainer">
          <div class="menuButton drawMenuButton" id="behaviorsButton"
               style="display: block; opacity: 1;">
            <p>Gesture<br>Behaviors</p>
          </div>
        </div>
        <div class="menuContainer">
          <div class="menuButton drawMenuButton" id="artworkButton" 
               style="display: block; opacity: 1;">
            <p>Artwork</p>
          </div>
        </div>
        <div class="menuContainer">
          <div class="menuButton drawMenuButton" id="resetButton" 
               style="display: block; opacity: 1;">
            <p>Reset</p>
          </div>
        </div>
        <div class="menuContainer">
          <div class="menuButton drawMenuButton" id="undoButton" 
               style="display: block; opacity: 1;">
            <p>Undo</p>
          </div>
        </div>
        <div class="menuContainer">
          <div class="menuButton drawMenuButton" id="saveButton" 
               style="display: block; opacity: 1;">
            <p>Save</p>
          </div>
        </div>
      </div>
      <div class="mainMenu">
        <div class="mainButton" id="viewButton"><p>View</p></div>
      </div>
      <div class="mainMenu">
        <div class="secondaryButton" id="helpButton"><p>Help</p></div>
      </div>
      <div class="mainMenu">
        <div class="mainShare shareButtonSmall" id="mainTwitterButton">
          <img alt="twitter" src="images/twitter_icon.png" 
               style="opacity: 0.6;">
        </div>
        <div class="mainShare shareButtonLarge" id="mainFacebookButton">
          <img alt="facebook" src="images/facebook_icon.png">
        </div>
        <div class="mainShare shareButtonSmall" id="mainEmailButton">
          <img alt="email" src="images/email_icon.png" style="opacity: 0.6;">
        </div>
      </div>
    </div>

    So instead of using a nested list with classes for each button and the hierarchy in the nesting we have a lot of classes and a hand-rolled DIV construct. Instead of making buttons really disabled we rely on opacity and there is quite a mix of inline images and background images (if all were background images, they could be one sprite!). Keyboard navigation will have to be written for this and if you were to add a button you’d have to come up with another ID.

    HTML5 actually has a construct for this. There is a menu element with command child elements. These are in use in Chrome’s side bar for example and should be what we use. If you want to make it work for everyone, a nested list with button elements is what to go for.

    The overly complex DIV construct is quite used though – this was another example:

    <div class="uiScreen" id="startScreen">
      <div class="panelBackground" id="mainResumeButton" style="display: none;">
        <div class="largeUIButton" id="resumeButton">Resume Game</div>
      </div>
      <div class="panelBackground" id="mainStartButton">
        <div class="largeUIButton" id="startButton">Start Game</div>
      </div>		
      <div class="panelBackground" id="mainHelpButton">
        <div class="largeUIButton" id="helpButton">Help</div>
      </div>
      <div class="panelBackground" id="mainHighscoreButton">
        <div class="largeUIButton" id="highscoreButton">High Scores</div>
      </div>
    </div>

    When in doubt – add an ID and class to everything.

    Other buttons I encountered were actually links pointing to javascript:// using an inline style to indicate hierarchy:

    <ul class="navlist">
      <li><a id="play" href="javascript://" 
             style="display: inline;">Play</a></li>    
      <li><a id="pause" href="javascript://">Pause</a></li>
      <li style="padding-left: 2px;">Effects:</li>
      <li><a id="bw" href="javascript://">Bw</a></li>
      <li><a id="outline" href="javascript://">Outline</a></li>
      <li><a id="color" href="javascript://">Color</a></li>
      <li><a id="invert" href="javascript://">Invert</a></li>
      <li><a id="matrix" href="javascript://">Matrix</a></li>
      <li><a id="old" href="javascript://">Old</a></li>
    </ul>

    Talking of inline – here’s a great example of a tool generating a lot of code that could be replaced by a single event handler and event delegation:

    <div id="tools">
      <span onmouseout="buttonOut(1)" onmouseover="buttonOver(1)"
            onclick="buttonClicked(1)" id="button1" class="button">
        <img alt="" src="image/button/1.png">
      </span>
      <span onmouseout="buttonOut(2)" onmouseover="buttonOver(2)"
            onclick="buttonClicked(2)" id="button2" class="button">
            <img alt="" src="image/button/2.png">
      </span>
     
      [...repeated 20 times...]
     
    <div id="toolsSeparator">&nbsp;</div>
      <a title="" id="toolbarButtonSave" class="toolbarButton"
         href="javascript:saveCanvas()">
        <img alt="" src="image/save.png">
      </a>
      <a title="New" id="toolbarButtonNew" class="toolbarButton"
         href="javascript:newCanvas()">
        <img alt="New" src="image/new.png"></a>
      <!--[if !IE]><![IGNORE[--><!--[IGNORE[]]-->
      <a id="toolbarButtonMenu" class="toolbarButton"
         onmouseout="closeMenuDelay()" onmouseover="showMenuHover()"
         href="javascript:showMenu()">
        <img alt="&gt;" src="image/menu.png">
      </a>
      <div onmouseout="closeMenuDelay()" onmouseover="overMenu()" id="menu">
        <a class="saveMenu" onmouseout="closeMenuDelay()" 
           onmouseover="overMenu()" href="javascript:saveCanvas()">
          save normal
        </a>
        <a class="saveMenu" onmouseout="closeMenuDelay()" 
           onmouseover="overMenu()" href="javascript:saveCanvasHi()">
          save high quality
          <span class="footnote">&nbsp; (rename to *.png)</span>
        </a>
        <a onmouseout="closeMenuDelay()" onmouseover="overMenu()"
           href="javascript:showAbout()">
          about...
        </a>
        <a class="lastMenu" target="_top" onmouseout="closeMenuDelay()"
           onmouseover="overMenu()" 
           href="{url}">
          <span class="footnote">
            <em>elsewhere:&nbsp;</em>
          </span>
          <em>a sound memory game</em>
        </a>
      </div>
      <!--<![endif]-->
    </div>

    Notice that if the images for the button couldn’t be loaded for one reason or another (or you can’t see them at all) this application is very confusing indeed – no alternative text for the images and no text content to fall back to. I am also very much sure that the in and out handlers trigger visual effects CSS could handle better.

    Reasons and effects

    I know there are probably good reasons for all of this, and I am sure I will also do things wrongly when I am rushed or want to get things out quickly. What we have to understand though is that right now we have a responsibility to show the best of breed demos we can.

    We cannot preach the open web and technologies and view-source over closed systems and at the same time forget the things we learnt in the last decade. Some of these things I found look like code Frontpage or Dreamweaver could have produced in the 90ies and resulted in a lot of badly performing, hard to maintain products that either still annoy people who have to use them or get replaced every 2 years.

    We have a mandate to educate the new developers coming to the web. Unlearning something is much harder than learning it – so let’s not start with bloat and quick fixes that work but forget to advocate clean code and thinking about the impact your product has on the end users (thinking accessibility) and the poor sods that will have to maintain your product when you are gone. We are not here to advocate effects and products, we are here to promote the tools that allow anyone to easily build something cool that is also easy to understand.

    HTML5 is about evolving the web as a platform – we also need to evolve with it and take more responsibility. We have app and we have markup enthusiasts. Let’s make them work together to build things that are great functionality and clean semantics.

  7. HTML5 adoption stories: box.net and html5 drag and drop

    This is a guest post from Tomas Barreto, a developer who works at box.net. They recently adopted HTML5 drag and drop as a way to share files with other people using new features in Firefox. The included video is a pitch for the feature and service, but shows how easy it is to do simple HTML5-based upload progress even with multiple files. Tomas gives an overview of the relatively simple JavaScript required to do this, and how improvements in Firefox 4 will make things even easier. Also have a quick look at the bottom of the post for links to additional docs and resources.

    At Box.net, we’re always exploring new ways to help users get content quickly and securely onto our cloud content management platform. So when asked, “What feature would make you use Box more?” during the Box Hack Olympics in April, my colleague CJ and I decided to tackle the most intuitive way to upload files: simply dragging them from the desktop into Box.

    We considered technologies ranging from Gears to Firefox plugins, but only HTML5 had sufficient adoption. By using some of the JavaScript APIs defined in the HTML5 standard, CJ and I could create a seamless drag and drop experience for our users on supporting browsers. Furthermore, using an HTML5-based upload feature would allow us to enable users to select multiple files at once, and also display progress on the client without polling. And with HTML5 adoption across the latest versions of three of the top four browsers, we felt confident about building an upload method based on this new technology without the trade-offs of using a third-party plug-in.

    We rolled out the first rev of our drag and drop feature a few weeks ago, and we’re impressed with how quickly it has been adopted. It’s already one of the most popular ways to get files onto Box, and in its first week it surpassed our old AJAX upload method. You can check out our demo video to get a feel for the feature:

    To build this feature, we referenced a handful of online examples that explained how to use Firefox 3 FileReader object and the drag and drop file event support. Our first implementation used this object to load the file into memory and then took advantage of the latest XMLHttpRequest events to track progress on the client.

    var files = event.originalEvent.dataTransfer.files; // drop event
    var reader = new FileReader();
     
    reader.onload = function(event) {
      var file_contents = event.target.result;
      var request = new XMLHttpRequest();
     
      ... // attach event listeners to monitor progress and detect errors
     
      var post_body = '';
     
      .. // build post body
     
      post_body += file_contents;
     
      .. // finish post body
     
      var url = 'http://example.com/file_upload';
     
      var request = new XMLHttpRequest();
     
      request.open("POST",  url, true); // open asynchronous post request
      request.setRequestHeader('content-type', 'multipart/form-data; boundary=""'); // make sure to set a boundary
      request.send(post_body);
    }
     
    reader.readAsBinaryString(files[0]);

    This approach worked well because we could use the same server processing code that we previously used for uploads. The main disadvantage here is that the FileReader object reads the entire file into memory, which is not optimal for a general upload use case. Our current HTML5 implementation uses this logic and has forced us to restrict drag and drop uploads to just 25mb. However, thanks to recommendations from the Mozilla team, we’ll be taking an alternative approach for V2 of drag and drop, where the file is read chunks as needed by the request. Here’s how we’re going to do it:

    var files = event.originalEvent.dataTransfer.files; // drop event
    var url = 'http://example.com/file_upload';
     
    var request = new XMLHttpRequest();
    request.open("POST",  url, true); // open asynchronous post request
    request.send(files[0]);

    Since this approach is not formatted as a multipart form-data, it will require some adjustments on our back-end to support receiving file uploads in this way. However, it’s definitely worth the trade-off since we’ll get all the benefits of the previous method and we don’t need special file size restrictions. In the future, we’ll consider using yet another way to efficiently upload files that is supported in Firefox 4 and uses the traditional multi-part form:

    var files = event.originalEvent.dataTransfer.files; // drop event
    var url = 'http://example.com/file_upload';
     
    var request = new XMLHttpRequest();
     
    var fd = new FormData;
    fd.append("myFile", files[0]);
     
    request.open("POST",  url, true); // open asynchronous post request
    request.send(fd);

    We’re already exploring more ways to enrich the Box experience using HTML5. With HTML5, we can build faster, richer and more interactive features with native browser support, and bridge the traditional gap between desktop software and web applications. Here are just a few cool new upload-related features on our roadmap:

    • Pause/Resume uploads using the Blob slice API to split files into chunks (this will be a huge robustness boost, especially for large uploads)
    • Allowing uploads to resume even after the browser closes by caching the file using IndexedDB support (possibly in Firefox 4)

    We’d also like to begin a discussion about supporting the reverse drag and drop use case: dragging files from the browser to the desktop. Based on our users’ enthusiasm around the drag and drop upload feature, we think the reverse functionality would well received. If you are interested in contributing to a specification for this feature, please let us know (html5 [-at$] box.net)!

    Resources:

  8. Firefox 4: Better performance with Lazy Frame Construction

    This is a re-post from Timothy Nikkel’s blog.

    Lazy Frame Construction is new to Gecko and allows many DOM operations (appendChild, insertBefore, etc) to not trigger immediate reflows. This can vastly improve the interactive performance of very complex web pages. If you want to test this out, you should get a Firefox Nightly.

    Lazy frame construction recently landed on mozilla-central. To explain what this means and how this improves things we need some background. Each node in the DOM tree of a webpage has a frame created for it that is used to determine where on the page the node is drawn and its size. A frame corresponds closely to the concept of a box from the CSS spec. We used to create frames for DOM nodes eagerly; that is as soon as a node was inserted into the document we would create a frame for it. But this can create wasted effort in many situations. For example if a script inserts a large number of nodes into the DOM we would create a frame for each node when it is inserted. But with lazy frame construction we can process all those nodes at once in a big batch, saving overhead. Furthermore the time it takes to create those frames no longer blocks that script, so the script can go and do what it needs to and the frames will get created when they are needed. There are other situations where a script would insert nodes into the document and remove them immediately, so there is no need to ever create a frame for these as they would never be painted on screen.

    So now when a node is inserted into a document the node is flagged for needing a frame created for it, and then the next time the refresh driver notifies (currently at 20 ms intervals) the frame is created. The refresh driver is also what drives reflow of webpages and CSS & SVG animations.

    Let’s look at two examples where lazy frame construction helps.

    In this example we insert 80000 div elements and then we flush all pending layout to time how long it takes before the changes made by the script are done and visible to the user. The script can continue executing without flushing layout, but we do it here to measure how long the actual work takes.

    var stime = new Date();
    var container = document.getElementById("container");
    var lastchild = document.getElementById("lastchild");
    for (var i = 0; i < 80000; i++) {
      var div = document.createElement("div");
      container.insertBefore(div, lastchild);
    }
    document.documentElement.offsetLeft; // flush layout
    var now = new Date();
    var millisecondselapsed = (now.getTime() - stime.getTime());

    With lazy frame construction we are able to process the insertion of all 80000 div elements in one operation, saving the overhead of 80000 different inserts. In a build without lazy frame construction I get an average time of 1358 ms, with lazy frame construction I get 777 ms.

    This example comes from a real webpage. We append a div and then set “div.style.position = ‘absolute’;”, and repeat that 2000 times, and then we flush all pending layout to time how long it takes before the changes made by the script are done and visible to the user.

    var stime = new Date();
    var container = document.getElementById("container2");
    for (var i = 0; i < 2000; i++) {
      var div = document.createElement("div");
      container.appendChild(div);
      div.style.position = "absolute";
    }
    document.documentElement.offsetLeft; // flush layout
    var now = new Date();
    var millisecondselapsed = (now.getTime() - stime.getTime());

    With lazy frame construction we don't even bother creating the frame for the div until after the position has been set to absolute, so we don't waste any effort. In a build without lazy frame construction I get an average time of 4730 ms, with lazy frame construction I get 130 ms.