Mozilla

DOM Articles

Sort by:

View:

  1. The state of Web Components

    Web Components have been on developers’ radars for quite some time now. They were first introduced by Alex Russell at Fronteers Conference 2011. The concept shook the community up and became the topic of many future talks and discussions.

    In 2013 a Web Components-based framework called Polymer was released by Google to kick the tires of these new APIs, get community feedback and add some sugar and opinion.

    By now, 4 years on, Web Components should be everywhere, but in reality Chrome is the only browser with ‘some version’ of Web Components. Even with polyfills it’s clear Web Components won’t be fully embraced by the community until the majority of browsers are on-board.

    Why has this taken so long?

    To cut a long story short, vendors couldn’t agree.

    Web Components were a Google effort and little negotiation was made with other browsers before shipping. Like most negotiations in life, parties that don’t feel involved lack enthusiasm and tend not to agree.

    Web Components were an ambitious proposal. Initial APIs were high-level and complex to implement (albeit for good reasons), which only added to contention and disagreement between vendors.

    Google pushed forward, they sought feedback, gained community buy-in; but in hindsight, before other vendors shipped, usability was blocked.

    Polyfills meant theoretically Web Components could work on browsers that hadn’t yet implemented, but these have never been accepted as ‘suitable for production’.

    Aside from all this, Microsoft haven’t been in a position to add many new DOM APIs due to the Edge work (nearing completion). And Apple, have been focusing on alternative features for Safari.

    Custom Elements

    Of all the Web Components technologies, Custom Elements have been the least contentious. There is general agreement on the value of being able to define how a piece of UI looks and behaves and being able to distribute that piece cross-browser and cross-framework.

    ‘Upgrade’

    The term ‘upgrade’ refers to when an element transforms from a plain old HTMLElement into a shiny custom element with its defined life-cycle and prototype. Today, when elements are upgraded, their createdCallback is called.

    var proto = Object.create(HTMLElement.prototype);
    proto.createdCallback = function() { ... };
    document.registerElement('x-foo', { prototype: proto });

    There are five proposals so far from multiple vendors; two stand out as holding the most promise.

    ‘Dmitry’

    An evolved version of the createdCallback pattern that works well with ES6 classes. The createdCallback concept lives on, but sub-classing is more conventional.

    class MyEl extends HTMLElement {
      createdCallback() { ... }
    }
    
    document.registerElement("my-el", MyEl);

    Like in today’s implementation, the custom element begins life as HTMLUnknownElement then some time later the prototype is swapped (or ‘swizzled’) with the registered prototype and the createdCallback is called.

    The downside of this approach is that it’s different from how the platform itself behaves. Elements are ‘unknown’ at first, then transform into their final form at some point in the future, which can lead to developer confusion.

    Synchronous constructor

    The constructor registered by the developer is invoked by the parser at the point the custom element is created and inserted into the tree.

    class MyEl extends HTMLElement {
      constructor() { ... }
    }
    
    document.registerElement("my-el", MyEl);

    Although this seems sensible, it means that any custom elements in the initial downloaded document will fail to upgrade if the scripts that contain their registerElement definition are loaded asynchronously. This is not helpful heading into a world of asynchronous ES6 modules.

    Additionally synchronous constructors come with platform issues related to .cloneNode().

    A direction is expected to be decided by vendors at a face-to-face meeting in July 2015.

    is=””

    The is attribute gives developers the ability to layer the behaviour of a custom element on top of a standard built-in element.

    <input type="text" is="my-text-input">

    Arguments for

    1. Allows extending the built-in features of a element that aren’t exposed as primitives (eg. accessibility characteristics, <form> controls, <template>).
    2. They give means to ‘progressively enhance’ an element, so that it remains functional without JavaScript.

    Arguments against

    1. Syntax is confusing.
    2. It side-steps the underlying problem that we’re missing many key accessibility primitives in the platform.
    3. It side-steps the underlying problem that we don’t have a way to properly extend built-in elements.
    4. Use-cases are limited; as soon as developers introduce Shadow DOM, they lose all built-in accessibility features.

    Consensus

    It is generally agreed that is is a ‘wart’ on the Custom Elements spec. Google has already implemented is and sees it as a stop-gap until lower-level primitives are exposed. Right now Mozilla and Apple would rather ship a Custom Elements V1 sooner and address this problem properly in a V2 without polluting the platform with ‘warts’.

    HTML as Custom Elements is a project by Domenic Denicola that attempts to rebuild built-in HTML elements with custom elements in an attempt to uncover DOM primitives the platform is missing.

    Shadow DOM

    Shadow DOM yielded the most contention by far between vendors. So much so that features had to be split into a ‘V1′ and ‘V2′ agenda to help reach agreement quicker.

    Distribution

    Distribution is the phase whereby children of a shadow host get visually ‘projected’ into slots inside the host’s Shadow DOM. This is the feature that enables your component to make use of content the user nests inside it.

    Current API

    The current API is fully declarative. Within the Shadow DOM you can use special <content> elements to define where you want the host’s children to be visually inserted.

    <content select="header"></content>

    Both Apple and Microsoft pushed back on this approach due to concerns around complexity and performance.

    A new Imperative API

    Even at the face-to-face meeting, agreement couldn’t be made on a declarative API, so all vendors agreed to pursue an imperative solution.

    All four vendors (Microsoft, Google, Apple and Mozilla) were tasked with specifying this new API before a July 2015 deadline. So far there have been three suggestions. The simplest of the three looks something like:

    var shadow = host.createShadowRoot({
      distribute: function(nodes) {
        var slot = shadow.querySelector('content');
        for (var i = 0; i < nodes.length; i++) {
          slot.add(nodes[i]);
        }
      }
    });
    
    shadow.innerHTML = '<content></content>';
    
    // Call initially ...
    shadow.distribute();
    
    // then hook up to MutationObserver

    The main obstacle is: timing. If the children of the host node change and we redistribute when the MutationObserver callback fires, asking for a layout property will return an incorrect result.

    myHost.appendChild(someElement);
    someElement.offsetTop; //=> old value
    
    // distribute on mutation observer callback (async)
    
    someElement.offsetTop; //=> new value

    Calling offsetTop will perform a synchronous layout before distribution!

    This might not seems like the end of the world, but scripts and browser internals often depend on the value of offsetTop being correct to perform many different operations, such as: scrolling elements into view.

    If these problems can’t be solved we may see a retreat back to discussions over a declarative API. This will either be in the form of the current <content select> style, or the newly proposed ‘named slots’ API (from Apple).

    A new Declarative API – ‘Named Slots’

    The ‘named slots’ proposal is a simpler variation of the current ‘content select’ API, whereby the component user must explicitly label their content with the slot they wish it to be distributed to.

    Shadow Root of <x-page>:

    <slot name="header"></slot>
    <slot></slot>
    <slot name="footer"></slot>
    <div>some shadow content</div>
    

    Usage of <x-page>:

    <x-page>
      <header slot="header">header</header>
      <footer slot="footer">footer</footer>
      <h1>my page title</h1>
      <p>my page content<p>
    </x-page>

    Composed/rendered tree (what the user sees):

    <x-page>
      <header slot="header">header</header>
      <h1>my page title</h1>
      <p>my page content<p>
      <footer slot="footer">footer</footer>
      <div>some shadow content</div>
    </x-page>

    The browser has looked at the direct children of the shadow host (myXPage.children) and seen if any of them have a slot attribute that matches the name of a <slot> element in the host’s shadowRoot.

    When a match is found, the node is visually ‘distributed’ in place of the corresponding <slot> element. Any children left undistributed at the end of this matching process are distributed to a default (unamed) <slot> element (if one exists).

    For:
    1. Distribution is more explicit, easier to understand, less ‘magic’.
    2. Distribution is simpler for the engine to compute.
    Against:
    1. Doesn’t explain how built-in elements, like <select>, work.
    2. Decorating content with slot attributes is more work for the user.
    3. Less expressive.

    ‘closed’ vs. ‘open’

    When a shadowRoot is ‘closed’ the it cannot be accessed via myHost.shadowRoot. This gives a component author some assurance that users won’t poke into implementation details, similar to how you can use closures to keep things private.

    Apple felt strongly that this was an important feature that they would block on. They argued that implementation details should never be exposed to the outside world and that ‘closed’ mode would be a required feature when ‘isolated’ custom elements became a thing.

    Google on the other hand felt that ‘closed’ shadow roots would prevent some accessibility and component tooling use-cases. They argued that it’s impossible to accidentally stumble into a shadowRoot and that if people want to they likely have a good reason. JS/DOM is open, let’s keep it that way.

    At the April meeting it became clear that to move forward, ‘mode’ needed to be a feature, but vendors were struggling to reach agreement on whether this should default to ‘open’ or ‘closed’. As a result, all agreed that for V1 ‘mode’ would be a required parameter, and thus wouldn’t need a specified default.

    element.createShadowRoot({ mode: 'open' });
    element.createShadowRoot({ mode: 'closed' });

    Shadow piercing combinators

    A ‘piercing combinator’ is a special CSS ‘combinator’ that can target elements inside a shadow root from the outside world. An example is /deep/ later renamed to >>>:

    .foo >>> div { color: red }

    When Web Components were first specified it was thought that these were required, but after looking at how they were being used it seemed to only bring problems, making it too easy to break the style boundaries that make Web Components so appealing.

    Performance

    Style calculation can be incredibly fast inside a tightly scoped Shadow DOM if the engine doesn’t have to take into consideration any outside selectors or state. The very presence of piercing combinators forbids these kind of optimisations.

    Alternatives

    Dropping shadow piercing combinators doesn’t mean that users will never be able to customize the appearance of a component from the outside.

    CSS custom-properties (variables)

    In Firefox OS we’re using CSS Custom Properties to expose specific style properties that can be defined (or overridden) from the outside.

    External (user):

    x-foo { --x-foo-border-radius: 10px; }
    

    Internal (author):

    .internal-part { border-radius: var(--x-foo-border-radius, 0); }
    Custom pseudo-elements

    We have also seen interest expressed from several vendors in reintroducing the ability to define custom pseudo selectors that would expose given internal parts to be styled (similar to how we style parts of <input type=”range”> today).

    x-foo::my-internal-part { ... }

    This will likely be considered for a Shadow DOM V2 specification.

    Mixins – @extend

    There is proposed specification to bring SASS’s @extend behaviour to CSS. This would be a useful tool for component authors to allow users to provide a ‘bag’ of properties to apply to a specific internal part.

    External (user):

    .x-foo-part {
      background-color: red;
      border-radius: 4px;
    }

    Internal (author):

    .internal-part {
      @extend .x-foo-part;
    }

    Multiple shadow roots

    Why would I want more than one shadow root on the same element?, I hear you ask. The answer is: inheritance.

    Let’s imagine I’m writing an <x-dialog> component. Within this component I write all the markup, styling, and interactions to give me an opening and closing dialog window.

    <x-dialog>
      <h1>My title</h1>
      <p>Some details</p>
      <button>Cancel</button>
      <button>OK</button>
    </x-dialog>

    The shadow root pulls any user provided content into div.inner via the <content> insertion point.

    <div class="outer">
      <div class="inner">
      <content></content>
      </div>
    </div>

    I also want to create <x-dialog-alert> that looks and behaves just like <x-dialog> but with a more restricted API, a bit like alert('foo').

    <x-dialog-alert>foo</x-dialog-alert>
    var proto = Object.create(XDialog.prototype);
    
    proto.createdCallback = function() {
      XDialog.prototype.createdCallback.call(this);
      this.createShadowRoot();
      this.shadowRoot.innerHTML = templateString;
    };
    
    document.registerElement('x-dialog-alert', { prototype: proto });
    

    The new component will have its own shadow root, but it’s designed to work on top of the parent class’s shadow root. The <shadow> represents the ‘older’ shadow root and allows us to project content inside it.

    <shadow>
      <h1>Alert</h1>
      <content></content>
      <button>OK</button>
    </shadow>

    Once you get your head round multiple shadow roots, they become a powerful concept. The downside is they bring a lot of complexity and introduce a lot of edge cases.

    Inheritance without multiple shadows

    Inheritance is still possible without multiple shadow roots, but it involves manually mutating the super class’s shadow root.

    
    var proto = Object.create(XDialog.prototype);
    
    proto.createdCallback = function() {
      XDialog.prototype.createdCallback.call(this);
      var inner = this.shadowRoot.querySelector('.inner');
    
      var h1 = document.createElement('h1');
      h1.textContent = 'Alert';
      inner.insertBefore(h1, inner.children[0]);
    
      var button = document.createElement('button');
      button.textContent = 'OK';
      inner.appendChild(button);
    
      ...
    };
    
    document.registerElement('x-dialog-alert', { prototype: proto });
    

    The downsides of this approach are:

    1. Not as elegant.
    2. Your sub-component is dependent on the implementation details of the super-component.
    3. This wouldn’t be possible if the super component’s shadow root was ‘closed’, as this.shadowRoot would be undefined.

    HTML Imports

    HTML Imports provide a way to import all assets defined in one .html document, into the scope of another.

    <link rel="import" href="/path/to/imports/stuff.html">

    As previously stated, Mozilla is not currently intending to implementing HTML Imports. This is in part because we’d like to see how ES6 modules pan out before shipping another way of importing external assets, and partly because we don’t feel they enable much that isn’t already possible.

    We’ve been working with Web Components in Firefox OS for over a year and have found using existing module syntax (AMD or Common JS) to resolve a dependency tree, registering elements, loaded using a normal <script> tag seems to be enough to get stuff done.

    HTML Imports do lend themselves well to a simpler/more declarative workflow, such as the older <element> and Polymer’s current registration syntax.

    With this simplicity has come criticism from the community that Imports don’t offer enough control to be taken seriously as a dependency management solution.

    Before the decision was made a few months ago, Mozilla had a working implementation behind a flag, but struggled through an incomplete specification.

    What will happen to them?

    Apple’s Isolated Custom Elements proposal makes use of an HTML Imports style approach to provide custom elements with their own document scope;: Perhaps there’s a future there.

    At Mozilla we want to explore how importing custom element definitions can align with upcoming ES6 module APIs. We’d be prepared to implement if/when they appear to enable developers to do stuff they can’t already do.

    To conclude

    Web Components are a prime example of how difficult it is to get large features into the browser today. Every API added lives indefinitely and remains as an obstacle to the next.

    Comparable to picking apart a huge knotted ball of string, adding a bit more, then tangling it back up again. This knot, our platform, grows ever larger and more complex.

    Web Components have been in planning for over three years, but we’re optimistic the end is near. All major vendors are on board, enthusiastic, and investing significant time to help resolve the remaining issues.

    Let’s get ready to componentize the web!

    More

  2. Mozilla and Web Components: Update

    Editor’s note: Mozilla has a long history of participating in standards development. The post below shows a real-time slice of how standards are debated and adopted. The goal is to update developers who are most affected by implementation decisions we make in Firefox. We are particularly interested in getting feedback from JavaScript library and framework developers.

    Mozilla has been working on Web Components — a technology encompassing HTML imports, custom elements, and shadow DOM — for a while now and testing this approach in Gaia, the frontend of Firefox OS. Unfortunately, our feedback into the standards process has not always resulted in the changes required for us to ship Web Components. Therefore we decided to reevaluate our stance with members of the developer community.

    We came up with the following tentative plan for shipping Web Components in Firefox and we would really appreciate input from the developer community as we move this forward. Web Components changes a core aspect of the Web Platform and getting it right is important. We believe the way to do that is by having the change be driven by the hard learned lessons from JavaScript library developers.

    • Mozilla will not ship an implementation of HTML Imports. We expect that once JavaScript modules — a feature derived from JavaScript libraries written by the developer community — is shipped, the way we look at this problem will have changed. We have also learned from Gaia and others, that lack of HTML Imports is not a problem as the functionality can easily be provided for with a polyfill if desired.
    • Mozilla will ship an implementation of custom elements. Exposing the lifecycle is a very important aspect for the creation of components. We will work with the standards community to use Symbol-named properties for the callbacks to prevent name collisions. We will also ensure the strategy surrounding subclassing is sound with the latest work on that front in JavaScript and that the callbacks are sufficiently capable to describe the lifecycle of elements or can at least be changed in that direction.
    • Mozilla will ship an implementation of shadow DOM. We think work needs to be done to decouple style isolation from event retargeting to make event delegation possible in frameworks and we would like to ensure distribution is sufficiently extensible beyond Selectors. E.g Gaia would like to see this ability.

    Our next steps will be working with the standards community to make these changes happen, making sure there is sufficient test coverage in web-platform-tests, and making sure the specifications become detailed enough to implement from.

    So please let us know what you think here in the comments or directly on the public-webapps standards list!

  3. ServiceWorkers and Firefox

    Since early 2013, Mozillians have been involved with the design of the Service Worker. Thanks to work by Google, Samsung, Mozilla, and others, this exciting new feature of the web platform has evolved to the point that it is being implemented in various web browser engines.

    What are Service Workers?

    At their simplest, Service Workers are scripts that act as client-side proxies for web pages. JavaScript code can intercept network requests, deliver manufactured responses and perform granular caching based on the unique needs of the application, a feature that the web platform has lacked before now. This powerful capability being made available to web developers enables, among other things, the creation of fully-functioning offline experiences. Jake Archibald has summarized some of these features in his blog post.

    Since Service Workers run in the “background”, they open up several possibilities for the Web that were previously only available on native platforms. Apart from the networking capabilities provided by the base specification, Service Workers are intended to be used by the Push API and the Background Sync API to deliver messages from the user-agent to web applications.

    Service Workers in Firefox

    A number of Mozillians have been hard at work implementing Service Workers in Gecko while Anne van Kesteren and Jonas Sicking help with the design and specification. Members of the Necko team and others have provided input from networking and related perspectives. Nikhil Marathe recently published a blog post about the status of Service Workers in Gecko.

    The Service Worker implementation in Gecko is landing in pieces as soon as they are finished and reviewed. For the time being, as the specification continues toward stability and other implementations — notably Blink’s — progress, all functionality in Gecko is behind the dom.serviceWorkers.enabled preference which is set to false by default but can be toggled in about:config.

    Our plan is that web developers will soon be able to exercise most Service Worker functionality in Firefox Nightly with the above preference flipped to true. The best plans can always be waylaid but we hope for this to happen by the end of September 2014 at the latest.

    Status of Service Worker implementations

    The inimitable Jake Archibald has written a tool to easily see the status of Service Worker implementations. You can follow along with the gecko implementation via the meta bug.

  4. Easy audio capture with the MediaRecorder API

    The MediaRecorder API is a simple construct, used inside Navigator.getUserMedia(), which provides an easy way of recording media streams from the user’s input devices and instantly using them in web apps. This article provides a basic guide on how to use MediaRecorder, which is supported in Firefox Desktop/Mobile 25, and Firefox OS 2.0.

    What other options are available?

    Capturing media isn’t quite as simple as you’d think on Firefox OS. Using getUserMedia() alone yields raw PCM data, which is fine for a stream, but then if you want to capture some of the audio or video you start having to perform manual encoding operations on the PCM data, which can get complex very quickly.

    Then you’ve got the Camera API on Firefox OS, which until recently was a certified API, but has been downgraded to privileged recently.

    Web activities are also available to allow you to grab media via other applications (such as Camera).

    the only trouble with these last two options is that they would capture only video with an audio track, and you would still have separate the audio if you just wanted an audio track. MediaRecorder provides an easy way to capture just audio (with video coming later — it is _just_ audio for now.)

    A sample application: Web Dictaphone

    An image of the Web dictaphone sample app - a sine wave sound visualization, then record and stop buttons, then an audio jukebox of recorded tracks that can be played back.

    To demonstrate basic usage of the MediaRecorder API, we have built a web-based dictaphone. It allows you to record snippets of audio and then play them back. It even gives you a visualization of your device’s sound input, using the Web Audio API. We’ll concentrate on the recording and playback functionality for this article.

    You can see this demo running live, or grab the source code on Github (direct zip file download.)

    CSS goodies

    The HTML is pretty simple in this app, so we won’t go through it here; there are a couple of slightly more interesting bits of CSS worth mentioning, however, so we’ll discuss them below. If you are not interested in CSS and want to get straight to the JavaScript, skip to the “Basic app setup” section.

    Keeping the interface constrained to the viewport, regardless of device height, with calc()

    The calc function is one of those useful little utility features that’s cropped up in CSS that doesn’t look like much initially, but soon starts to make you think “Wow, why didn’t we have this before? Why was CSS2 layout so awkward?” It allows you do a calculation to determine the computed value of a CSS unit, mixing different units in the process.

    For example, in Web Dictaphone we have theee main UI areas, stacked vertically. We wanted to give the first two (the header and the controls) fixed heights:

    header {
      height: 70px;
    }
     
    .main-controls {
      padding-bottom: 0.7rem;
      height: 170px;
    }

    However, we wanted to make the third area (which contains the recorded samples you can play back) take up whatever space is left, regardless of the device height. Flexbox could be the answer here, but it’s a bit overkill for such a simple layout. Instead, the problem was solved by making the third container’s height equal to 100% of the parent height, minus the heights and padding of the other two:

    .sound-clips {
      box-shadow: inset 0 3px 4px rgba(0,0,0,0.7);
      background-color: rgba(0,0,0,0.1);
      height: calc(100% - 240px - 0.7rem);
      overflow: scroll;
    }

    Note: calc() has good support across modern browsers too, even going back to Internet Explorer 9.

    Checkbox hack for showing/hiding

    This is fairly well documented already, but we thought we’d give a mention to the checkbox hack, which abuses the fact that you can click on the <label> of a checkbox to toggle it checked/unchecked. In Web Dictaphone this powers the Information screen, which is shown/hidden by clicking the question mark icon in the top right hand corner. First of all, we style the <label> how we want it, making sure that it has enough z-index to always sit above the other elements and therefore be focusable/clickable:

    label {
        font-family: 'NotoColorEmoji';
        font-size: 3rem;
        position: absolute;
        top: 2px;
        right: 3px;
        z-index: 5;
        cursor: pointer;
    }

    Then we hide the actual checkbox, because we don’t want it cluttering up our UI:

    input[type=checkbox] {
       position: absolute;
       top: -100px;
    }

    Next, we style the Information screen (wrapped in an <aside> element) how we want it, give it fixed position so that it doesn’t appear in the layout flow and affect the main UI, transform it to the position we want it to sit in by default, and give it a transition for smooth showing/hiding:

    aside {
       position: fixed;
       top: 0;
       left: 0;
       text-shadow: 1px 1px 1px black;
       width: 100%;
       height: 100%;
       transform: translateX(100%);
       transition: 0.6s all;
       background-color: #999;
        background-image: linear-gradient(to top right, rgba(0,0,0,0), rgba(0,0,0,0.5));
    }

    Last, we write a rule to say that when the checkbox is checked (when we click/focus the label), the adjacent <aside> element will have it’s horizontal translation value changed and transition smoothly into view:

    input[type=checkbox]:checked ~ aside {
      transform: translateX(0);
    }

    Basic app setup

    To grab the media stream we want to capture, we use getUserMedia() (gUM for short). We then use the MediaRecorder API to record the stream, and output each recorded snippet into the source of a generated <audio> element so it can be played back.

    First, we’ll add in a forking mechanism to make gUM work, regardless of browser prefixes, and so that getting the app working on other browsers once they start supporting MediaRecorder will be easier in the future.

    navigator.getUserMedia = ( navigator.getUserMedia ||
                           navigator.webkitGetUserMedia ||
                           navigator.mozGetUserMedia ||
                           navigator.msGetUserMedia);

    Then we’ll declare some variables for the record and stop buttons, and the <article> that will contain the generated audio players:

    var record = document.querySelector('.record');
    var stop = document.querySelector('.stop');
    var soundClips = document.querySelector('.sound-clips');

    Finally for this section, we set up the basic gUM structure:

    if (navigator.getUserMedia) {
       console.log('getUserMedia supported.');
       navigator.getUserMedia (
          // constraints - only audio needed for this app
          {
             audio: true
          },
     
          // Success callback
          function(stream) {
     
     
          },
     
          // Error callback
          function(err) {
             console.log('The following gUM error occured: ' + err);
          }
       );
    } else {
       console.log('getUserMedia not supported on your browser!');
    }

    The whole thing is wrapped in a test that checks whether gUM is supported before running anything else. Next, we call getUserMedia() and inside it define:

    • The constraints: Only audio is to be captured; MediaRecorder only supports audio currently anyway.
    • The success callback: This code is run once the gUM call has been completed successfully.
    • The error/failure callback: The code is run if the gUM call fails for whatever reason.

    Note: All of the code below is placed inside the gUM success callback.

    Capturing the media stream

    Once gUM has grabbed a media stream successfully, you create a new Media Recorder instance with the MediaRecorder() constructor and pass it the stream directly. This is your entry point into using the MediaRecorder API — the stream is now ready to be captured straight into a Blob, in the default encoding format of your browser.

    var mediaRecorder = new MediaRecorder(stream);

    There are a series of methods available in the MediaRecorder interface that allow you to control recording of the media stream; in Web Dictaphone we just make use of two. First of all, MediaRecorder.start() is used to start recording the stream into a Blob once the record button is pressed:

    record.onclick = function() {
      mediaRecorder.start();
      console.log(mediaRecorder.state);
      console.log("recorder started");
      record.style.background = "red";
      record.style.color = "black";
    }

    When the MediaRecorder is recording, the MediaRecorder.state property will return a value of “recording”.

    Second, we use the MediaRecorder.stop() method to stop the recording when the stop button is pressed, and finalize the Blob ready for use somewhere else in our application.

    stop.onclick = function() {
      mediaRecorder.stop();
      console.log(mediaRecorder.state);
      console.log("recorder stopped");
      record.style.background = "";
      record.style.color = "";
    }

    When recording has been stopped, the state property returns a value of “inactive”.

    Note that there are other ways that a Blob can be finalized and ready for use:

    • If the media stream runs out (e.g. if you were grabbing a song track and the track ended), the Blob is finalized.
    • If the MediaRecorder.requestData() method is invoked, the Blob is finalized, but recording then continues in a new Blob.
    • If you include a timeslice property when invoking the start() method — for example start(10000) — then a new Blob will be finalized (and a new recording started) each time that number of milliseconds has passed.

    Grabbing and using the blob

    When the blob is finalized and ready for use as described above, a dataavailable event is fired, which can be handled using a mediaRecorder.ondataavailable handler:

    mediaRecorder.ondataavailable = function(e) {
      console.log("data available");
     
      var clipName = prompt('Enter a name for your sound clip');
     
      var clipContainer = document.createElement('article');
      var clipLabel = document.createElement('p');
      var audio = document.createElement('audio');
      var deleteButton = document.createElement('button');
     
      clipContainer.classList.add('clip');
      audio.setAttribute('controls', '');
      deleteButton.innerHTML = "Delete";
      clipLabel.innerHTML = clipName;
     
      clipContainer.appendChild(audio);
      clipContainer.appendChild(clipLabel);
      clipContainer.appendChild(deleteButton);
      soundClips.appendChild(clipContainer);
     
      var audioURL = window.URL.createObjectURL(e.data);
      audio.src = audioURL;
     
      deleteButton.onclick = function(e) {
        evtTgt = e.target;
        evtTgt.parentNode.parentNode.removeChild(evtTgt.parentNode);
      }
    }

    Let’s go through the above code and look at what’s happening.

    First, we display a prompt asking the user to name their clip.

    Next, we create an HTML structure like the following, inserting it into our clip container, which is a <section> element.

    <article class="clip">
      <audio controls></audio>
      <p><em>your clip name</em></p>
      <button>Delete</button>
    </article>

    After that, we create an object URL pointing to the event’s data attribute, using window.URL.createObjectURL(e.data): this attribute contains the Blob of the recorded audio. We then set the value of the <audio> element’s src attribute to the object URL, so that when the play button is pressed on the audio player, it will play the Blob.

    Finally, we set an onclick handler on the delete button to be a function that deletes the whole clip HTML structure.

    Conclusion

    And there you have it; MediaRecorder should serve to make your app media recording needs easier. Have a play around with it and let us know what you think: we are looking forward to seeing what you’ll build!

  5. getUserMedia is ready to roll!

    We blogged about some of our WebRTC efforts back in April. Today we have an exciting update for you on that front: getUserMedia has landed on mozilla-central! This means you will be able to use the API on the latest Nightly versions of Firefox, and it will eventually make its way to a release build.

    getUserMedia is a DOM API that allows web pages to obtain video and audio input, for instance, from a webcam or microphone. We hope this will open the possibility of building a whole new class of web pages and applications. This DOM API is one component of the WebRTC project, which also includes APIs for peer-to-peer communication channels that will enable exchange of video steams, audio streams and arbitrary data.

    We’re still working on the PeerConnection API, but getUserMedia is a great first step in the progression towards full WebRTC support in Firefox! We’ve certainly come a long way since the first image from a webcam appeared on a web page via a DOM API. (Not to mention audio recording support in Jetpack before that.)

    We’ve implemented a prefixed version of the “Media Capture and Streams” standard being developed at the W3C. Not all portions of the specification have been implemented yet; most notably, we do not support the Constraints API (which allows the caller to request certain types of audio and video based on various parameters).

    We have also implemented a Mozilla specific extension to the API: the first argument to mozGetUserMedia is a dictionary that will also accept the property {picture: true} in addition to {video: true} or {audio: true}. The picture API is an experiment to see if there is interest in a dedicated mechanism to obtain a single picture from the user’s camera, without having to set up a video stream. This could be useful in a profile picture upload page, or a photo sharing application, for example.

    Without further ado, let’s start with a simple example! Make sure to create a pref named “media.navigator.enabled” and set it to true via about:config first. We’ve put the pref in place because we haven’t implemented a permissions model or any UI for prompting the user to authorize access to the camera or microphone. This release of the API is aimed at developers, and we’ll enable the pref by default after we have a permission model and UI that we’re happy with.

    %CODEgum%

    There’s also a demo page where you can test the audio, video and picture capabilities of the API. Give it a whirl, and let us know what you think! We’re especially interested in feedback from the web developer community about the API and whether it will meet your use cases. You can leave comments on this post, or on the dev-media mailing list or newsgroup.

    We encourage you to get involved with the project – there’s a lot of information about our ongoing efforts on the project wiki page. Posting on the mailing list with your questions, comments and suggestions is great way to get started. We also hang out on the #media IRC channel, feel free to drop in for an informal chat.

    Happy hacking!

  6. DOM MutationObserver – reacting to DOM changes without killing browser performance.

    DOM Mutation Events seemed like a great idea at the time – as web developers create a more dynamic web it seems natural that we would welcome the ability to listen for changes in the DOM and react to them. In practice however DOM Mutation Events were a major performance and stability issue and have been deprecated for over a year.

    The original idea behind DOM Mutation Events is still appealing, however, and so in September 2011 a group of Google and Mozilla engineers announced a new proposal that would offer similar functionality with improved performance: DOM MutationObserver. This new DOM Api is available in Firefox and Webkit nightly builds, as well as Chrome 18.

    At it’s simplest, a MutationObserver implementation looks like this:

    // select the target node
    var target = document.querySelector('#some-id');
     
    // create an observer instance
    var observer = new MutationObserver(function(mutations) {
        mutations.forEach(function(mutation) {
            console.log(mutation.type);
        });
    });
     
    // configuration of the observer:
    var config = { attributes: true, childList: true, characterData: true }
     
    // pass in the target node, as well as the observer options
    observer.observe(target, config);
     
    // later, you can stop observing
    observer.disconnect();

    The key advantage to this new specification over the deprecated DOM Mutation Events spec is one of efficiency. If you are observing a node for changes, your callback will not be fired until the DOM has finished changing. When the callback is triggered, it is supplied a list of the changes to the DOM, which you can then loop through and choose to react to.

    This also means that any code you write will need to process the observer results in order to react to the changes you are looking for. Here is a compact example of an observer that listens for changes in an editable ordered list:

    <!DOCTYPE html>
    <ol contenteditable oninput="">
      <li>Press enter</li>
    </ol>
    <script>
      var MutationObserver = window.MutationObserver || window.WebKitMutationObserver || window.MozMutationObserver;
      var list = document.querySelector('ol');
     
      var observer = new MutationObserver(function(mutations) {
        mutations.forEach(function(mutation) {
          if (mutation.type === 'childList') {
            var list_values = [].slice.call(list.children)
                .map( function(node) { return node.innerHTML; })
                .filter( function(s) {
                  if (s === '<br />') {
                    return false;
                  }
                  else {
                    return true;
                  }
            });
            console.log(list_values);
          }
        });
      });
     
      observer.observe(list, {
      	attributes: true,
      	childList: true,
      	characterData: true
       });
    </script>

    If you want to see this code running, I’ve put it up on jsbin here:

    http://jsbin.com/ivamoh/53/edit

    If you play with the live example, you’ll notice some quirks in behaviour, in particular that the callback is triggered when you press enter in each li, in particular when the user action results in a node being added or removed from the DOM. This is an important distinction to be made from other techniques such as binding events to key presses or more common events like ‘click’. MutationObservers work differently from these techniques because they are triggered by changes in the DOM itself, not by events generated either via JS or user interaction.

    So what are these good for?

    I don’t expect most JS hackers are going to run out right now and start adding mutation observers to their code. Probably the biggest audience for this new api are the people that write JS frameworks, mainly to solve problems and create interactions they could not have done previously, or at least not with reasonable performance. Another use case would be situations where you are using frameworks that manipulate the DOM and need to react to these modifications efficiently ( and without setTimeout hacks! ).

    Another common use of the Dom Mutation Events api is in browser extensions, and in the next week or so I’m going to publish a follow-up post on how MutationObservers are particularly useful when interacting with web content in a Firefox Add-on.

    Resources

  7. Announcing Firefox Aurora 10

    We’re happy to announce the availability of Aurora 10.
    (Download and Test Aurora 10)

    In additional to the normal improvements that you’ve come to expect like performance, security and bug fixes, Aurora 10 focuses in HTML5 enhancements.

    New additions

    Developer Tools

    Aurora 10 also implements incremental enhancements like IndexedDB setVersion API changes. Ongoing detailed attention to evolving specifications help to keep Firefox at the front of the Web revolution. (Read more about IndexedDB on MDN.)

    DOM

    • We now fire a “load” event on stylesheet linking when the sheet load finishes or “error” if the load fails.
    • We turn the POSTDATA prompt into an information page (when navigating in session history).
    • We only forward event attributes on body/frameset to the window if we also forward the corresponding on* property.
    • We no longer allow more than one call to window.open() when we allow popups.
    • We fixed a bug where a success callback never fired when a position update is triggered after getCurrentPosition().
    • We removed replaceWholeText().
    • We fixed an error with createPattern(zero-size canvas).
    • We now handle putImageData(nonfinite) correctly.
    • We now throw INVALID_STATE_ERR when dispatching uninitialized events.
    • We’ve made Document.documentURI readonly.
    • We fixed document.importNode to comply with optional argument omitted.

    Web workers

    • We now allow data URLs.
    • We implemented event.stopImmediatePropagation in workers.
    • We made XHR2 response/responseType work in Web Workers.

    Graphics

    • We implement the WebGL OES_standard_derivatives extension.
    • We implement minimal-capabilities WebGL mode.

    JavaScript

    • The function caller property no longer skips over eval frames.
    • We fixed E4X syntax so that it is not accepted in ES5 strict mode.
    • weakmap.set no longer returns itself instead of undefined.
    • We implemented the battery API.

    Offline: IndexedDB enhancements

    • IndexedDB setVersion API changes
    • Added support for IDBObjectStore/IDBIndex.count
    • Various methods accept both keys and KeyRanges.
    • Added support for IDBCursor.advance.
    • Implemented deleteDatabase.
    • objectStoreNames are no longer updated on closed databases when another connection adds or removes object stores
    • IDBObjectStore.delete and IDBCursor.delete now return undefined.
    • No longer throws an error if there are unknown properties in the options objects to createObjectStore/createIndex.
    • We now the errorCode to “ABORT_ERR” for all pending requests when IDBTransaction.abort() is called.
    • Fixed the sort order for indexes.

    Layout

    • We have updated the current rule for handling malformed media queries.
    • We now support the HTML5 <bdi> element and CSS property unicode-bidi: isolate.
    • The CSS3 implementation now supports unicode-bidi: plaintext.

    Media

    • Implemented Document.mozFullScreenEnabled.
    • Enabled the DOM full-screen API on desktop Firefox by default.
  8. Tagging docs for sprint at JSConf.eu October 1-2

    We’re very excited to announce that Mozilla is sponsoring the Hacker Lounge at JSConf.eu and we will be holding a doc sprint at and during the conference. The focus of this doc sprint will naturally be docs for JavaScript and DOM. We hope to encourage attendees at the conference to contribute at least a little to improving the JS and DOM docs on MDN. I and a handful of MDN community members will be there to show them how.

    To that end, we want to tag as many JS and DOM articles as possible, with tags indicating what work needs to be done on those articles. That way, anybody dropping in during the sprint (or anytime later) can look for tagged articles and find something to work on.

    Here are some of the tags we use to indicate that an article needs help:

    • NeedsTechnicalReview: needs someone to verify that the technical information is complete and correct.
    • NeedsExample: needs one or more illustrative code examples of the item documented.
    • NeedsContent: the item is incomplete and needs to be filled out.
    • NeedsJSVersion: needs information about the version of JavaScript and EcmaScript this item first appears in.
    • NeedsBrowserCompatibility: needs a browser compatibility table or needs the table filled out.
    • MakeBrowserAgnostic: the article is written with a focus on Gecko, when it is actually about a standard function or feature, which should be rewritten to be generic.

    Please help by tagging articles in MDN that need work with the appropriate tag. You can use the Talk page for each article to elaborate on what needs to be done, if the tag is not descriptive enough. To modify tags on an article, login to MDN and click Edit Tags at the bottom of the page.

    The wiki page for the doc sprint has links to queries for some of these tags.

    If you will be at JSConf.eu, I look forward to seeing you there! If you will be participating remotely, I’ll see you online!