Mozilla

Articles

Sort by:

View:

  1. ES6 In Depth: let and const

    ES6 In Depth is a series on new features being added to the JavaScript programming language in the 6th Edition of the ECMAScript standard, ES6 for short.

    The feature I’d like to talk about today is at once humble and startlingly ambitious.

    When Brendan Eich designed the first version of JavaScript back in 1995, he got plenty of things wrong, including things that have been part of the language ever since, like the Date object and objects automatically converting to NaN when you accidentally multiply them. However, the things he got right are stunningly important things, in hindsight: objects; prototypes; first-class functions with lexical scoping; mutability by default. The language has good bones. It was better than anyone realized at first.

    Still, Brendan made one particular design decision that bears on today’s article—a decision that I think can be fairly characterized as a mistake. It’s a little thing. A subtle thing. You might use the language for years and not even notice it. But it matters, because this mistake is in the side of the language that we now think of as “the good parts”.

    It has to do with variables.

    Problem #1: Blocks are not scopes

    The rule sounds so innocent: The scope of a var declared in a JS function is the whole body of that function. But there are two ways this can have groan-inducing consequences.

    One is that the scope of variables declared in blocks is not just the block. It’s the entire function.

    You may never have noticed this before. I’m afraid it’s one of those things you won’t be able to un-see. Let’s walk through a scenario where it leads to a tricky bug.

    Say you have some existing code that uses a variable named t:

    function runTowerExperiment(tower, startTime) {
      var t = startTime;
    
      tower.on("tick", function () {
        ... code that uses t ...
      });
      ... more code ...
    }
    

    Everything works great, so far. Now you want to add bowling ball speed measurements, so you add a little if-statement to the inner callback function.

    function runTowerExperiment(tower, startTime) {
      var t = startTime;
    
      tower.on("tick", function () {
        ... code that uses t ...
        if (bowlingBall.altitude() <= 0) {
          var t = readTachymeter();
          ...
        }
      });
      ... more code ...
    }
    

    Oh, dear. You’ve unwittingly added a second variable named t. Now, in the “code that uses t”, which was working fine before, t refers to the new inner variable t rather than the existing outer variable.

    The scope of a var in JavaScript is like the bucket-of-paint tool in Photoshop. It extends in both directions from the declaration, forwards and backwards, and it just keeps going until it reaches a function boundary. Since this variable t’s scope extends so far backwards, it has to be created as soon as we enter the function. This is called hoisting. I like to imagine the JS engine lifting each var and function to the top of the enclosing function with a tiny code crane.

    Now, hoisting has its good points. Without it, lots of perfectly cromulent techniques that work fine in the global scope wouldn’t work inside an IIFE. But in this case, hoisting is causing a nasty bug: all your calculations using t will start producing NaN. It’ll be hard to track down, too, especially if your code is larger than this toy example.

    Adding a new block of code caused a mysterious error in code before that block. Is it just me, or is that really weird? We don’t expect effects to precede causes.

    But this is a piece of cake compared to the second var problem.

    Problem #2: Variable oversharing in loops

    You can guess what happens when you run this code. It’s totally straightforward:

    var messages = ["Hi!", "I'm a web page!", "alert() is fun!"];
    
    for (var i = 0; i < messages.length; i++) {
      alert(messages[i]);
    }
    

    If you’ve been following this series, you know I like to use alert() for example code. Maybe you also know that alert() is a terrible API. It’s synchronous. So while an alert is visible, input events are not delivered. Your JS code—and in fact your whole UI—is basically paused until the user clicks OK.

    All of which makes alert() the wrong choice for almost anything you want to do in a web page. I use it because I think all those same things make alert() a great teaching tool.

    Still, I could be persuaded to give up all that clunkiness and bad behavior… if it means I can make a talking cat.

    var messages = ["Meow!", "I'm a talking cat!", "Callbacks are fun!"];
    
    for (var i = 0; i < messages.length; i++) {
      setTimeout(function () {
        cat.say(messages[i]);
      }, i * 1500);
    }
    

    See this code working incorrectly in action!

    But something’s wrong. Instead of saying all three messages in order, the cat says “undefined” three times.

    Can you spot the bug?

    (Photo of a caterpillar well camouflaged on the bark of a tree. Gujarat, India.)

    Photo credit: nevil saveri

    The problem here is that there is only one variable i. It’s shared by the loop itself and all three timeout callbacks. When the loop finishes running, the value of i is 3 (because messages.length is 3), and none of the callbacks have been called yet.

    So when the first timeout fires, and calls cat.say(messages[i]), it’s using messages[3]. Which of course is undefined.

    There are many ways to fix this (here’s one), but this is a second problem caused by the var scoping rules. It would be awfully nice never to have this kind of problem in the first place.

    let is the new var

    For the most part, design mistakes in JavaScript (other programming languages too, but especially JavaScript) can’t be fixed. Backwards compatibility means never changing the behavior of existing JS code on the Web. Even the standard committee has no power to, say, fix the weird quirks in JavaScript’s automatic semicolon insertion. Browser makers simply will not implement breaking changes, because that kind of change punishes their users.

    So about ten years ago, when Brendan Eich decided to fix this problem, there was really only one way to do it.

    He added a new keyword, let, that could be used to declare variables, just like var, but with better scoping rules.

    It looks like this:

    let t = readTachymeter();
    

    Or this:

    for (let i = 0; i < messages.length; i++) {
      ...
    }
    

    let and var are different, so if you just do a global search-and-replace throughout your code, that could break parts of your code that (probably unintentionally) depend on the quirks of var. But for the most part, in new ES6 code, you should just stop using var and use let everywhere instead. Hence the slogan: “let is the new var”.

    What exactly are the differences between let and var? Glad you asked!

    • let variables are block-scoped. The scope of a variable declared with let is just the enclosing block, not the whole enclosing function.

      There’s still hoisting with let, but it’s not as indiscriminate. The runTowerExperiment example can be fixed simply by changing var to let. If you use let everywhere, you will never have that kind of bug.

    • Global let variables are not properties on the global object. That is, you won’t access them by writing window.variableName. Instead, they live in the scope of an invisible block that notionally encloses all JS code that runs in a web page.

    • Loops of the form for (let x...) create a fresh binding for x in each iteration.

      This is a very subtle difference. It means that if a for (let...) loop executes multiple times, and that loop contains a closure, as in our talking cat example, each closure will capture a different copy of the loop variable, rather than all closures capturing the same loop variable.

      So the talking cat example, too, can be fixed just by changing var to let.

      This applies to all three kinds of for loop: forof, forin, and the old-school C kind with semicolons.

    • It’s an error to try to use a let variable before its declaration is reached. The variable is uninitialized until control flow reaches the line of code where it’s declared. For example:

      function update() {
        console.log("current time:", t);  // ReferenceError
        ...
        let t = readTachymeter();
      }
      

      This rule is there to help you catch bugs. Instead of NaN results, you’ll get an exception on the line of code where the problem is.

      This period when the variable is in scope, but uninitialized, is called the temporal dead zone. I keep waiting for this inspired bit of jargon to make the leap to science fiction. Nothing yet.

      (Crunchy performance details: In most cases, you can tell whether the declaration has run or not just by looking at the code, so the JavaScript engine does not actually need to perform an extra check every time the variable is accessed to make sure it’s been initialized. However, inside a closure, it sometimes isn’t clear. In those cases the JavaScript engine will do a run-time check. That means let can be a touch slower than var.)

      (Crunchy alternate-universe scoping details: In some programming languages, the scope of a variable starts at the point of the declaration, instead of reaching backwards to cover the whole enclosing block. The standard committee considered using that kind of scoping rule for let. That way, the use of t that causes a ReferenceError here simply wouldn’t be in the scope of the later let t, so it wouldn’t refer to that variable at all. It could refer to a t in an enclosing scope. But this approach did not work well with closures or with function hoisting, so it was eventually abandoned.)

    • Redeclaring a variable with let is a SyntaxError.

      This rule, too, is there to help you detect trivial mistakes. Still, this is the difference that is most likely to cause you some issues if you attempt a global let-to-var conversion, because it applies even to global let variables.

      If you have several scripts that all declare the same global variable, you’d better keep using var for that. If you switch to let, whichever script loads second will fail with an error.

      Or use ES6 modules. But that’s a story for another day.

    (Crunchy syntax details: let is a reserved word in strict mode code. In non-strict-mode code, for the sake of backward compatibility, you can still declare variables, functions, and arguments named let—you can write var let = 'q';! Not that you would do that. And let let; is not allowed at all.)

    Apart from those differences, let and var are pretty much the same. They both support declaring multiple variables separated by commas, for example, and they both support destructuring.

    Note that class declarations behave like let, not var. If you load a script containing a class multiple times, the second time you’ll get an error for redeclaring the class.

    const

    Right, one more thing!

    ES6 also introduces a third keyword that you can use alongside let: const.

    Variables declared with const are just like let except that you can’t assign to them, except at the point where they’re declared. It’s a SyntaxError.

    const MAX_CAT_SIZE_KG = 3000; // 🙀
    
    MAX_CAT_SIZE_KG = 5000; // SyntaxError
    MAX_CAT_SIZE_KG++; // nice try, but still a SyntaxError
    

    Sensibly enough, you can’t declare a const without giving it a value.

    const theFairest;  // SyntaxError, you troublemaker
    

    Secret agent namespace

    “Namespaces are one honking great idea—let’s do more of those!” —Tim Peters, “The Zen of Python”

    Behind the scenes, nested scopes are one of the core concepts that programming languages are built around. It’s been this way since what, ALGOL? Something like 57 years. And it’s truer today than ever.

    Before ES3, JavaScript only had global scopes and function scopes. (Let’s ignore with statements.) ES3 introduced trycatch statements, which meant adding a new kind of scope, used only for the exception variable in catch blocks. ES5 added a scope used by strict eval(). ES6 adds block scopes, for-loop scopes, the new global let scope, module scopes, and additional scopes that are used when evaluating default values for arguments.

    All the extra scopes added from ES3 onward are necessary to make JavaScript’s procedural and object-oriented features work as smoothly, precisely, and intuitively as closures—and cooperate seamlessly with closures. Maybe you never noticed any of these scoping rules before today. If so, the language is doing its job.

    Can I use let and const now?

    Yes. To use them on the web, you’ll have to use an ES6 compiler such as Babel, Traceur, or TypeScript. (Babel and Traceur do not support the temporal dead zone yet.)

    io.js supports let and const, but only in strict-mode code. Node.js support is the same, but the --harmony option is also required.

    Brendan Eich implemented the first version of let in Firefox nine years ago. The feature was thoroughly redesigned during the standardization process. Shu-yu Guo is upgrading our implementation to match the standard, with code reviews by Jeff Walden and others.

    Well, we’re in the home stretch. The end of our epic tour of ES6 features is in sight. In two weeks, we’ll finish up with what’s probably the most eagerly awaited ES6 feature of them all. But first, next week we’ll have a post that extends our earlier coverage of a new feature that’s just super. So please join us as Eric Faust returns with a look at ES6 subclassing in depth.

  2. Keyboard events in Firefox OS TV

    Getting started

    The behavior of input events via hardware keys in Firefox OS varies widely from app to app. Early smartphones came with a limited number of keys — Power, Home, Volume up, Volume down — so it was easy for the software to determine an appropriate response for each keypress event. However, Smart TV remotes now come with many hardware keys, and defining the appropriate behavior when a key is pressed has become an important issue on the Firefox OS TV platform. If a hardware key on a smart remote can be used both by apps and by the system, it’s important to determine which response is triggered when the key is pressed.

    This post will introduce the challenges of programming a TV remote to manage keyboard events on the Firefox OS Smart TV platform. We’ll classify keyboard events into four scenarios and describe these dispatch scenarios and how they interact. This is the first of two posts about keyboard events for Firefox OS Smart TV. Next week, in Part 2, we’ll drill down into the implementation details and and take a look at sample code for event handling on TV remote.

    f1

    [Fig 1]

    We begin with the ‘Info’ key on a TV remote. Often, it’s used by the hardware to display system information, although it’s possible for an application to use the same key to display app information. When a user presses the key, what action will be shown on screen — system info or app info?

    Four keyboard event scenarios

    To determine the appropriate behavior when hardware keys are pressed, we start by describing four scenarios for keyboard events.

    Scenario Description Event order
    SYSTEM-ONLY For keys which should be handled by mozbrowser-iframe-host-page only. system
    SYSTEM-FIRST For keys which can be handled by mozbrowser-iframe-host-page first and can then also be handled by mozbrowser-iframe-embedded-page. system->app
    APP-CANCELLED For keys which should be handled by the mozbrowser-iframe-embedded-page only. app
    APP-FIRST For keys which can be handled by mozbrowser-iframe-embedded-page first and can then also be handled by mozbrowser-iframe-host-page. app->system

    [Table 1]

    [Fig 2]

    [Fig 2]

    The mozbrowser-iframe-host-page and mozbrowser-iframe-embedded-page mentioned in Table 1 are illustrated in Fig.2 above. If A.html represents a host page whose source is B.html, then A.html is the mozbrowser-iframe-host-page, and B.html is mozbrowser-iframe-embedded-page. mozbrowser uses the non-standard Firefox Browser API, built for the implementation of key features and content experiences in Firefox OS apps. Learn more about mozbrowser on MDN.

    Suitable responses for any given keyboard event depend on the scenario. In the case illustrated above, let’s suppose that the ‘Info’ keyboard event is categorized as APP-FIRST and the default action set by system is to show system information. Thus, when we press the ‘Info’ key with app Z in the foreground, there are two possible results:

      1. If app Z has an event handler that tells the ‘Info’ key to show app information, then app information will appear on screen when the user presses the ‘Info’ key on the remote.
      2. If app Z doesn’t set an event handler for the ‘Info’ key, the default action is triggered. That is, the screen will show the system information.

    How to implement the four scenarios

    To implement the four keyboard event scenarios described above, we’ve introduced four new keyboard events:

        • mozbrowserbeforekeydown – fired before the keydown event
        • mozbrowserafterkeydown – fired after the keydown event
        • mozbrowserbeforekeyup – fired before the keyup event
        • mozbrowserafterkeyup – fired after the keyup event

    These four keyboard events are only received by the window that embeds a mozbrowser-iframe.

    The keyboard events occur in a specific sequence over time: mozbrowserbeforekeydown, mozbrowserafterkeydown, mozbrowserbeforekeyup, keyup, mozbrowserafterkeyup.

    This gives developers a way to implement the four scenarios mentioned above. Conceptually, the scenarios SYSTEM-ONLY, SYSTEM-FIRST and APP-CANCELLED, APP-FIRST can be implemented by setting proper handlers for the mozbrowserbeforekey* and mozbrowserafterkey* events. The SYSTEM-ONLY and SYSTEM-FIRST scenarios can be implemented by setting proper handlers for mozbrowserbeforekey* events and the APP-CANCELLED and APP-FIRST scenarios can be implemented via mozbrowserafterkey* events.

    iframe structure in Firefox OS

    [Fig 3]

    [Fig 3]

    To understand how to implement the four scenarios, let’s first take a look at iframe structure in Firefox OS. The outermost iframe in Firefox OS is shell.html. It embeds an in-process iframe which is sourced from system/index.html. The system app (system/index.html) contains several web apps (essentially iframes) which can be in-process (remote=”false”) or out-of-process (remote=”true”). As a result, the relationship of these three layers can be displayed in the following table:

    mozbrowser iframe host page mozbrowser iframe embedded page
    shell.html system/index.html
    system/index.html web apps(essentially iframes)

    [Table 2]

    Dispatch order of keyboard events

    [Fig 4]

    [Fig 4]

    When a keydown event is sent to some element in a mozbrowser-iframe-embedded-page, the owner of the embedded iframe, i.e., the mozbrowser-iframe-host-page, will receive the mozbrowserbeforekeydown event before the keydown event is sent and the mozbrowserafterkeydown event after the event is sent to the mozbrowser-iframe-embedded-page.

    In Gecko, once there is one keydown event with the target in an out-of-process iframe embedded in an HTML document, the keydown event is duplicated to the HTML document as well. The target of this duplicated event is set as the embedded iframe element.

    This results in the keyboard event sequence shown in Fig 4. It illustrates all related keydown events and their relationship when a keydown event with a target in a mozbrowser-iframe-embedded-page needs to be dispatched. In brief, events follow this sequence:

        1. Before dispatching any keydown event, the mozbrowserbeforekeydown event is first dispatched to the window of mozbrowser-iframe-host-page.
        2. Then, the original keydown event (with a target in a mozbrowser-iframe-embedded-page) will be duplicated to the mozbrowser-iframe-host-page HTML document. Its target will be set to be the iframe that contains the mozbrowser-iframe-embedded-page.
        3. Next, the original keydown event will be dispatched to its target.
        4. After the original keydown event dispatch is complete, the mozbrowserafterkeydown event will be dispatched to the window of mozbrowser-iframe-host-page.

    Notice that the event dispatch process described above follows the DOM tree event flow. Event sequence and event targets can be organized into the following table:

    Order event target
    1 mozbrowserbeforekeydown window in mozbrowser-iframe-host-page
    2 keydown iframe that contains the mozbrowser-iframe-embedded-page in mozbrowser-iframe-host-page
    3 keydown original one in mozbrowser-iframe-embedded-page
    4 mozbrowserafterkeydown window in mozbrowser-iframe-host-page

    [Table 3]

    f5

    [Fig 5]

    The keyboard events mozbrowserbeforekeydown, keydown, and mozbrowserafterkeydown, can be extended to nested mozbrowser iframes, like the iframe structure in Firefox OS described in Table 2. In this case, the mozbrowserbeforekeydown and mozbrowserafterkeydown events will be dispatched to the innermost mozbrowser-iframe-host-page as well as the outer one. Thus, in Firefox OS, mozbrowserkeydown and mozbrowserafterkeydown will be dispatched to the window of system/index.html and the window of shell.html. Fig. 5 illustrates the whole dispatch sequence of related events when a keydown event is dispatched to a web app. The sequence of events is demonstrated in Table 4.

    Order event target
    1 mozbrowserbeforekeydown window in shell.html
    2 mozbrowserbeforekeydown window in system/index.html
    3 keydown iframe that contains the web app in system/index.html
    4 keydown original one in web app
    5 mozbrowserafterkeydown window in system/index.html
    6 mozbrowserafterkeydown window in shell.html

    [Table 4]

    f6

    [Fig 6]

    Although the keyup event must be fired after keydown, the keydown event and the keyup event are independent of each other. Moreover, the path mozbrowserbeforekeyup, keyup, mozbrowserafterkeyup is independent of the path mozbrowserbeforekeydown, keydown, mozbrowserafterkeydown. Therefore, it’s possible for these two paths to cross each other. The mozbrowserbeforekeyup may arrive before the keydown event.

    In Firefox OS, most apps run out-of-process. This means that the app runs on its own process, not on the main process. After dispatching a given key* event (the duplicate) to the system app, it takes time to send the original key* event to the process where the mozbrowser-iframe-embedded-page is located. In a similar manner, after a given key* event is dispatched to the mozbrowser-iframe-embedded-page’s process, time is required to send mozbrowserafterkey* back to the process where the mozbrowser-iframe-host-page is located.

    Consequently, the mozbrowserbeforekeyup event may arrive in the main Firefox OS process (where the system app lives), before the keydown event is dispatched to the app’s own process. Common results of the order of the key* events are demonstrated above in Fig. 6. The yellow series represents the keydown path, and the blue series show the keyup path. And yes, these two paths may cross each other.

    Next steps

    If you’re looking for implementation details for the four scenarios we’ve described, as well as event handler sample code to help with your implementation of keyboard events for smart TV remotes, keep an eye for next week’s post, and stay tuned for more complete documentation arriving very soon on MDN.

  3. ES6 In Depth: Classes

    ES6 In Depth is a series on new features being added to the JavaScript programming language in the 6th Edition of the ECMAScript standard, ES6 for short.

    Today, we get a bit of a respite from the complexity that we’ve seen in previous posts in this series. There are no new never-before-seen ways of writing code with Generators; no all-powerful Proxy objects which provide hooks into the inner algorithmic workings of the JavaScript language; no new data structures that obviate the need for roll-your-own solutions. Instead, we get to talk about syntactic and idiomatic cleanups for an old problem: object constructor creation in JavaScript.

    The Problem

    Say we want to create the most quintessential example of object-oriented design principles: the Circle class. Imagine we are writing a Circle for a simple Canvas library. Among other things, we might want to know how to do the following:

    • Draw a given Circle to a given Canvas.
    • Keep track of the total number of Circles ever made.
    • Keep track of the radius of a given Circle, and how to enforce invariants on its value.
    • Calculate the area of a given Circle.

    Current JS idioms say that we should first create the constructor as a function, then add any properties we might want to the function itself, then replace the prototype property of that constructor with an object. This prototype object will contain all of the properties that instance objects created by our constructor should start with. For even a simple example, by the time you get it all typed out, this ends up being a lot of boilerplate:

    function Circle(radius) {
        this.radius = radius;
        Circle.circlesMade++;
    }
    
    Circle.draw = function draw(circle, canvas) { /* Canvas drawing code */ }
    
    Object.defineProperty(Circle, "circlesMade", {
        get: function() {
            return !this._count ? 0 : this._count;
        },
    
        set: function(val) {
            this._count = val;
        }
    });
    
    Circle.prototype = {
        area: function area() {
            return Math.pow(this.radius, 2) * Math.PI;
        }
    };
    
    Object.defineProperty(Circle.prototype, "radius", {
        get: function() {
            return this._radius;
        },
    
        set: function(radius) {
            if (!Number.isInteger(radius))
                throw new Error("Circle radius must be an integer.");
            this._radius = radius;
        }
    });
    

    Not only is the code cumbersome, it’s also far from intuitive. It requires having a non-trivial understanding of the way functions work, and how various installed properties make their way onto created instance objects. If this approach seems complicated, don’t worry. The whole point of this post is to show off a much simpler way of writing code that does all of this.

    Method Definition Syntax

    In a first attempt to clean this up, ES6 offered a new syntax for adding special properties to an object. While it was easy to add the area method to Circle.prototype above, it felt much heavier to add the getter/setter pair for radius. As JS moved towards a more object-oriented approach, people became interested in designing cleaner ways to add accessors to objects. We needed a new way of adding “methods” to an object exactly as if they had been added with obj.prop = method, without the weight of Object.defineProperty. People wanted to be able to do the following things easily:

    1. Add normal function properties to an object.
    2. Add generator function properties to an object.
    3. Add normal accessor function properties to an object.
    4. Add any of the above as if you had done it with [] syntax on the finished object. We’ll call these Computed property names.

    Some of these things couldn’t be done before. For example, there is no way to define a getter or setter with assignments to obj.prop. Accordingly, new syntax had to be added. You can now write code that looks like this:

    var obj = {
        // Methods are now added without a function keyword, using the name of the
        // property as the name of the function.
        method(args) { ... },
    
        // To make a method that's a generator instead, just add a '*', as normal.
        *genMethod(args) { ... },
    
        // Accessors can now go inline, with the help of |get| and |set|. You can
        // just define the functions inline. No generators, though.
    
        // Note that a getter installed this way must have no arguments
        get propName() { ... },
    
        // Note that a setter installed this way must have exactly one argument
        set propName(arg) { ... },
    
        // To handle case (4) above, [] syntax is now allowed anywhere a name would
        // have gone! This can use symbols, call functions, concatenate strings, or
        // any other expression that evaluates to a property id. Though I've shown
        // it here as a method, this syntax also works for accessors or generators.
        [functionThatReturnsPropertyName()] (args) { ... }
    };
    

    Using this new syntax, we can now rewrite our snippet above:

    function Circle(radius) {
        this.radius = radius;
        Circle.circlesMade++;
    }
    
    Circle.draw = function draw(circle, canvas) { /* Canvas drawing code */ }
    
    Object.defineProperty(Circle, "circlesMade", {
        get: function() {
            return !this._count ? 0 : this._count;
        },
    
        set: function(val) {
            this._count = val;
        }
    });
    
    Circle.prototype = {
        area() {
            return Math.pow(this.radius, 2) * Math.PI;
        },
    
        get radius() {
            return this._radius;
        },
        set radius(radius) {
            if (!Number.isInteger(radius))
                throw new Error("Circle radius must be an integer.");
            this._radius = radius;
        }
    };
    

    Pedantically, this code isn’t exactly identical to the snippet above. Method definitions in object literals are installed as configurable and enumerable, while the accessors installed in the first snippet will be non-configurable and non-enumerable. In practice, this is rarely noticed, and I decided to elide enumerability and configurability above for brevity.

    Still, it’s getting better, right? Unfortunately, even armed with this new method definition syntax, there’s not much we can do for the definition of Circle, as we have yet to define the function. There’s no way to get properties onto a function as you’re defining it.

    Class Definition Syntax

    Though this was better, it still didn’t satisfy people who wanted a cleaner solution to object-oriented design in JavaScript. Other languages have a construct for handling object-oriented design, they argued, and that construct is called a class.

    Fair enough. Let’s add classes, then.

    We want a system that will allow us to add methods to a named constructor, and add methods to its .prototype as well, so that they will appear on constructed instances of the class. Since we have our fancy new method definition syntax, we should definitely use it. Then, we only need a way to dfferentiate between what is generalized over all instances of the class, and what functions are specific to a given instance. In C++ or Java, the keyword for that is static. Seems as good as any. Let’s use it.

    Now it would be useful to have a way to designate one of the methods of the bunch to be the function that gets called as the constructor. In C++ or Java, that would be named the same as the class, with no return type. Since JS doesn’t have return types, and we need a .constructor property anyway, for backwards compatibility, let’s call that method constructor.

    Putting it together, we can rewrite our Circle class as it was always meant to be:

    class Circle {
        constructor(radius) {
            this.radius = radius;
            Circle.circlesMade++;
        };
    
        static draw(circle, canvas) {
            // Canvas drawing code
        };
    
        static get circlesMade() {
            return !this._count ? 0 : this._count;
        };
        static set circlesMade(val) {
            this._count = val;
        };
    
        area() {
            return Math.pow(this.radius, 2) * Math.PI;
        };
    
        get radius() {
            return this._radius;
        };
        set radius(radius) {
            if (!Number.isInteger(radius))
                throw new Error("Circle radius must be an integer.");
            this._radius = radius;
        };
    }
    

    Wow! Not only can we group everything related to a Circle together, but everything looks so… clean. This is definitely better than what we started with.

    Even so, some of you are likely to have questions or to find edge cases. I’ll try to anticipate and address some of these below:

    • What’s with the semicolons? – In an attempt to “make things look more like traditional classes,” we decided to go with a more traditional separator. Don’t like it? It’s optional. No delimiter is required.

    • What if I don’t want a constructor, but still want to put methods on created objects? – That’s fine. The constructor method is totally optional. If you don’t supply one, the default is as if you had typed constructor() {}.

    • Can constructor be a generator? – Nope! Adding a constructor that’s not a normal method will result in a TypeError. This includes both generators and accessors.

    • Can I define constructor with a computed property name? – Unfortunately not. That would be really hard to detect, so we don’t try. If you define a method with a computed property name that ends up being named constructor, you will still get a method named constructor, it just won’t be the class’s constructor function.

    • What if I change the value of Circle? Will that cause new Circle to misbehave? – Nope! Much like function expressions, classes get an internal binding of their given name. This binding cannot be changed by external forces, so no matter what you set the Circle variable to in the enclosing scope, Circle.circlesMade++ in the constructor will function as expected.

    • OK, but I could pass an object literal directly as a function argument. This new class thing looks like it won’t work anymore. – Luckily, ES6 also adds class expressions! They can be either named or unnamed, and will behave exactly the same way as described above, except they won’t create a variable in the scope in which you declare them.

    • What about those shenanigans above with enumerability and so on? – People wanted to make it so that you could install methods on objects, but that when you enumerated the object’s properties, you only got the added data properties of the object. Makes sense. Because of this, installed methods in classes are configurable, but not enumerable.

    • Hey, wait… what..? Where are my instance variables? What about static constants? – You caught me. They currently don’t exist in class definitions in ES6. Good news, though! Along with others involved in the spec process, I am a strong proponent of both static and const values being installable in class syntax. In fact, it’s already come up in spec meetings! I think we can look forward to more discussion of this in the future.

    • OK, even still, these are awesome! Can I use them yet? – Not exactly. There are polyfill options (especially Babel) so that you can play around with them today. Unfortunately, it’s going to be a little while before they are natively implemented in all major browsers. I’ve implemented everything we discussed here today in the Nightly version of Firefox, and it’s implemented but not enabled by default in Edge and Chrome. Unfortunately, it looks like there’s no current implementation in Safari.

    • Java and C++ have subclassing and a super keyword, but there’s nothing mentioned here. Does JS have that? – It does! However, that’s a whole other post’s worth of discussion. Check back with us later for an update about subclassing, where we’ll discuss more about the power of JavaScript classes.

    I would not have been able to implement classes without the guidance and enormous code review responsiblity of Jason Orendorff and Jeff Walden.

    Next week, Jason Orendorff returns from a week’s vacation and takes up the subject of let and const.

  4. ES6 In Depth: Proxies

    ES6 In Depth is a series on new features being added to the JavaScript programming language in the 6th Edition of the ECMAScript standard, ES6 for short.

    Here is the sort of thing we are going to do today.

    var obj = new Proxy({}, {
      get: function (target, key, receiver) {
        console.log(`getting ${key}!`);
        return Reflect.get(target, key, receiver);
      },
      set: function (target, key, value, receiver) {
        console.log(`setting ${key}!`);
        return Reflect.set(target, key, value, receiver);
      }
    });
    

    That’s a little complicated for a first example. I’ll explain all the parts later. For now, check out the object we created:

    > obj.count = 1;
        setting count!
    > ++obj.count;
        getting count!
        setting count!
        2
    

    What’s going on here? We are intercepting property accesses on this object. We are overloading the "." operator.

    How it’s done

    The best trick in computing is called virtualization. It’s a very general-purpose technique for doing astonishing things. Here’s how it works.

    1. Take any picture.

      (picture of a coal power plant)

      Photo credit: Martin Nikolaj Bech
    2. Draw an outline around something in the picture.

      (same photo, with the power plant circled)
    3. Now replace either everything inside the outline, or everything outside the outline, with something totally unexpected. There is just one rule, the Rule of Backwards Compatibility. Your replacement must behave enough like what was there before that nobody on the other side of the line notices that anything has changed.

      (the circled part is replaced with a wind farm)

      Photo credit: Beverley Goodwin.

    You’ll be familiar with this kind of hack from classic computer science films such as The Truman Show and The Matrix, where a person is inside the outline, and the rest of the world has been replaced with an elaborate illusion of normalcy.

    In order to satisfy the Rule of Backwards Compatibility, your replacement may need to be cunningly designed. But the real trick is in drawing the right outline.

    By outline, I mean an API boundary. An interface. Interfaces specify how two bits of code interact and what each part expects of the other. So if an interface is designed into the system, the outline is already drawn for you. You know you can replace either side, and the other side won’t care.

    It’s when there’s not an existing interface that you have to get creative. Some of the coolest software hacks of all time have involved drawing an API boundary where previously there was none, and bringing that interface into existence via a prodigious engineering effort.

    Virtual memory, Hardware virtualization, Docker, Valgrind, rr—to various degrees all of these projects involved driving new and rather unexpected interfaces into existing systems. In some cases, it took years and new operating system features and even new hardware to make the new boundary work well.

    The best virtualization hacks bring with them a new understanding of whatever’s being virtualized. To write an API for something, you have to understand it. Once you understand, you can do amazing things.

    ES6 introduces virtualization support for JavaScript’s most fundamental concept: the object.

    What is an object?

    No, really. Take a moment. Think it over. Scroll down when you know what an object is.

    (picture of Auguste Rodin’s sculpture, The Thinker)

    Photo credit: Joe deSousa.

    This question is too hard for me! I’ve never heard a really satisfying definition.

    Is that surprising? Defining fundamental concepts is always hard—check out the first few definitions in Euclid’s Elements sometime. The ECMAScript language specification is in good company, therefore, when it unhelpfully defines an object as a “member of the type Object.”

    Later, the spec adds that “An Object is a collection of properties.” That’s not bad. If you want a definition, that will do for now. We’ll come back to it later.

    I said before that to write an API for something, you have to understand it. So in a way, I’ve promised that if we get through all this, we’re going to understand objects better, and we’ll be able to do amazing things.

    So let’s follow in the footsteps of the ECMAScript standard committee and see what it would take to define an API, an interface, for JavaScript objects. What sort of methods do we need? What can objects do?

    That depends somewhat on the object. DOM Element objects can do certain things; AudioNode objects do other things. But there are a few fundamental abilities all objects share:

    • Objects have properties. You can get and set properties, delete them, and so on.
    • Objects have prototypes. This is how inheritance works in JS.
    • Some objects are functions or constructors. You can call them.

    Almost everything JS programs do with objects is done using properties, prototypes, and functions. Even the special behavior of an Element or AudioNode object is accessed by calling methods, which are just inherited function properties.

    So when the ECMAScript standard committee defined a set of 14 internal methods, the common interface for all objects, it should come as no surprise that they ended up focusing on these three fundamental things.

    The full list can be found in tables 5 and 6 of the ES6 standard. Here I’ll just describe a few. The weird double brackets, [[ ]], emphasize that these are internal methods, hidden from ordinary JS code. You can’t call, delete, or overwrite these like ordinary methods.

    • obj.[[Get]](key, receiver) – Get the value of a property.

      Called when JS code does: obj.prop or obj[key].

      obj is the object currently being searched; receiver is the object where we first started searching for this property. Sometimes we have to search several objects. obj might be an object on receiver’s prototype chain.

    • obj.[[Set]](key, value, receiver) – Assign to a property of an object.

      Called when JS code does: obj.prop = value or obj[key] = value.

      In an assignment like obj.prop += 2, the [[Get]] method is called first, and the [[Set]] method afterwards. Same goes for ++ and --.

    • obj.[[HasProperty]](key) – Test whether a property exists.

      Called when JS code does: key in obj.

    • obj.[[Enumerate]]() – List obj’s enumerable properties.

      Called when JS code does: for (key in obj) ....

      This returns an iterator object, and that’s how a forin loop gets an object’s property names.

    • obj.[[GetPrototypeOf]]() – Return obj’s prototype.

      Called when JS code does: obj.__proto__ or Object.getPrototypeOf(obj).

    • functionObj.[[Call]](thisValue, arguments) – Call a function.

      Called when JS code does: functionObj() or x.method().

      Optional. Not every object is a function.

    • constructorObj.[[Construct]](arguments, newTarget) – Invoke a constructor.

      Called when JS code does: new Date(2890, 6, 2), for example.

      Optional. Not every object is a constructor.

      The newTarget argument plays a role in subclassing. We’ll cover it in a future post.

    Maybe you can guess at some of the other seven.

    Throughout the ES6 standard, wherever possible, any bit of syntax or builtin function that does anything with objects is specified in terms of the 14 internal methods. ES6 drew a clear boundary around the brains of an object. What proxies let you do is replace the standard kind of brains with arbitrary JS code.

    When we start talking about overriding these internal methods in a moment, remember, we’re talking about overriding the behavior of core syntax like obj.prop, builtin functions like Object.keys(), and more.

    Proxy

    ES6 defines a new global constructor, Proxy. It takes two arguments: a target object and a handler object. So a simple example would look like this:

    var target = {}, handler = {};
    var proxy = new Proxy(target, handler);
    

    Let’s set aside the handler object for a moment and focus on how proxy and target are related.

    I can tell you how proxy is going to behave in one sentence. All of proxy’s internal methods are forwarded to target. That is, if something calls proxy.[[Enumerate]](), it’ll just return target.[[Enumerate]]().

    Let’s try it out. We’ll do something that causes proxy.[[Set]]() to be called.

    proxy.color = "pink";
    

    OK, what just happened? proxy.[[Set]]() should have called target.[[Set]](), so that should have made a new property on target. Did it?

    > target.color
        "pink"
    

    It did. And the same goes for all the other internal methods. This proxy will, for the most part, behave exactly the same as its target.

    There are limits to the fidelity of the illusion. You’ll find that proxy !== target. And a proxy will sometimes flunk type checks that the target would pass. Even if a proxy’s target is a DOM Element, for example, the proxy isn’t really an Element; so something like document.body.appendChild(proxy) will fail with a TypeError.

    Proxy handlers

    Now let’s return to the handler object. This is what makes proxies useful.

    The handler object’s methods can override any of the proxy’s internal methods.

    For example, if you’d like to intercept all attempts to assign to an object’s properties, you can do that by defining a handler.set() method:

    var target = {};
    var handler = {
      set: function (target, key, value, receiver) {
        throw new Error("Please don't set properties on this object.");
      }
    };
    var proxy = new Proxy(target, handler);
    
    > proxy.name = "angelina";
        Error: Please don't set properties on this object.
    

    The full list of handler methods is documented on the MDN page for Proxy. There are 14 methods, and they line up with the 14 internal methods defined in ES6.

    All handler methods are optional. If an internal method is not intercepted by the handler, then it’s forwarded to the target, as we saw before.

    Example: “Impossible” auto-populating objects

    We now know enough about proxies to try using them for something really weird, something that’s impossible without proxies.

    Here’s our first exercise. Make a function Tree() that can do this:

    > var tree = Tree();
    > tree
        { }
    > tree.branch1.branch2.twig = "green";
    > tree
        { branch1: { branch2: { twig: "green" } } }
    > tree.branch1.branch3.twig = "yellow";
        { branch1: { branch2: { twig: "green" },
                     branch3: { twig: "yellow" }}}
    

    Note how all the intermediate objects branch1, branch2, and branch3, are magically autocreated when they’re needed. Convenient, right? How could it possibly work?

    Until now, there’s no way it could work. But with proxies this is only a few lines of code. We just need to tap into tree.[[Get]](). If you like a challenge, you might want to try implementing this yourself before reading on.

    (picture of a tap in a maple tree)

    Not the right way to tap into a tree in JS. Photo credit: Chiot’s Run.

    Here’s my solution:

    function Tree() {
      return new Proxy({}, handler);
    }
    
    var handler = {
      get: function (target, key, receiver) {
        if (!(key in target)) {
          target[key] = Tree();  // auto-create a sub-Tree
        }
        return Reflect.get(target, key, receiver);
      }
    };
    

    Note the call to Reflect.get() at the end. It turns out there’s an extremely common need, in proxy handler methods, to be able to say “now just do the default behavior of delegating to target.” So ES6 defines a new Reflect object with 14 methods on it that you can use to do exactly that.

    Example: A read-only view

    I think I may have given the false impression that proxies are easy to use. Let’s do one more example to see if that’s true.

    This time our assignment is more complex: we have to implement a function, readOnlyView(object), that takes any object and returns a proxy that behaves just like that object, except without the ability to mutate it. So, for example, it should behave like this:

    > var newMath = readOnlyView(Math);
    > newMath.min(54, 40);
        40
    > newMath.max = Math.min;
        Error: can't modify read-only view
    > delete newMath.sin;
        Error: can't modify read-only view
    

    How can we implement this?

    The first step is to intercept all internal methods that would modify the target object if we let them through. There are five of those.

    function NOPE() {
      throw new Error("can't modify read-only view");
    }
    
    var handler = {
      // Override all five mutating methods.
      set: NOPE,
      defineProperty: NOPE,
      deleteProperty: NOPE,
      preventExtensions: NOPE,
      setPrototypeOf: NOPE
    };
    
    function readOnlyView(target) {
      return new Proxy(target, handler);
    }
    

    This works. It prevents assignment, property definition, and so on via the read-only view.

    Are there any loopholes in this scheme?

    The biggest problem is that the [[Get]] method, and others, may still return mutable objects. So even if some object x is a read-only view, x.prop may be mutable! That’s a huge hole.

    To plug it, we must add a handler.get() method:

    var handler = {
      ...
    
      // Wrap other results in read-only views.
      get: function (target, key, receiver) {
        // Start by just doing the default behavior.
        var result = Reflect.get(target, key, receiver);
    
        // Make sure not to return a mutable object!
        if (Object(result) === result) {
          // result is an object.
          return readOnlyView(result);
        }
        // result is a primitive, so already immutable.
        return result;
      },
    
      ...
    };
    

    This is not sufficient either. Similar code is needed for other methods, including getPrototypeOf and getOwnPropertyDescriptor.

    Then there are further problems. When a getter or method is called via this kind of proxy, the this value passed to the getter or method will typically be the proxy itself. But as we saw earlier, many accessors and methods perform a type check that the proxy won’t pass. It would be better to substitute the target object for the proxy here. Can you figure out how to do it?

    The lesson to draw from this is that creating a proxy is easy, but creating a proxy with intuitive behavior is quite hard.

    Odds and ends

    • What are proxies really good for?

      They’re certainly useful whenever you want to observe or log accesses to an object. They’ll be handy for debugging. Testing frameworks could use them to create mock objects.

      Proxies are useful if you need behavior that’s just slightly past what an ordinary object can do: lazily populating properties, for example.

      I almost hate to bring this up, but one of the best ways to see what’s going on in code that uses proxies… is to wrap a proxy’s handler object in another proxy that logs to the console every time a handler method is accessed.

      Proxies can be used to restrict access to an object, as we did with readOnlyView. That sort of use case is rare in application code, but Firefox uses proxies internally to implement security boundaries between different domains. They’re a key part of our security model.

    • Proxies ♥ WeakMaps. In our readOnlyView example, we create a new proxy every time an object is accessed. It could save a lot of memory to cache every proxy we create in a WeakMap, so that however many times an object is passed to readOnlyView, only a single proxy is created for it.

      This is one of the motivating use cases for WeakMap.

    • Revocable proxies. ES6 also defines another function, Proxy.revocable(target, handler), that creates a proxy, just like new Proxy(target, handler), except this proxy can be revoked later. (Proxy.revocable returns an object with a .proxy property and a .revoke method.) Once a proxy is revoked, it simply doesn’t work anymore; all its internal methods throw.

    • Object invariants. In certain situations, ES6 requires proxy handler methods to report results that are consistent with the target object’s state. It does this in order to enforce rules about immutability across all objects, even proxies. For example, a proxy can’t claim to be inextensible unless its target really is inextensible.

      The exact rules are too complex to go into here, but if you ever see an error message like "proxy can't report a non-existent property as non-configurable", this is the cause. The most likely remedy is to change what the proxy is reporting about itself. Another possibility is to mutate the target on the fly to reflect whatever the proxy is reporting.

    What is an object now?

    I think where we left it was: “An Object is a collection of properties.”

    I’m not totally happy with this definition, even taking for granted that we throw in prototypes and callability as well. I think the word “collection” is too generous, given how poorly defined a proxy can be. Its handler methods could do anything. They could return random results.

    By figuring out what an object can do, standardizing those methods, and adding virtualization as a first-class feature that everyone can use, the ECMAScript standard committee has expanded the realm of possibilities.

    Objects can be almost anything now.

    Maybe the most honest answer to the question “What is an object?” now is to take the 12 required internal methods as a definition. An object is something in a JS program that has a [[Get]] operation, a [[Set]] operation, and so on.


    Do we understand objects better after all that? I’m not sure! Did we do amazing things? Yeah. We did things that were never possible in JS before.

    Can I use Proxies today?

    Nope! Not on the Web, anyway. Only Firefox and Microsoft Edge support proxies, and there is no polyfill.

    Using proxies in Node.js or io.js requires both an off-by-default option (--harmony_proxies) and the harmony-reflect polyfill, since V8 implements an older version of the Proxy specification. (A previous version of this article had incorrect information about this. Thanks to Mörre and Aaron Powell for correcting my mistakes in the comments.)

    So feel free to experiment with proxies! Create a hall of mirrors where there seem to be thousands of copies of every object, all alike, and it’s impossible to debug anything! Now is the time. There’s little danger of your ill-advised proxy code escaping into production… yet.

    Proxies were first implemented in 2010, by Andreas Gal, with code reviews by Blake Kaplan. The standard committee then completely redesigned the feature. Eddy Bruel implemented the new spec in 2012.

    I implemented Reflect, with code reviews by Jeff Walden. It’ll be in Firefox Nightly starting this weekend—all except Reflect.enumerate(), which is not implemented yet.

    Next up, we’ll be talking about the most controversial feature in ES6, and who better to present it than the person who’s implementing it in Firefox? So please join us next week as Mozilla engineer Eric Faust presents ES6 classes in depth.

  5. ES6 In Depth: Generators, continued

    ES6 In Depth is a series on new features being added to the JavaScript programming language in the 6th Edition of the ECMAScript standard, ES6 for short.

    Welcome back to ES6 In Depth! I hope you had as much fun as I did during our summer break. But the life of a programmer cannot be all fireworks and lemonade. It’s time to pick up where we left off—and I’ve got the perfect topic to resume with.

    Back in May, I wrote about generators, a new kind of function introduced in ES6. I called them the most magical feature in ES6. I talked about how they might be the future of asynchronous programming. And then I wrote this:

    There is more to say about generators… But I think this post is long and bewildering enough for now. Like generators themselves, we should pause, and take up the rest another time.

    Now is the time.

    You can find part 1 of this article here. It’s probably best to read that before reading this. Go on, it’s fun. It’s… a little long and bewildering. But there’s a talking cat!

    A quick revue

    Last time, we focused on the basic behavior of generators. It’s a little strange, perhaps, but not hard to understand. A generator-function is a lot like a regular function. The main difference is that the body of a generator-function doesn’t run all at once. It runs a little bit at a time, pausing each time execution reaches a yield expression.

    There’s a detailed explanation in part 1, but we never did a thorough worked example of how all the parts come together. Let’s do that now.

    function* someWords() {
      yield "hello";
      yield "world";
    }
    
    for (var word of someWords()) {
      alert(word);
    }
    

    This script is straightforward enough. But if you could observe everything that happens here, as though all the various bits of code were characters in a play, that would be a rather different script. It might go something like this:

    SCENE - INTERIOR COMPUTER, DAY
    
    FOR LOOP stands alone onstage, wearing a hard hat and
    carrying a clipboard, all business.
    
                              FOR LOOP
                             (calling)
                            someWords()!
                                  
    The GENERATOR appears: a tall, brass, clockwork gentleman.
    It looks friendly enough, but it's still as a statue.
    
                              FOR LOOP
                   (clapping her hands smartly)
               All right! Let's get some stuff done.
                         (to the generator)
                              .next()!
    
    The GENERATOR springs to life.
    
                             GENERATOR
                   {value: "hello", done: false}
    
    It freezes in a goofy pose.
    
                              FOR LOOP
                               alert!
    
    Enter ALERT at a sprint, wide-eyed and breathless. We sense
    he's always like that.
    
                              FOR LOOP
                       Tell the user "hello".
    
    ALERT turns around and sprints offstage.
    
                               ALERT
                       (offstage, screaming)
                          Stop everything!
              The web page at hacks.mozilla.org says,
                              "hello"!
    
    A few seconds' pause, then ALERT races back on, crossing all
    the way over to FOR LOOP and skidding to a halt.
    
                               ALERT
                         The user says OK.
    
                              FOR LOOP
                   (clapping her hands smartly)
               All right! Let's get some stuff done.
                  (turning back to the GENERATOR)
                              .next()!
    
    The GENERATOR again springs to life.
    
                             GENERATOR
                   {value: "world", done: false}
    
    It freezes in a different goofy pose.
    
                              FOR LOOP
                               alert!
    
                               ALERT
                         (already running)
                               On it!
                       (offstage, screaming)
                          Stop everything!
              The web page at hacks.mozilla.org says,
                              "world"!
    
    Again, a pause, then ALERT trudges back onstage, suddenly
    crestfallen.
    
                               ALERT
                   The user says OK again, but...
                 but please prevent this page from
                   creating additional dialogues.
    
    He exits, pouting.
    
                              FOR LOOP
                   (clapping her hands smartly)
               All right! Let's get some stuff done.
                  (turning back to the GENERATOR)
                              .next()!
    
    The GENERATOR springs to life a third time.
    
                             GENERATOR
                           (with dignity)
                   {value: undefined, done: true}
    
    Its head comes to rest on its chest and the lights go out of
    its eyes. It will never move again.
    
                              FOR LOOP
                      Time for my lunch break.
    
    She exits.
    
    After a while, the GARBAGE COLLECTOR enters, picks up the
    lifeless GENERATOR, and carries it offstage.
    

    All right—it’s not exactly Hamlet. But you get the picture.

    As you can see in the play, when a generator object first appears, it is paused. It wakes up and runs for a bit each time its .next() method is called.

    The action is synchronous and single-threaded. Note that only one of these characters is actually doing anything at any given time. The characters never interrupt each other or talk over one another. They take turns speaking, and whoever’s talking can go on as long as they want. (Just like Shakespeare!)

    And some version of this drama unfolds each time a generator is fed to a forof loop. There is always this sequence of .next() method calls that do not appear anywhere in your code. Here I’ve put it all onstage, but for you and your programs, all this will happen behind the scenes, because generators and the forof loop were designed to work together, via the iterator interface.

    So to summarize everything up to this point:

    • Generator objects are polite brass robots that yield values.
    • Each robot’s programming consists of a single chunk of code: the body of the generator function that created it.

    How to shut down a generator

    Generators have several fiddly extra features that I didn’t cover in part 1:

    • generator.return()
    • the optional argument to generator.next()
    • generator.throw(error)
    • yield*

    I skipped them mainly because without understanding why those features exist, it’s hard to care about them, much less keep them all straight in your head. But as we think more about how our programs will use generators, we’ll see the reasons.

    Here’s a pattern you’ve probably used at some point:

    function doThings() {
      setup();
      try {
        // ... do some things ...
      } finally {
        cleanup();
      }
    }
    
    doThings();
    

    The cleanup might involve closing connections or files, freeing system resources, or just updating the DOM to turn off an “in progress” spinner. We want this to happen whether our work finishes successfully or not, so it goes in a finally block.

    How would this look in a generator?

    function* produceValues() {
      setup();
      try {
        // ... yield some values ...
      } finally {
        cleanup();
      }
    }
    
    for (var value of produceValues()) {
      work(value);
    }
    

    This looks all right. But there is a subtle issue here: the call work(value) isn’t inside the try block. If it throws an exception, what happens to our cleanup step?

    Or suppose the forof loop contains a break or return statement. What happens to the cleanup step then?

    It executes anyway. ES6 has your back.

    When we first discussed iterators and the forof loop, we said the iterator interface contains an optional .return() method which the language automatically calls whenever iteration exits before the iterator says it’s done. Generators support this method. Calling myGenerator.return() causes the generator to run any finally blocks and then exit, just as if the current yield point had been mysteriously transformed into a return statement.

    Note that the .return() is not called automatically by the language in all contexts, only in cases where the language uses the iteration protocol. So it is possible for a generator to be garbage collected without ever running its finally block.

    How would this feature play out on stage? The generator is frozen in the middle of a task that requires some setup, like building a skyscraper. Suddenly someone throws an error! The for loop catches it and sets it aside. She tells the generator to .return(). The generator calmly dismantles all its scaffolding and shuts down. Then the for loop picks the error back up, and normal exception handling continues.

    Generators in charge

    So far, the conversations we’ve seen between a generator and its user have been pretty one-sided. To break with the theater analogy for a second:

    (A fake screenshot of iPhone text messages between a generator and its user, with the user just saying 'next' repeatedly and the generator replying with values.)

    The user is in charge. The generator does its work on demand. But this isn’t the only way to program with generators.

    In part 1, I said that generators could be used for asynchronous programming. Things you currently do with asynchronous callbacks or promise chaining could be done with generators instead. You may have wondered how exactly that is supposed to work. Why is the ability to yield (which after all is a generator’s only special power) sufficient? After all, asynchronous code doesn’t just yield. It makes stuff happen. It calls for data from files and databases. It fires off requests to servers. And then it returns to the event loop to wait for those asynchronous processes to finish. How exactly will generators do this? And without callbacks, how does the generator receive data from those files and databases and servers when it comes in?

    To start working toward the answer, consider what would happen if we just had a way for the .next() caller to pass a value back into the generator. With just this one change, we could have a whole new kind of conversation:

    (A fake screenshot of iPhone text messages between a generator and its caller; each value the generator yields is an imperious demand, and the caller passes whatever the generator wants as an argument the next time it calls .next().)

    And a generator’s .next() method does in fact take an optional argument, and the clever bit is that the argument then appears to the generator as the value returned by the yield expression. That is, yield isn’t a statement like return; it’s an expression that has a value, once the generator resumes.

      var results = yield getDataAndLatte(request.areaCode);
    

    This does a lot of things for a single line of code:

    • It calls getDataAndLatte(). Let’s say that function returns the string "get me the database records for area code..." that we saw in the screenshot.
    • It pauses the generator, yielding the string value.
    • At this point, any amount of time could pass.
    • Eventually, someone calls .next({data: ..., coffee: ...}). We store that object in the local variable results and continue on the next line of code.

    To show that in context, here’s code for the entire conversation shown above:

    function* handle(request) {
      var results = yield getDataAndLatte(request.areaCode);
      results.coffee.drink();
      var target = mostUrgentRecord(results.data);
      yield updateStatus(target.id, "ready");
    }
    

    Note how yield still just means exactly what it meant before: pause the generator and pass a value back to the caller. But how things have changed! This generator expects very specific supportive behavior from its caller. It seems to expect the caller to act like an administrative assistant.

    Ordinary functions are not usually like that. They tend to exist to serve their caller’s needs. But generators are code you can have a conversation with, and that makes for a wider range of possible relationships between generators and their callers.

    What might this administrative assistant generator-runner look like? It doesn’t have to be all that complicated. It might look like this.

    function runGeneratorOnce(g, result) {
      var status = g.next(result);
      if (status.done) {
        return;  // phew!
      }
    
      // The generator has asked us to fetch something and
      // call it back when we're done.
      doAsynchronousWorkIncludingEspressoMachineOperations(
        status.value,
        (error, nextResult) => runGeneratorOnce(g, nextResult));
    }
    

    To get the ball rolling, we would have to create a generator and run it once, like this:

      runGeneratorOnce(handle(request), undefined);
    

    In May, I mentioned Q.async() as an example of a library that treats generators as asynchronous processes and automatically runs them as needed. runGeneratorOnce is that sort of thing. In practice, generator will not yield strings spelling out what they need the caller to do. They will probably yield Promise objects.

    If you already understand promises, and now you understand generators, you might want to try modifying runGeneratorOnce to support promises. It’s a difficult exercise, but once you’re done, you’ll be able to write complex asynchronous algorithms using promises as straight-line code, not a .then() or a callback in sight.

    How to blow up a generator

    Did you notice how runGeneratorOnce handles errors? It ignores them!

    Well, that’s not good. We would really like to report the error to the generator somehow. And generators support this too: you can call generator.throw(error) rather than generator.next(result). This causes the yield expression to throw. Like .return(), the generator will typically be killed, but if the current yield point is in a try block, then catch and finally blocks are honored, so the generator may recover.

    Modifying runGeneratorOnce to make sure .throw() gets called appropriately is another great exercise. Keep in mind that exceptions thrown inside generators are always propagated to the caller. So generator.throw(error) will throw error right back at you unless the generator catches it!

    This completes the set of possibilities when a generator reaches a yield expression and pauses:

    • Someone may call generator.next(value). In this case, the generator resumes execution right where it left off.
    • Someone may call generator.return(), optionally passing a value. In this case, the generator does not resume whatever it was doing. It executes finally blocks only.
    • Someone may call generator.throw(error). The generator behaves as if the yield expression were a call to a function that threw error.
    • Or, maybe nobody will do any of those things. The generator might stay frozen forever. (Yes, it is possible for a generator to enter a try block and simply never execute the finally block. A generator can even be reclaimed by the garbage collector while it’s in this state.)

    This is not much more complicated than a plain old function call. Only .return() is really a new possibility.

    In fact, yield has a lot in common with function calls. When you call a function, you’re temporarily paused, right? The function you called is in control. It might return. It might throw. Or it might just loop forever.

    Generators working together

    Let me show off one more feature. Suppose we write a simple generator-function to concatenate two iterable objects:

    function* concat(iter1, iter2) {
      for (var value of iter1) {
        yield value;
      }
      for (var value of iter2) {
        yield value;
      }
    }
    

    ES6 provides a shorthand for this:

    function* concat(iter1, iter2) {
      yield* iter1;
      yield* iter2;
    }
    

    A plain yield expression yields a single value; a yield* expression consumes an entire iterator and yields all values.

    The same syntax also solves another funny problem: the problem of how to call a generator from within a generator. In ordinary functions, we can scoop up a bunch of code from one function and refactor it into a separate function, without changing behavior. Obviously we’ll want to refactor generators too. But we’ll need a way to call the factored-out subroutine and make sure that every value we were yielding before is still yielded, even though it’s a subroutine that’s producing those values now. yield* is the way to do that.

    function* factoredOutChunkOfCode() { ... }
    
    function* refactoredFunction() {
      ...
      yield* factoredOutChunkOfCode();
      ...
    }
    

    Think of one brass robot delegating subtasks to another. You can see how important this idea is for writing large generator-based projects and keeping the code clean and organized, just as functions are crucial for organizing synchronous code.

    Exeunt

    Well, that’s it for generators! I hope you enjoyed that as much as I did, too. It’s good to be back.

    Next week, we’ll talk about yet another mind-blowing feature that’s totally new in ES6, a new kind of object so subtle, so tricky, that you may end up using one without even knowing it’s there. Please join us next week for a look at ES6 proxies in depth.

  6. Developer Edition 41: View source in a tab, screenshot elements, HAR files, and more

    When we introduced the new Performance tools a few weeks ago, we also talked about how the Firefox Dev Tools team had spent a lot of time focusing on user feedback and what we call ‘polish’ bugs – things reported via our UserVoice feedback channel and Bugzilla. Even though the Firefox 41 was a short release cycle for us, this focus on user feedback continues to pay off — several new features that our community had been asking for landed in time for the release. Here’s a closer look:

    Screenshot the selected node in the Inspector

    New contributor Léon McGregor implemented an interesting suggestion that was posted in UserVoice. This functionality has been available via the gcli ‘screenshot’ command for quite some time, but is much more discoverable and useful as a context menu item. When the screenshot is created, Firefox copies it to your configured downloads directory.

    Create a screenshot of the currently selected element.

    Create a screenshot of the currently selected element.

    View source in tab

    Starting with Firefox 41, when you right-click and select View Page Source, the html source view will open in a tab instead of a new window. This was a hugely popular request and we would have shipped it earlier but what started out as a seemingly simple change was actually quite involved: See the bug linked below for all the gory details. Importantly, we have also ensured that View Page Source provides you with the source of the page as-is from Firefox’s cache – we do not fetch a new version.

    'View source' always opens in a tab now.

    View Page Source always opens in a tab now.

    Add Rules button

    It’s very convenient to be able to add a new rule to the Inspector as you work, and this is a feature from Firebug that users have requested for some time. During this last cycle, we spent some time polishing our implementation, and provided the convenience of a UI button in addition to the context menu command.

    We've added a button to the inspector so you can quickly add a new css rule.

    We’ve added a button to the inspector so you can quickly add a new css rule.

    “Copy as HAR” and “Save all as HAR”

    Another feature from Firebug that is particularly popular with Selenium users is the ability to export HAR archives for the current page.

    You can now export HAR archives directly from the network monitor.

    You can now export HAR archives directly from the network monitor.

    Other notable changes

    In total, 140 Developer Tools bugs have been fixed in Firefox since June 1st. On behalf of the team, I’d like to thank all of the people who reported bugs, tested patches, and spent many hours working to improve this version of Firefox Developer Edition, and especially these contributors that fixed bugs: edoardo.putti, fayolle-florent, 15electronicmotor, veeti.paananen, sr71pav, sjakthol, ntim, MattN, lemcgregor3, and indiasuny000. Thanks!.

    • Bug 1164210 – $$() should return a true Array
    • Bug 1077339 – Display keyboard shortcuts when hovering panel tabs
    • Bug 1163183 – Show HTML5 Forms pseudo elements in the rule view
    • Bug 1165576 – Netmonitor theme refresh
    • Bug 1049888 – Make the storage actor work in e10s and Firefox OS
    • Bug 987365 – Add pseudo-class lock options to rule view
    • Bug 1059882 – Frame selection command button should be visible by default
    • Bug 1143224 – Opening the netmonitor slows down requests on the page
    • Bug 1119133 – Keyboard shortcut to toggle devtools docking mode between last two positions
    • Bug 1024693 – Copy CSS declarations
    • Bug 1050691 – Click on a function on the console should go to the debugger

    Download Firefox Developer Edition 41 now. Let us know what you think and what you’d like to see in future releases. We’re paying attention.

  7. Compacting Garbage Collection in SpiderMonkey

    Overview

    Compacting is a new feature of our garbage collector, released in Firefox 38, that allows us to reduce external fragmentation in the JavaScript heap. The aim is to use less memory in general and to be able to recover from more out-of-memory situations. So far, we have only implemented compacting for JavaScript objects, which are one of several kinds of garbage-collected cells in the heap.

    The problem

    The JavaScript heap is made up of 4K blocks of memory called arenas, each of which is divided into fixed-size cells. Different arenas are used to allocate different kinds of cells; each arena only contains cells of the same size and kind.

    The heap contains various kinds of cell, including those for JavaScript objects, strings, and symbols, as well as several internal kinds such as scripts (used to represent units of JS code), shapes (used to determine the layout of object properties in memory), and jitcode (compiled JIT code). Of these, object cells usually take up the most memory.

    An arena cannot be freed while it contains any live cells. Cells allocated at the same time may have different lifetimes and so a heap may end up in a state where there are many arenas that contain only a few cells. New cells of the same kind can be allocated into this space, but the space cannot be used for cells of a different kind or returned to the operating system if memory is low.

    Here is a simplified diagram of some data on the heap showing arenas containing two different kinds of cell:

    heap layout smallest

    Note that if the free space in arena 3 were used to hold the cells in arena 5, we could free up a whole arena.

    Measuring wasted heap space

    You can see how much memory these free cells are taking up by navigating to about:memory and hitting the ‘Measure’ button. The totals for the different kinds of cell are shown under the section js-main-runtime-gc-heap-committed/unused/gc-things. (If you’re not used to interpreting the about:memory reports, there is some documentation here).

    Here’s a screenshot of the whole js-main-runtime-gc-heap-committed section with compacting GC disabled, showing the difference between ‘used’ and ‘unused’ sizes:

    unused heap screenshot

    I made some rough measurements of my normal browsing profile with and without compacting GC (details of how to do this are below at the end of the post). The profile consisted of Google Mail, Calendar, many bugzilla tabs and various others (~50 tabs total), and I obtained the following readings:

    Total explicit allocations Unused cells
    Before compacting 1,324.46 MiB 69.58 MiB
    After compacting 1,296.28 MiB 40.18 MiB

    This shows a reduction of 29.4MiB (mebibytes) worth of explicit allocations. That’s only about 2% of total allocations, but accounts for over 8% of the space taken up by the JS heap.

    How does compacting work?

    To free up this space we have to allow the GC to move cells between arenas. That way it can consolidate the live cells in fewer arenas and reuse the unused space. Of course, this is easier said than done, as every pointer to a moved cell must be updated. Missing a single one is a sure-fire way to make the browser crash!

    Also, this is a potentially expensive operation as we have to scan many cells to find the pointers we need to update. Therefore the idea is to compact the heap only when memory is low or the user is inactive.

    The algorithm works in three phases:

    1. Select the cells to move.
    2. Move the cells.
    3. Update the pointers to those cells.

    Selecting the cells to move

    We want to move the minimum amount of data and we want to do it without allocating any more memory, as we may be doing this when we don’t have any free memory. To do this, we take all the arenas with free space in them and put them in a list arranged in decreasing order of the number of free cells they contain. We split this list into two parts at the first point at which the preceding arenas have enough free cells to contain the used cells in the subsequent arenas. We will move all the cells out of the subsequent arenas.

    Moving the cells

    We allocate a new cell from one of the arenas we are not moving. The previous step ensures there is always enough space for this. Then we copy the data over from the original location.

    In some cases, we know the cell contains pointers to itself and these are updated at this point. The browser may have external references to some kinds of object and so we also call an optional hook here to allow these to be updated.

    When we have moved a cell, we update the original location with a forwarding pointer to the new location, so we can find it later. This also marks the cell, indicating to the GC that the cell has been moved, when updating pointers in the next phase.

    Updating pointers to moved cells

    This is the most demanding part of the compacting process. In general, we don’t know which cells may contain pointers to cells we have moved, so it seems we have to iterate through all cells in the heap. This would be very expensive.

    We cut down this cost in a number of ways. Firstly, note that the heap is split into several zones (there is a zone per browser tab, and others for system use). Compacting is performed per-zone, since in general cells do not have cross-zone pointers (these are handled separately). Compacting per zone allows us to spread the total cost over many incremental slices.

    Secondly, not every kind of cell can contain pointers to every other kind of cell (indeed not all kinds of cells can contain pointers) so some kinds of cell can be excluded from the search.

    Finally, we can parallelise this work and make use of all CPU resources available.

    It’s important to note that this work was enabled by our shift to exact stack rooting, described in this blog post. It is only possible to move objects if we know which stack locations are roots, otherwise we could overwrite unrelated data on the stack if it happened to look like a moved cell pointer.

    Scheduling heap compaction

    As mentioned earlier, compacting GC doesn’t run every time we collect. Currently it is triggered on three events:

    • We ran out of memory and we’re performing a last-ditch attempt to free up some space
    • The OS has sent us a memory pressure event
    • The user has been inactive for some length of time (currently 20 seconds)

    The first two should allow us to avoid some out-of-memory situations, while the last aims to free up memory without affecting the user’s browsing experience.

    Conclusion

    Hopefully this has explained the problem compacting GC is trying to solve, and how it’s done.

    One unexpected benefit of implementing compacting GC is that it showed us a couple of places where we weren’t correctly tracing cell pointers. Errors like this can cause hard-to-reproduce crashes or potential security vulnerabilities, so this was an additional win.

    Ideas for future work

    The addition of compacting is an important step in improving our GC, but it’s not the end by any means. There are several ways in which we can continue to develop this:

    Currently we only compact cells corresponding to JavaScript objects, but there are several other kinds of cells in the heap. Moving these would bring greater memory savings.

    Is it possible to determine in advance which cells contain pointers to cells we want to move? If we had this information we could cut the cost of compacting. One possibility is to scan the heap in the background to determine this information, but we would need to be able to detect changes made by the mutator.

    The current algorithm mixes together cells allocated at different times. Cells with similar lifetimes are often allocated at the same time, so this may not be the best strategy.

    If compacting can be made fast enough, we might be able to do it whenever the collector sees a certain level of fragmentation in the heap.

    How to measure heap space freed up by compacting

    To measure roughly how much space is freed by compacting, you can perform the following steps:

    1. Disable compacting by navigating to about:config and setting javascript.options.mem.gc_compacting to false.
    2. It makes it easier to disable multiprocess Firefox as well at this point. This can be done from the main Preferences page.
    3. Restart the browser and open some tabs. I used ‘Reload all tabs’ to open all my pages from last time. Wait for everything to load.
    4. Open about:memory and force a full GC by clicking ‘Minimize memory usage’ and then click ‘Measure.’ Since memory usage can take a while to settle down, I repeated this a few times until I got a consistent number.
    5. Note the total ‘explicit’ size and that of js-main-runtime-gc-heap-committed/unused/gc-things.
    6. Enable compacting again by setting javascript.options.mem.gc_compacting to true. There is no need to restart for this to take effect.
    7. Click ‘Minimize memory usage’ again and then ‘Measure.’
    8. Compare the new readings to the previous.

    This does not give precise readings as all kinds of things might be happening in the background, but it can provide a good ballpark figure.

  8. How fast are web workers?

    The next version of Firefox OS, the mobile operating system, will unleash the power of devices by taking full advantage of their multi-core processors. Classically, JavaScript has been executed on a single thread, but web workers offer a way to execute code in parallel. Doing so frees the browser of anything that may get in the way of the main thread so that it can smoothly animate the UI.

    A brief introduction to web workers

    There are several types of web workers:

    They each have specific properties, but share a similar design. The code running in a worker is executed in its own separate thread and runs in parallel with the main thread and other workers. The different types of workers all share a common interface.

    Web workers

    Dedicated web workers are instantiated by the main process and they can communicate only with it.

    Shared workers

    Shared workers can be reached by all processes running on the same origin (different browser tabs, iframes or other shared workers).

    Service workers

    Service workers have gained a lot of attention recently. They make it possible to proxy a web server programmatically to deliver specific content to the requester (e.g. the main thread). One use case for service workers is to serve content while offline. Service workers are a very new API, not fully implemented in all browsers, and are not covered in this article.

    In order to verify that web workers make Firefox OS run faster, we need to validate their speed by benchmarking them.

    The cost of creating web workers

    This article focuses on Firefox OS. All measurement are made on a Flame device, powered by middle-end hardware.

    The first set of benchmarks will look at the time it takes to create web workers. To do that, we set up a script that instantiates a web worker and sends a minimal message, to which the worker replies immediately. Once the response is received by the main thread, the time that the operation takes is calculated. The web worker is destroyed and the operation is repeated enough times to get a good idea of how long it takes on average to get a functional web worker. Instantiating a web worker is as easy as:

    // Start a worker.
    var worker = new Worker('worker-script.js');
     
    // Terminate a worker.
    worker.terminate();

    The same method is applied to the creation of broadcast channel:

    // Open a broadcast channel.
    var channel = new window.BroadcastChannel('channel-name');
     
    // Close a broadcast channel.
    channel.close();

    Shared workers can’t really be benchmarked here because once they are created, the developer can’t destroy them. The browser is entirely responsible for their lifetime. For that reason, we can’t create and destroy shared workers at will to get a meaningful benchmark.

    Web workers take about 40 ms to be instantiated. Also, this time is pretty stable with variations of only a few milliseconds. Setting up a broadcast channel is usually done within 1 ms.

    Under normal circumstances, the browser UI is refreshed at a rate of 60 frames per second. This means that no JavaScript code should run longer than the time needed by a frame, i.e., 16.66ms (60 frames per second). Otherwise, you may introduce jankiness and lag in your application.

    Instantiating web workers is pretty efficient, but still may not fit in the time allocated for a single frame. That’s why it’s important to create as few web workers as possible and reuse them.

    Message latency

    A critical aspect of web workers is having fast communication between your main thread and the workers. There are two different ways the main browser thread can communicate with a web worker.

    postMessage

    This API is the default and preferred way to send and receive messages from a web worker. postMessage() is easy to use:

    // Send a message to the worker.
    worker.postMessage(myMessage);
     
    // Listen to messages from the worker.
    worker.onmessage = evt => {
      var message = evt.data;
    };

    Broadcast Channel

    This is a newly implemented API, only available in Firefox at the time of this writing. It lets us broadcast messages to all contexts sharing the same origin. All browser tabs, iframes, or workers served from the same origin can emit and receive messages:

    // Send a message to the broadcast channel.
    channel.postMessage(myMessage);
     
    // Listen to messages from the broadcast channel.
    channel.onmessage = evt => {
      var message = evt.data;
    };

    To benchmark this, we use a script similar to the one described above, except that the web worker is not destroyed and reused at each operation. The time to get a round trip response should be divided by two.

    As you might expect, the simple postMessage is fast. It usually takes between 0 to 1 ms to send a message, whether to a web or shared worker. Broadcast channel API takes about 1 to 2 ms.

    Under normal circumstances, exchanging messages with workers is fast and you should not feel too concerned about speed here. However, larger messages can take longer.

    The size of messages

    There are 2 ways to send messages to web workers:

    • Copying the message
    • Transferring the message

    In the first case, the message is serialized, copied, and sent over. In the latter, the data is transferred. This means that the original sender can no longer use it once sent. Transferring data is almost instantaneous, so there is no real point in benchmarking that. However, only ArrayBuffer is transferable.

    As expected, serializing, copying, and de-serializing data adds significant overhead to the message transmission. The bigger the message, the longer it takes to be sent.

    The benchmark here sends a typed array to a web worker. Its size is progressively increased at each iteration. There is a linear correlation between size of the message and transfer time. For each measurement, we can divide the size (in kilobytes) by the time (in milliseconds) to get the transfer speed in kb/ms.

    Typically, on a Flame, the transfer speed is 80 kB/ms for postMessage and 12 kB/ms using broadcast channel. This means that if you want your message to fit in a single frame, you should keep it under 1,300 kB with postMessage and under 200 kB when using the broadcast channel. Otherwise, it may introduce frame drop in your application.

    In this benchmark, we use typed arrays, because it makes it possible to determine their size in kilobytes precisely. You can also transfer JavaScript objects, but due to the serialization process, they take longer to post. For small objects, this doesn’t really matter, but if you need to send huge objects, you may as well serialize them to a binary format. You can use something similar to Protocol Buffer.

    Web workers are fast if used correctly

    Here is a quick summary of various benchmarks related to web workers, as measured on a Flame:

    Operation Value
    Instantiation of a web worker 40 ms
    Instantiation of a broadcast channel 1 ms
    Communication latency with postMessage 0.5 ms
    Communication latency with broadcast channel 1.5 ms
    Communication speed with postMessage 80 kB/ms
    Communication speed with broadcast channel 12 kB/ms
    Maximum message size with postMessage 1,300 kB
    Maximum message size with broadcast channel 200 kB

    Benchmarking is the only way to make sure that the solution you are implementing is fast. This process takes much of the guesswork out of web development.

    If you want to run these benchmarks on a specific device, the app I built to make these measurements, web workers benchmark, is open source. You are also welcome to contribute by submitting new types of benchmarks.

  9. Streaming media on demand with Media Source Extensions

    Introducing MSE

    Media Source Extensions (MSE) is a new addition to the Web APIs available in all major browsers.  This API allows for things like adaptive bitrate streaming of video directly in our browser, free of plugins. Where previously we may have used proprietary solutions like RTSP (Real Time Streaming Protocol) and Flash, we can now use simpler protocols like HTTP to fetch content, and MSE to smoothly stitch together video segments of varied quality.

    All browsers that support HTMLMediaElements, such as audio and video tags, already make byte-range requests for subsequent segments of media assets.  One problem is that it’s up to each browser’s implementation of a media engine to decide when and how much to fetch.  It’s also tough to stitch together or deliver smooth playback of segments of different quality without pauses, gaps, flashes, clicks, or pops.  MSE gives us finer-grained control at the application level for fetching and playing back content.

    In order to begin streaming, we need to figure out how to transcode our assets into a meaningful byte format for browsers’ media engines, determine what abstractions MSE provides, and figure out how to instruct the browser to play them back.

    Having multiple resolutions of content allows us to switch between them while maintaining a constant viewport size.  This is known as upscaling, and it’s a common technique for real-time rendering in video games to meet a required frame time.  By switching to a lower quality video resolution, we can meet bandwidth limitations at the cost of fidelity.  The loss of fidelity causes such artifacts as aliasing, in which curves appear jagged and blocky.  This technique can often be seen by Netflix subscribers during peak viewing hours.

    Rather than having an advanced protocol like RTSP handle bandwidth estimates, we can use a simpler network protocol like HTTP and move the advanced logic up one level into the application logic.

    Transcoding

    My recommended tools, ffmpeg and Bento4, are both free and open-source software (FOSS). ffmpeg is our Swiss army knife of transcoding, and Bento4 is a collection of great tools for working with mp4.  While I’m partial to non-licensed codecs like webm/vp8-9/opus, current browser support for those containers and codecs is rather poor, so in this post we’ll just be working with mp4/h.264/aac.  Both of the tools I’m working with are command line utilities; if you have nice GUI tools in your asset pipeline you’d like to recommend to our readers, let us know in the comments below.

    We’ll start with a master of some file, and end up transcoding it into multiple files each of smaller resolutions, then segmenting the smaller-res whole files into a bunch of tiny files.  Once we have a bunch of small files (imagine splitting your video into a bunch of 10-second segments), the client can use more advanced heuristics for fetching the preferred next segment.

    MSE multiple resolutions

    Our smaller-res copies of the master asset

     Proper fragmentation

    When working with mp4 and MSE, it helps to know that the mp4 files should be structured so that metadata is fragmented across pieces of the container, and across the actual audio/video streams being fragmented, instead of clustered together.  This is specified in the ISO BMFF Byte Stream Format spec, section 3:

    “An ISO BMFF initialization segment is defined in this specification as a single File Type Box (ftyp) followed by a single Movie Header Box (moov).”

    This is really important: Simply transcoding to an mp4 container in ffmpeg does not have the expected format and thus fails when trying to play back in a browser with MSE.  To check and see if your mp4 is properly fragmented, you can run Bento4’s mp4dump on your mp4.

    If you see something like:

      $ ./mp4dump ~/Movies/devtools.mp4 | head
      [ftyp] size=8+24
        ...
      [free] size=8+0
      [mdat] size=8+85038690
      [moov] size=8+599967
        ...

    Then your mp4 won’t be playable since the [ftyp] “atom” is not followed immediately by a [moov] “atom.”  A properly fragmented mp4 looks something like this —

      $ ./mp4fragment ~/Movies/devtools.mp4 devtools_fragmented.mp4
      $ ./mp4dump devtools_fragmented.mp4 | head
      [ftyp] size=8+28
        ...
      [moov] size=8+1109
        ...
      [moof] size=8+600
        ...
      [mdat] size=8+138679
      [moof] size=8+536
        ...
      [mdat] size=8+24490
        ...
      ...

    — where mp4fragment is another Bento4 utility.  The properly fragmented mp4 has the [ftyp] followed immediately by a [moov], then subsequent [moof]/[mdat] pairs.

    It’s possible to skip the need for mp4fragment by using the -movflags frag_keyframe+empty_moov flags when transcoding to an mp4 container with ffmpeg, then checking with mp4dump:

      $ ffmpeg -i bunny.y4m -movflags frag_keyframe+empty_moov bunny.mp4
    Creating multiple resolutions

    If we want to switch resolutions, we can then run our fragmented mp4 through Bento4’s mp4-dash-encode.py script to get multiple resolutions of our video.  This script will fire up ffmpeg and other Bento4 tools, so make sure they are both available in your $PATH environment variable.

    $ python2.7 mp4-dash-encode.py -b 5 bunny.mp4
    $ ls
    video_00500.mp4 video_00875.mp4 video_01250.mp4 video_01625.mp4 video_02000.mp4
    Segmenting

    We now have 5 different copies of our video with various bit rates and resolutions. To be able to switch between them easily during playback, based on our effective bandwidth that changes constantly over time, we need to segment the copies and produce a manifest file to facilitate playback on the client.  We’ll create a Media Presentation Description (MPD)-style manifest file. This manifest file containing info about the segments, such as the threshold effective bandwidth for fetching the requisite segment.

    Bento4’s mp4-dash.py script can take multiple input files, perform the segmentation, and emit a MPD manifest that most DASH clients/libraries understand.

    $ python2.7 mp4-dash.py --exec-dir=. video_0*
    ...
    $ tree -L 1 output
    output
    ├── audio
    │   └── und
    ├── stream.mpd
    └── video
        ├── 1
        ├── 2
        ├── 3
        ├── 4
        └── 5
    
    8 directories, 1 file

    We should now have a folder with segmented audio and segmented video of various resolutions.

    MSE & Playback

    With an HTMLMediaElement such as an audio or video tag, we simply assign a URL to the element’s src attribute and the browser handles fetching and playback.  With MSE, we will fetch the content ourselves with XMLHttpRequests (XHRs) treating the response as an ArrayBuffer (raw bytes), and assigning the src attribute of the media element to a URL that points to a MediaSource object.  We may then append SourceBuffer objects to the MediaSource.

    Pseudocode for the MSE workflow might look like:

    let m = new MediaSource
    m.onsourceopen = () =>
      let s = m.addSourceBuffer('codec')
      s.onupdateend = () =>
        if (numChunks === totalChunks)
          m.endOfStream()
        else
          s.appendBuffer(nextChunk)
      s.appendBuffer(arrayBufferOfContent)
    video.src = URL.createObjectURL(m)
    

    Here’s a trick to get the size of a file: make an XHR with the HTTP HEAD method.  A response to a HEAD request will have the content-length header specifying the body size of the response, but unlike a GET, it does not actually have a body.  You can use this to preview the size of a file without actually requesting the file contents.  We can naively subdivide the video and fetch the next segment of video when we’re 80% of the way through playback of the current segment.  Here’s a demo of this in action and a look at the code.

    Note: You’ll need the latest Firefox Developer Edition browser to view the demo and test the code. More information below in the Compatibility section. The MSE primer from WebPlatform.org docs is another great resource to consult.

    My demo is a little naive and has a few issues:

    • It doesn’t show how to properly handle seeking during playback.
    • It assumes bandwidth is constant (always fetching the next segment at 80% playback of the previous segment), which it isn’t.
    • It starts off by loading only one segment (it might be better to fetch the first few, then wait to fetch the rest).
    • It doesn’t switch between segments of varying resolution, instead only fetching segments of one quality.
    • It doesn’t remove segments (part of the MSE API), although this can be helpful on memory constrained devices. Unfortunately, this requires you to re-fetch content when seeking backwards.

    These issues can all be solved with smarter logic on the client side with Dynamic Adaptive Streaming over HTTP (DASH).

    Compatibility

    Cross-browser codec support is a messy story right now; we can use MediaSource.isTypeSupported to detect codec support.  You pass isTypeSupported a string of the MIME type of the container you’re looking to play.  mp4 has the best compatibility currently. Apparently, for browsers that use the Blink rendering engine, MediaSource.isTypeSupported requires the full codec string to be specified.  To find this string, you can use Bento4’s mp4info utility:

    ./mp4info bunny.mp4| grep Codec
        Codecs String: avc1.42E01E

    Then in our JavaScript:

    if (MediaSource.isTypeSupported('video/mp4; codecs="avc1.42E01E, mp4a.40.2"')) {
    // we can play this
    }

    — where mp4a.40.2 is the codec string for low complexity AAC, the typical audio codec used in an mp4 container.

    Some browsers also currently whitelist certain domains for testing MSE, or over-aggressively cache CORS content, which makes testing frustratingly difficult.  Consult your browser for how to disable the whitelist or CORS caching when testing.

    DASH

    Using the MPD file we created earlier, we can grab a high quality DASH client implemented in JavaScript such as Shaka Player or dash.js.  Both clients implement numerous features, but could use more testing, as there are some subtle differences between media engines of various browsers.  Advanced clients like Shaka Player use an exponential moving average of three to ten samples to estimate the bandwidth, or even let you specify your own bandwidth estimator.

    If we serve our output directory created earlier with Cross Origin Resource Sharing (CORS) enabled, and point either DASH client to http://localhost:<port>/output/stream.mpd, we should be able to see our content playing.  Enabling video cycling in Shaka, or clicking the +/- buttons in dash.js should allow us to watch the content quality changing.  For more drastic/noticeable changes in quality, try encoding fewer bitrates than the five we demonstrated.

    Shaka Player in Firefox Dev Edition

    Shaka Player in Firefox Developer Edition

    dash.js running in Firefox Developer Edition

    dash.js in Firefox Developer Edition

    In conclusion

    In this post, we looked at how to prep video assets for on-demand streaming by pre-processing and transcoding.  We also took a peek at the MSE API, and how to use more advanced DASH clients.  In an upcoming post, we’ll explore live content streaming using the MSE API, so keep an eye out.  I recommend you use Firefox Developer Edition to test out MSE; lots of hard work is going into our implementation.

    Here are some additional resources for exploring MSE:

  10. Trainspotting: Firefox 39

    Trainspotting is a series of articles highlighting features in the lastest version of Firefox. A new version of Firefox is shipped every six weeks – we at Mozilla call this pattern “release trains.”

    A new version of Firefox is here, and with it come some great improvements and additions to the Web platform and developer tools. This post will call out a few highlights.

    For a full list of changes and additions, take a look at the Firefox 39 release notes.

    DevTools Love

    The Firefox Developer Tools are constantly getting better. We’re listening to developers on UserVoice, and using their feedback to make tools that are more powerful and easier to use. One requested feature was the ability to re-order elements in the Inspector:

    Editing and tweaking CSS Animations is easier than ever – Firefox 39 lets developers pause, restart, slow down, and preview new timings without having to switch applications.

    Menu of animation easing presets in the Inspector

    CSS Scroll Snap Points

    CSS Scroll Snap Points in action

    CSS Scroll Snap Points let web developers instruct the browser to smoothly snap element scrolling to specific points along an axis, creating smoother, easier to interact with interfaces with fewer lines of code.

    Improvements to Firefox on Mac OS X

    Firefox gets some Mac- specific improvements and updates in version 39:

    • Project Silk enabled – Improves scrolling and animation performance by more closely timing painting with hardware vsync. Read more about Project Silk.
    • Unicode 8.0 skin tone emoji – Fixed a bug in the rendering of skin tone modifiers for emoji.
    • Dashed line performance – Rendering of dotted and dashed lines is vastly improved. Check out the fixed bug for more information.

    Service Workers Progress

    Firefox’s implementation of the Service Workers API continues – fetch is enabled for workers and is now generally available to web content, and the Cache and CacheStorage are now available behind a flag.

    There’s lots more changes and improvements in Firefox 39 – check out the Developer Release Notes for developer-oriented changes or the full list of bugs fixed in this release. Enjoy!