Mozilla

Featured Articles

Sort by:

View:

  1. Embedding an HTTP Web Server in Firefox OS

    Nearing the end of last year, Mozilla employees were gathered together for a week of collaboration and planning. During that week, a group was formed to envision what the future of Firefox OS might be surrounding a more P2P-focused Web. In particular, we’ve been looking at harnessing technologies to collectively enable offline P2P connections such as Bluetooth, NFC and WiFi Direct.

    Since these technologies only provide a means to communicate between devices, it became immediately clear that we would also need a protocol for apps to send and receive data. I quickly realized that we already have a standard protocol for transmitting data in web apps that we could leverage – HTTP.

    By utilizing HTTP, we would already have everything we’d need for apps to send and receive data on the client side, but we would still need a web server running in the browser to enable offline P2P communications. While this type of HTTP server functionality might be best suited as part of a standardized WebAPI to be baked into Gecko, we actually already have everything we need in Firefox OS to implement this in JavaScript today!

    navigator.mozTCPSocket

    Packaged apps have access to both raw TCP and UDP network sockets, but since we’re dealing with HTTP, we only need to work with TCP sockets. Access to the TCPSocket API is exposed through navigator.mozTCPSocket which is currently only exposed to “privileged” packaged apps with the tcp-socket permission:

    "type": "privileged",
    "permissions": {
      "tcp-socket": {}
    },

    In order to respond to incoming HTTP requests, we need to create a new TCPSocket that listens on a known port such as 8080:

    var socket = navigator.mozTCPSocket.listen(8080);

    When an incoming HTTP request is received, the TCPSocket needs to handle the request through the onconnect handler. The onconnect handler will receive a TCPSocket object used to service the request. The TCPSocket you receive will then call its own ondata handler each time additional HTTP request data is received:

    socket.onconnect = function(connection) {
      connection.ondata = function(evt) {
        console.log(evt.data);
      };
    };

    Typically, an HTTP request will result in a single calling of the ondata handler. However, in cases where the HTTP request payload is very large, such as for file uploads, the ondata handler will be triggered each time the buffer is filled, until the entire request payload is delivered.

    In order to respond to the HTTP request, we must send data to the TCPSocket we received from the onconnect handler:

    connection.ondata = function(evt) {
      var response = 'HTTP/1.1 200 OK\r\n';
      var body = 'Hello World!';
     
      response += 'Content-Length: ' + body.length + '\r\n';
      response += '\r\n';
      response += body;
     
      connection.send(response);
      connection.close();
    };

    The above example sends a proper HTTP response with “Hello World!” in the body. Valid HTTP responses must contain a status line which consists of the HTTP version HTTP/1.1, the response code 200 and the response reason OK terminated by a CR+LF \r\n character sequence. Immediately following the status line are the HTTP headers, one per line, separated by a CR+LF character sequence. After the headers, an additional CR+LF character sequence is required to separate the headers from the body of the HTTP response.

    FxOS Web Server

    Now, it is likely that we will want to go beyond simple static “Hello World!” responses to do things like parsing the URL path and extracting parameters from the HTTP request in order to respond with dynamic content. It just so happens that I’ve already implemented a basic-featured HTTP server library that you can include in your own Firefox OS apps!

    FxOS Web Server can parse all parts of the HTTP request for various content types including application/x-www-form-urlencoded and multipart/form-data. It can also gracefully handle large HTTP requests for file uploads and can send large binary responses for serving up content such as images and videos. You can either download the source code for FxOS Web Server on GitHub to include in your projects manually or you can utilize Bower to fetch the latest version:

    bower install justindarc/fxos-web-server --save

    Once you have the source code downloaded, you’ll need to include dist/fxos-web-server.js in your app using a <script> tag or a module loader like RequireJS.

    Simple File Storage App

    Next, I’m going to show you how to use FxOS Web Server to build a simple Firefox OS app that lets you use your mobile device like a portable flash drive for storing and retrieving files. You can see the source code for the finished product on GitHub.

    Before we get into the code, let’s set up our app manifest to get permission to access DeviceStorage and TCPSocket:

    {
      "version": "1.0.0",
      "name": "WebDrive",
      "description": "A Firefox OS app for storing files from a web browser",
      "launch_path": "/index.html",
      "icons": {
        "128": "/icons/icon_128.png"
      },
      "type": "privileged",
      "permissions": {
        "device-storage:sdcard": { "access": "readwrite" },
        "tcp-socket": {}
      }
    }

    Our app won’t need much UI, just a listing of files in the “WebDrive” folder on the device, so our HTML will be pretty simple:

    <!DOCTYPE html>
    <html>
    <head>
      <meta charset="utf-8">
      <title>WebDrive</title>
      <meta name="description" content="A Firefox OS app for storing files from a web browser">
      <meta name="viewport" content="width=device-width, user-scalable=no, initial-scale=1">
      <script src="bower_components/fxos-web-server/dist/fxos-web-server.js"></script>
      <script src="js/storage.js"></script>
      <script src="js/app.js"></script>
    </head>
    <body>
      <h1>WebDrive</h1>
      <hr>
      <h3>Files</h3>
      <ul id="list"></ul>
    </body>
    </html>

    As you can see, I’ve included fxos-web-server.js in addition to app.js. I’ve also included a DeviceStorage helper module called storage.js since enumerating files can get somewhat complex. This will help keep the focus on our code specific to the task at hand.

    The first thing we’ll need to do is create new instances of the HTTPServer and Storage objects:

    var httpServer = new HTTPServer(8080);
    var storage = new Storage('sdcard');

    This will initialize a new HTTPServer on port 8080 and a new instance of our Storage helper pointing to the device’s SD card. In order for our HTTPServer instance to be useful, we must listen for and handle the “request” event. When an incoming HTTP request is received, the HTTPServer will emit a “request” event that passes the parsed HTTP request as an HTTPRequest object to the event handler.

    The HTTPRequest object contains various properties of an HTTP request including the HTTP method, path, headers, query parameters and form data. In addition to the request data, an HTTPResponse object is also passed to the “request” event handler. The HTTPResponse object allows us to send our response as a file or string and set the response headers:

    httpServer.addEventListener('request', function(evt) {
      var request  = evt.request;
      var response = evt.response;
     
      // Handle request here...
    });

    When a user requests the root URL of our web server, we’ll want to present them with a listing of files stored in the “WebDrive” folder on the device along with a file input for uploading new files. For convenience, we’ll create two helper functions to generate the HTML string to send in our HTTP response. One will just generate the listing of files which we’ll reuse to display the files on the device locally and the other will generate the entire HTML document to send in the HTTP response:

    function generateListing(callback) {
      storage.list('WebDrive', function(directory) {
        if (!directory || Object.keys(directory).length === 0) {
          callback('<li>No files found</li>');
          return;
        }
     
        var html = '';
        for (var file in directory) {
          html += `<li><a href="/${encodeURIComponent(file)}" target="_blank">${file}</a></li>`;
        }
     
        callback(html);
      });
    }
     
    function generateHTML(callback) {
      generateListing(function(listing) {
        var html =
    `<!DOCTYPE html>
    <html>
    <head>
      <meta charset="utf-8">
      <title>WebDrive</title>
    </head>
    <body>
      <h1>WebDrive</h1>
      <form method="POST" enctype="multipart/form-data">
        <input type="file" name="file">
        <button type="submit">Upload</button>
      </form>
      <hr>
      <h3>Files</h3>
      <ul>${listing}</ul>
    </body>
    </html>`;
     
        callback(html);
      });
    }

    You’ll notice that we’re using ES6 Template Strings for generating our HTML. If you’re not familiar with Template Strings, they allow us to have multi-line strings that automatically include whitespace and new lines and we can do basic string interpolation that automatically inserts values inside the ${} syntax. This is especially useful for generating HTML because it allows us to span multiple lines so our template markup remains highly readable when embedded within JavaScript code.

    Now that we have our helper functions, let’s send our HTML response in our “request” event handler:

    httpServer.addEventListener('request', function(evt) {
      var request  = evt.request;
      var response = evt.response;
     
      generateHTML(function(html) {
        response.send(html);
      });
    });

    As of right now, our “request” event handler will always respond with an HTML page listing all the files in the device’s “WebDrive” folder. However, we must first start the HTTPServer before we can receive any requests. We’ll do this once the DOM is ready and while we’re at it, let’s also render the file listing locally:

    window.addEventListener('DOMContentLoaded', function(evt) {
      generateListing(function(listing) {
        list.innerHTML = listing;
      });
     
      httpServer.start();
    });

    We should also be sure to stop the HTTPServer when the app is terminated, otherwise the network socket may never be freed:

    window.addEventListener('beforeunload', function(evt) {
      httpServer.stop();
    });

    At this point, our web server should be up and running! Go ahead and install the app on your device or simulator using WebIDE. Once installed, launch the app and point your desktop browser to your device’s IP address at port 8080 (e.g.: http://10.0.1.12:8080).

    You should see our index page load in your desktop browser, but the upload form still isn’t wired up and if you have any files in your “WebDrive” folder on your device, they cannot be downloaded yet. Let’s first wire up the file upload by first creating another helper function to save files received in an HTTPRequest:

    function saveFile(file, callback) {
      var arrayBuffer = BinaryUtils.stringToArrayBuffer(file.value);
      var blob = new Blob([arrayBuffer]);
     
      storage.add(blob, 'WebDrive/' + file.metadata.filename, callback);
    }

    This function will first convert the file’s contents to an ArrayBuffer using the BinaryUtils utility that comes with fxos-web-server.js. We then create a Blob that we pass to our Storage helper to save it to the SD card in the “WebDrive” folder. Note that the filename can be extracted from the file’s metadata object since it gets passed to the server using ‘multipart/form-data’ encoding.

    Now that we have a helper for saving an uploaded file, let’s wire it up in our “request” event handler:

    httpServer.addEventListener('request', function(evt) {
      var request  = evt.request;
      var response = evt.response;
     
      if (request.method === 'POST' && request.body.file) {
        saveFile(request.body.file, function() {
          generateHTML(function(html) {
            response.send(html);
          });
     
          generateListing(function(html) {
            list.innerHTML = html;
          });
        });
     
        return;
      }
     
      generateHTML(function(html) {
        response.send(html);
      });
    });

    Now, anytime an HTTP POST request is received that contains a “file” parameter in the request body, we will save the file to the “WebDrive” folder on the SD card and respond with an updated file listing index page. At the same time, we’ll also update the file listing on the local device to display the newly-added file.

    The only remaining part of our app to wire up is the ability to download files. Once again, let’s update the “request” event handler to do this:

    httpServer.addEventListener('request', function(evt) {
      var request  = evt.request;
      var response = evt.response;
     
      if (request.method === 'POST' && request.body.file) {
        saveFile(request.body.file, function() {
          generateHTML(function(html) {
            response.send(html);
          });
     
          generateListing(function(html) {
            list.innerHTML = html;
          });
        });
     
        return;
      }
     
      var path = decodeURIComponent(request.path);
      if (path !== '/') {
        storage.get('WebDrive' + path, function(file) {
          if (!file) {
            response.send(null, 404);
            return;
          }
     
          response.headers['Content-Type'] = file.type;
          response.sendFile(file);
        });
     
        return;
      }
     
      generateHTML(function(html) {
        response.send(html);
      });
    });

    This time, our “request” event handler will check the requested path to see if a URL other than the root is being requested. If so, we assume that the user is requesting to download a file which we then proceed to get using our Storage helper. If the file cannot be found, we return an HTTP 404 error. Otherwise, we set the “Content-Type” in the response header to the file’s MIME type and send the file with the HTTPResponse object.

    You can now re-install the app to your device or simulator using WebIDE and once again point your desktop browser to your device’s IP address at port 8080. Now, you should be able to upload and download files from your device using your desktop browser!

    The possible use cases enabled by embedding a web server into Firefox OS apps are nearly limitless. Not only can you serve up web content from your device to a desktop browser, as we just did here, but you can also serve up content from one device to another. That also means that you can use HTTP to send and receive data between apps on the same device! Since its inception, FxOS Web Server has been used as a foundation for several exciting experiments at Mozilla:

    • wifi-columns

      Guillaume Marty has combined FxOS Web Server with his amazing jsSMS Master System/Game Gear emulator to enable multi-player gaming across two devices in conjunction with WiFi Direct.

    • sharing

      Several members of the Gaia team have used FxOS Web Server and dns-sd.js to create an app that allows users to discover and share apps with friends over WiFi.

    • firedrop

      I have personally used FxOS Web Server to build an app that lets you share files with nearby users without an Internet connection using WiFi Direct. You can see the app in action here:

    I look forward to seeing all the exciting things that are built next with FxOS Web Server!

  2. Introducing node-firefox

    NOTE: we presented this project last Sunday at FOSDEM, but not everyone could make it to Brussels, so here’s a post explaining what node-firefox is and how can it help you superturbocharge your Firefox OS app development!

    At Mozilla we’re always looking for ways in which we can make developers’ lives easier. When aspiring app developers told us that it was cumbersome to get started writing Open Web Apps, we worked on turning App Manager into a more beginner friendly environment, which in turn gave way to WebIDE. This tool simplifies many actions that were slow and tedious before, such as creating a new app, downloading and installing simulators or running and debugging apps.

    But there was still a segment of developers that felt left out: power users! They already have their node.js-based build toolchains, with tasks such as asset optimisation, code hinting, or test running. They often also use tools such as Browserify, and perhaps they don’t even write JavaScript, favouring alternatives such as CoffeeScript instead, but all these goodies require you to build the app or website before you push it again to your device or reload the browser.

    Essentially, we were telling these developers to leave their beloved command line (or editor shortcuts!) to go to WebIDE and click a button to deploy the app, and then go back to their editor of choice. And they most unanimously answered: “But we don’t like to click! We like the terminal!”

    How can we make this more efficient?

    People didn’t like this because it implied changing contexts. It is inefficient, we are engineers, and if there one thing that engineers like more than building new things it is probably optimising and streamlining processes.

    Since we already have a build script, the only step that is left in order to get our apps onto the runtime is deploying, and that’s what we are using WebIDE for. So the obvious question would be: can we do whatever WebIDE is doing to deploy, but programmatically?

    Servers and actors

    Every Firefox runtime has something called the remote debugger server. This is not enabled by default, for obvious security reasons, but when enabled, clients can connect to it and take advantage of its various functionalities, such as installing apps, accessing the console, etc. And this is what WebIDE does internally.

    Each of these functionalities is provided by an actor. For example, suppose we want to list the installed apps. We would…

    • first find the webApps actor
    • then run the getAll command
    • and get a list of apps in response

    Another example would be installing a packaged app. The steps would be:

    • first zip the app contents, using whatever library or way you like
    • then get the webApps actor
    • call the uploadPackage command in the webApps actor with the contents of the ZIP file
    • the result of this call is a File actor
    • call the install command in the webApps actor with the returned File actor
    • done!

    Therefore all the magic for installing apps is not in WebIDE—it is in the servers! We can take advantage of this magic programmatically, but building a client from scratch involves establishing TCP connections and parsing packets, which is not what you want to be doing: you want to write apps and push them to devices instead!

    Despair not, as node-firefox will abstract that for you. It is not a monolithic piece of code, but a series of node.js modules, each one of them performing a different task, hosted on its own separate repository and published to the npm registry like good module citizens. You can use as many of them as you need in your scripts or task runners, and thus you can finally build and run your app without ever leaving the command line.

    Show, don’t tell

    But enough of talking and describing; let’s see how to write a script that starts a simulator!

    First install the module in your project, using npm:

    npm install --save node-firefox-start-simulator

    And this would be the script:

    var startSimulator = require('node-firefox-start-simulator');
     
    startSimulator({ version: '2.2' })
      .then(function(simulator) {
        console.log('Listening in port', simulator.port);
      });

    That’s it! With just a few lines of code you are able to programmatically start a version 2.2 simulator. If you don’t care about the version, just don’t pass in any option to startSimulator, and it will start the first simulator it finds:

    startSimulator().then(function(simulator) {
      // your code
    });

    We can also see this in action. Here’s us starting a simulator, installing an app and launching it, all from a node.js script:

    Starting simulator, running app from node.js

    The code for this example is actually the example for the node-firefox-uninstall-app module. Each of the node-firefox modules come with an examples folder so you can get started rather quickly.

    As we mentioned at the beginning, many web developers that move to app development want to keep using their task runners, so we also wrote an example of how to use node-firefox with gulp.

    Let’s run the default-one task. This starts a simulator, deploys an app, and for a bit more of a challenge, also keeps watching for CSS changes. If you edit and save any of the app’s stylesheets, the file watcher will detect the change, and send the new file contents to the runtime, which will replace them on the fly, without having to stop, push and relaunch the whole app. Look at me changing the background colour from austere dark blue to the timeless Paul Rouget pink!

    Starting simulator, launching app with gulp

    Live CSS reloading is really great to build and experiment with UI interfaces. Not having to reload the app and then navigate to the particular layout you want to work in saves lots of time—I wish I’d had that when I was programming Android apps!

    But we can outdo this. The default-all task will do the same as default-one, but for all the simulators installed in your system, so you can see the effect of your CSS changes in all the simulators at the same time:

    Starting all simulators, launching app and live CSS reload with gulp.

    Unfortunately there is a bug in the 2.1 and 2.2 simulators, and those don’t reload the stylesheet changes, but it’s been filed and will be fixed.

    What can we do so far?

    The current set of modules lets you find ports where runtimes are listening, find and start simulators; connect to runtimes; find, install, uninstall and launch apps, and reload stylesheets.

    Philosophy

    You might have noticed a pattern already, but just in case it wasn’t evident enough, we are trying to write deliberately simple modules. Each module should perform only one action, return a Promise and use as few dependencies as possible.

    Small modules are easier to understand, use, and test. Also, most of the future Web APIs are designed to work with Promises, and we want to write code for the future, not for the past. In addition, reducing the number of dependencies also makes it easier for new people to get started on contributing to a module, as there are fewer new unfamiliar elements to understand.

    Finally, since all the modules work the same way, when you know how to use one module you know how to use the rest—the only thing that changes is the parameters, and the result.

    Dream ideas (or: what we cannot do yet)

    There’s a number of things that we’d like to see happen in the future. Some people call them features, but I call them ‘dream ideas.’

    A recurrent one is the WebCLI: a counterpart equivalent to WebIDE, where everything you can do with WebIDE could be done with a command line tool. I keep switching back and forth between “this is a good idea” and “perhaps we don’t need this at all and a library of tasks will be enough”, but people seemed to like the idea, so maybe it’s not that bad!

    Another great feature would be the ability to attach the DevTools debugger to an app that was launched from the command line but that just crashed. Launching apps from the command line is great, but command line debuggers are not that exciting! Why not use the best of both worlds?

    Or maybe it would be neat to control any browser from the command line, interfacing via Valence!

    And finally, there is my favourite dream idea: Firefox OS custom editions. Imagine if we could just write a script that would create an empty Firefox OS slate, pull in our favourite apps and settings, and generate a whole Firefox OS image that we could then flash to devices. And since it is not a binary blob but a script, we could just publish it on its repository, and other people could remix and build their own Firefox OS based editions.

    How do we get there?

    There’s still a long way ahead of us, and lots of areas that need work. Perhaps the most urgent task is to get better multiplatform support. Currently we can only interact with runtimes through the network, but no physical devices. Also, support on platforms other than Mac OS is largely lacking.

    Testing is another important aspect. If we test early, often and profusely we will be able to detect problems such as the CSS bug I stumbled upon when building the gulp demo. We want to have these modules running on several platforms and connecting to other different platforms, including physical devices.

    Of course we need more modules, and more examples! To make sure no two people start writing the same module, we are discussing and proposing new modules in the top project issue tracker. And we’d love to see more examples, or even just better examples that hook existing functionality in other node modules with our code. For example, one could add manifest validation via the firefox-app-validator-manifest module.

    And, as always, we need you. We are not you, so we cannot know what you need or what thoughts cross your mind. And we certainly cannot use software the same way you use it either. We need your input and your contributions!

    We’re looking forward to seeing what you create with node-firefox. File issues, or talk to us on irc if you have questions. We hang out mostly in the #apps and #devtools channels in irc.mozilla.org.

    Thanks

    It would be dishonest not to thank Nicola Greco, whom I mentored last summer when he was interning at Mozilla. He came up with the initial idea of building individual node modules that would help you develop Firefox OS apps. Go check out his final intern presentation, as it’s really entertaining and illustrative!

    Many thanks to all the (infinitely patient) DevToolers Ryan Stinnet, Alexandre Poirot, Jeff Griffiths and Dave Camp, who helped us find our way around remote servers and actors and whatnot, and huge thanks to Heather Arthur who wrote firefox-client and made writing node-firefox way more pleasant than it would have been otherwise.

  3. Project Silk

    Editor’s Note: An earlier version of this post appeared on Mason Chang’s personal blog.

    For the past few months, I’ve been working on Project Silk which improves smoothness across the browser. Very much like Project Butter for Android, part of it is finally live on Firefox OS. Silk does three things:

    1. Align Painting with hardware vsync
    2. Resample touch input events based on hardware vsync
    3. Align composites with hardware vsync

    What is vsync, why vsync, and why does it matter at all?

    Vertical synchronization (vsync) occurs when the hardware display shows a new frame on the screen. This rate is set by specific hardware, but major displays in the US occur at a rate of 60 times a second, or every 16.6 ms (milliseconds). This is where you hear about 60 frames per second, one frame each time the hardware display refreshes. What this means in reality is that no matter how many frames are produced in software, the hardware display will still only show at most 60 unique frames per second.

    Currently in Firefox, we mimic 60 frames per second and therefore vsync with a software timer that schedules rendering every 16.6 ms. However, the software scheduler has two problems: (a) it’s noisy and (b) it can be scheduled at bad times relative to vsync.

    In regards to noise, software timers are much noisier than hardware timers. This creates micro-jank for a number of reasons. First, many animations are keyed off timestamps that are generated by the software scheduler to update the position of the animation. If you’ve ever used requestAnimationFrame, you get a timestamp from a software timer. If you want smooth animations, the timestamp provided to requestAnimationFrame should be uniform. Non-uniform timestamps will create non-uniform and janky animations. Here is a graph showing software versus hardware vsync timer uniformity:

    timer

    Wow! Big improvement with a hardware timer. We get a much more uniform, and therefore smoother, timestamp to key animations off of. So that addresses problem (a), noisy timers in software versus hardware.

    With part (b), software timers can be scheduled at bad times relative to vsync. Regardless of what the software does, the hardware display will refresh on its own clock. If our rendering pipeline finishes producing a frame before the next vsync, the display is updated with new content. If we fail to finish producing a frame before the next vsync, the previous frame will be displayed, causing jankiness. Some rendering functions can occur close to vsync and overflow until the next interval. Thus, we actually introduce more potential latency since the frame won’t be displayed on the screen anyway until the next vsync. Let’s look at this in graphic form:

    frames

    At time 0, we start producing frames. For example, let’s say all frames take a constant time of 10 ms. Our frame budget is 16.6 ms because we only have to finish producing a frame before the next hardware vsync occurs. Since frame 1 is finished 6 ms before the next vsync (time t=16 ms), everything is successful and life is good. The frame is produced in time and the hardware display will be refreshed with the updated content.

    Now let’s look at Frame 2. Since software timers are noisy, we start producing a frame 9 ms from the next vsync (time t=32). Since our frame takes 10 ms to produce, we actually finish producing this frame at 1 ms AFTER the next vsync. That means at vsync number 2 (t=32), there is no new frame to display, so the display still shows the previous frame. In addition, the frame just produced won’t be shown until vsync 3 (t=48), because that’s when the hardware updates itself. This creates jank since now the display will have skipped one frame and will try to catch up in the upcoming frames. This also produces one extra frame of latency, which is terrible for games.

    Vsync addresses both of these problems since we get a much more uniform timer and the maximum amount of frame budget time to produce a new frame. Now that we know what vsync is, we can finally go on to what Project Silk is and how it helps create smooth experiences in Firefox.

    The Rendering Pipeline

    In super simplified terms, Gecko’s rendering pipeline does three things:

    1. Paint / draw the new frame on the main thread.
    2. Send the updated content to the Compositor via a LayerTransaction.
    3. Composite the new content.

    In an ideal world, we’d be able to do all three steps within 16.6 ms, but that’s not the case most of the time. Both steps (1) and (3) occur on independent software timers. Thus, there is no real synchronizing clock between the three steps, they are all ad hoc. They also have no relation to vsync, so the timing of the pipeline isn’t related to when the display actually updates the screen with content. With Silk, we replace both independent software timers with the hardware vsync timer. For our purposes, (2) doesn’t really affect the outcome, but is presented here for completeness.

    Align Painting with Hardware Vsync

    Aligning the timer used to tick the refresh driver with vsync creates smoothness in a couple of ways. First, many animations are still done on the main thread, which means any animation using timestamps to set the position of an animation should be smoother. This includes requestAnimationFrame animations! The other nice thing is that we now have a very strict ordering of when rendering is kicked off. Instead of (1) and (3), which occur at separate synched offsets, we start rendering at a specific time.

    Resample Touch Input Events Based on Vsync

    With Silk, we can enable touch resampling, which improves smoothness while tracking your finger. Since I’ve already blogged about touch resampling quite a bit, I’ll keep this short. With Silk, we can finally enable it!

    Align Composites with Hardware Vsync

    Finally, the last part of Silk is about aligning composites with hardware vsync. Compositing takes all the painted content and merges it together to create the single image you see on the display. With Silk, all composites start right after a hardware vsync occurs. This has actually produced a rather nice side benefit — the reduced composite times seen here:

    compositeTimes

    Within the device driver on a Flame device, there’s a global lock that’s grabbed when close to vsync intervals. This lock can take 5-6 ms to get, greatly increasing the composite times. However, when we start a composite right after a vsync, there is little contention to grab the lock. Thus we can shave off the wait, therefore reducing composite times quite a bit. Not only do we get smoother animations, but also reduced composite times, and therefore better battery life. What a nice win!

    With all three pieces, we now have a nice strict ordering of the rendering pipeline. We paint and send the updated content to the Compositor within 16.6 ms. At the next vsync, we composite the updated content. At the vsync after that, the frame should have gone through the rendering pipeline and will be displayed on the screen. Keeping this order reduces jank because we reduce the chance that the timers will schedule each step at a bad time. In a best-case scenario in the current implementation without Silk, a frame could be painted and composited within a single 16.6 ms frame. This is great. However, if the next frame takes 2 frames instead, we’ve just created extra jank, even though no stage in the pipeline was really slow. Aligning the whole pipeline to create a strict sequence of events reduces the chance that we mis-schedule a frame.

    master

    Here’s a picture of the rendering pipeline without Silk. We have Composites (3) at the bottom of this profile. We have painting (1) in the middle, where you see Styles, Reflow, Displaylist, and Rasterize. We have Vsync, represented by those small orange boxes at the top. Finally we have Layer Transactions (2) at the bottom. At first, when we start, compositing and painting are not aligned, so animations are at different positions depending on whether they are on the main thread or the compositor thread. Second, we see long composites because the compositor is waiting on a global lock in the device driver. Lastly, it’s difficult to read any ordering or see if there is a problem without deep knowledge of why / when things should be happening.

    silk

    Here is a picture of the same pipeline with Silk. Composites are a little shorter, and the whole pipeline only starts at vsync intervals. Composite times are reduced because we start composites exactly at vsync intervals. There is a clear ordering of when things should happen. Both composites and painting are keyed off the same timestamp, ensuring smoother animations. Finally, there is a clear indicator that as long as everything finishes before the next Vsync, things will be smooth.

    Ultimately, Silk aims to create a smoother experience across Firefox and the Web. Numerous people contributed to the project. Thanks to Jerry Shih, Boris Chou, Jeff Hwang, Mike Lee, Kartikaya Gupta, Benoit Girard, Michael Wu, Ben Turner, and Milan Sreckovic for their help in making Silk happen.

  4. Canvas 2D: New docs, Path2D objects, hit regions

    Over the last year, a couple of new HTML Canvas 2D features were implemented in Firefox and other recent browsers, with the help of the Adobe Web platform team. Over on MDN, the documentation for Canvas 2D got a major update to reflect the current canvas standard and browser implementation status. Let’s have a look what is new and how you can use it to enhance your canvas graphics and gaming content.

    Path2D objects

    The new Path2D API (available from Firefox 31+) lets you store paths, which simplifies your canvas drawing code and makes it run faster. The constructor provides three ways to create a Path2D object:

    new Path2D();     // empty path object
    new Path2D(path); // copy from another path
    new Path2D(d);    // path from from SVG path data

    The third version, which takes SVG path data to construct, is especially handy. You can now re-use your SVG paths to draw the same shapes directly on a canvas as well:

    var p = new Path2D("M10 10 h 80 v 80 h -80 Z");

    When constructing an empty path object, you can use the usual path methods, which might be familiar to you from using them directly on the CanvasRenderingContext2D context.

    // create a circle
    var circle = new Path2D();
    circle.arc(50, 50, 50, 0, 2 * Math.PI);
    // stroke the circle onto the context ctx
    ctx.stroke(circle);

    To actually draw the path onto the canvas, several APIs of the context have been updated to take an optional Path2D path:

    Hit regions

    Starting with Firefox 32, experimental support for hit regions has been added. You need to switch the canvas.hitregions.enabled preference to true in order to test them. Hit regions provide a much easier way to detect if the mouse is in a particular area without relying on manually checking coordinates, which can be really difficult to check for complex shapes. The Hit Region API is pretty simple:

    CanvasRenderingContext2D.addHitRegion()
    Adds a hit region to the canvas.
    CanvasRenderingContext2D.removeHitRegion()
    Removes the hit region with the specified id from the canvas.
    CanvasRenderingContext2D.clearHitRegions()
    Removes all hit regions from the canvas.

    The addHitRegion method, well, adds a hit region to a current path or a Path2D path. The MouseEvent interface got extended with a region property, which you can use to check whether the mouse hit the region or not.

    Check out the code example on MDN to see it in action (and be sure to enable flags/preferences in at least Firefox and Chrome).

    Focus rings

    Also in Firefox 32, the drawFocusIfNeeded(element) property has been made available without a preference switch. This API allows you to draw focus rings on a canvas, if a provided fallback element inside the <canvas></canvas> element gains focus. If the fallback element gets focused (for example when tabbing through the page that contains the canvas), the pixel representation / shape of that element on the canvas can draw a focus ring to indicate the current focus.

    CSS/SVG filters exposed to Canvas

    Although still behind a preference (canvas.filters.enabled), and not yet in the latest canvas specification (but expected to be added), Firefox 35 gained support for filters on the canvas rendering context. The syntax is the same as the CSS filter property.

    Misc

    Documentation

    If you would like to read more about Canvas 2D graphics, check out the Canvas tutorial on MDN, which guides you through the Canvas APIs. A good bookmark is also the large CanvasRenderingContext2D interface, which you will use often when working with Canvas.

  5. Web Animation tools, Network Security insights, Font Inspector improvements and more – Firefox Developer Tools Episode 37

    A handful of bug fixes, improvements and some new features, of course, just landed in Firefox 37. Update your Firefox Developer Edition, or Nightly builds to try them out!

    Animation Inspector Panel

    A new API that’s quickly gaining traction is the Web Animations API, allowing developers to construct more complex animations using web technologies, rather than proprietary plugins. The foundation for animation tooling has begun in Firefox’s Developer Tools, with the first release revealing play/pause controls and a timeline scrubber. When selecting an element in the Inspector, an animations panel is now available alongside the rules, fonts, and other panels, if the element contains animations. Check out other videos of the animation inspector in action, or try it yourself in this web animation demo.

    This is the first iteration of the web animation tools, and we are looking to hear from you on where we should go with this, and what we should build! Share detailed feedback with us on UserVoice, leave us a comment right here, or tweet to @FirefoxDevTools. (More detail on how to reach us in the closing paragraph below.)

    Security Panel

     
    The Network Monitor is the home of our other new tool, the security panel. Selecting a request in the network panel now displays a security panel in the request inspector. The panel reveals a list of information about the request’s connection, host, as well as the certificate used.

    The security panel can help debug issues related to SSL protocol versions, such as sites not working because of the POODLEBITE issue, and can help ensure that sufficiently strong security measures are implemented.

    DevTools Security Panel, new in Firefox 37

    Some other features and small improvements in this release:

    • Font inspector now shows all fonts (including fonts in iframes) when clicking “show all fonts” button. (Bug 1097150)
    • Addons can now register custom actors. (Bug 1107888)
    • The inspector sidebar now loads and refreshes lazily, leading to better performance of the inspector. (Bug 1103993)
    • Box Model Highlighter colors have been updated so they are easier to see on a wider range of backgrounds, as well as more accessible. (Bug 989053)
    • A new Firefox CLI option was added, –start-debugging-server, for starting the Firefox debugging server on a specific port. (Bug 1119894)
    • The tools tabbar height has decreased to allow more vertical space in the toolbox. (Bug 1109288)

    As we mentioned above, your feedback guides our priorities. Add your comments here, talk to us on Twitter @FirefoxDevTools, or propose changes on the Developer Tools feedback channel. If you’d like to help out with Dev Tools, check out the guide to getting involved.

  6. 2014: Mozilla Hacks looks back

    Wherever you live, it’s a season of work holidays, school vacations, year-end blog posts, and lists. The Hacks blog will be back in early January 2015 to continue writing for developers about the products and technologies created by Mozilla and by builders of the Open Web around the world.

    In the (chronological) list below, we celebrate some 2014 milestones and memorable moments for Mozilla, Firefox, and the web platform:

    • The new improved Firefox Sync launched early in 2014, built atop Firefox Accounts. Here’s a deeper look at the new Sync protocol.
    • Mozilla worked with the Apache Cordova team to integrate Firefox OS with Cordova’s open source device APIs, enabling mobile app developers to access native device functions from JavaScript, and release Cordova apps for the Firefox OS platform. Cordova is the underlying software in Adobe PhoneGap: the integration made it possible for PhoneGap developers to easily port their apps to the Firefox Marketplace.
    • Mozilla Research made great progress on Servo, a prototype browser engine. Servo is written in Rust, a systems programming language designed to support concurrency and parallelism for modern hardware. Another Big Milestone for Servo—Acid2 was reported in the spring.
    • Australis was the codename for our major redesign of the Firefox Desktop browser, which launched in late April. Designer Stephen Horlander wrote a post about the process, titled (Re)Designing Firefox.
    • The summer release of Firefox 31 brought new capabilities for game developers, including access to emscripten for compiling to JavaScript. In Resources for HTML5 game developers, we described some of the tools and techniques for coding and debugging sophisticated, performant HTML5 games. On the Mozilla blog, we featured the first commercial games to leverage asm.js, demonstrating the readiness of HTML5 as a game platform.
    • In October, we shared a Preview of Firefox Hello, a WebRTC-powered voice/video chat service.
    • As part of the festivities surrounding Firefox’s 10th birthday, Mozilla Research launched MozVR to help bring virtual reality to the Web. Grab your oculi and hold on to your hats.
    • At the same time, the Firefox Developer Tools team released Firefox Developer Edition, the first browser created specifically for developers. Though still in its early days, the Developer Edition features Valence, an integrated add-on to let you develop and debug your app across multiple browsers and devices, as well as WebIDE, providing an editing environment for app development and remote debugging.
    • The evolution of asm.js continued with the release of Massive, an open source benchmarking tool that measures asm.js performance specifically. You can run Massive here.
    • Mozilla and partners announced the formation of the Internet Security Research Group (ISRG) and launched the Let’s Encrypt initiative. Let’s Encrypt is a new Certificate Authority that’s free, automated, and open. It’s due to arrive in summer 2015.
    • Our friends at Telenor introduced Gonzo, an ongoing project which explores the use of Firefox OS as an embedded platform. Telenor engineer and Firefox OS evangelist Jan Jongboom is hacking together a wireless camera and sharing what he learns along the way.
    • Firefox OS Expands to Nearly 30 Countries – It’s been an expansive year. At last count, Firefox OS is now up and running on 16 smartphones offered in 29 countries. And in December news, Mozilla and KDDI announced the release of Japan’s first Firefox OS smartphone, which went on sale on December 25. The Fx0 is the first high-spec Firefox OS smartphone and it’s a beauty!

    If you’re interested in writing for the Hacks blog in 2015, we’d love to hear from you. Email us. In the meantime, have a safe and hacky holiday!

  7. Pseudo elements, promise inspection, raw headers, and much more – Firefox Developer Edition 36

    Firefox 36 was just uplifted to the Developer Edition channel, so let’s take a look at the most important Developer Tools changes in this release. We will also cover some changes from Firefox 35 since it was released shortly before the initial Developer Edition announcement. There is a lot to talk about, so let’s get right to it.

    Inspector

    You can now inspect ::before and ::after pseudo elements.  They behave like other elements in the markup tree and inspector sidebars. (35, development notes)

    before-after-inspection

    There is a new “Show DOM Properties” context menu item in the markup tree. (35, development notes, documentation about this feature on MDN)

    show-dom-properties

    The box model highlighter now works on remote targets, so there is a full-featured highlighter even when working with pages on Firefox for Android or apps on Firefox OS. (36, development notes, and technical documentation for building your own custom highlighter)

    Shadow DOM content is now visible in the markup tree; note that you will need to set dom.webcomponents.enabled to true to test this feature (36, development notes, and see bug 1053898 for more work in this space).

    We borrowed a useful feature from Firebug and are now allowing more paste options when right clicking a node in the markup tree.  (36, development notes, documentation about this feature on MDN)

    paste-options

    Some more changes to the Inspector included in Firefox 35 & 36:

    • Deleting a node now selects the previous sibling instead of the parent (36, development notes)
    • There is higher contrast for the currently hovered node in the markup view (36, development notes)
    • Hover over a CSS selector in the computed view to highlight all the nodes that match that selector on the page. (35, development notes)

    Debugger / Console

    DOM Promises are now inspectable. You can inspect the promises state and value at any moment. (36, development notes)

    promise-inspection

    The debugger now recognizes and works with eval’ed sources. (36, development notes)

    Eval’ed sources support the //# sourceURL=path.js syntax, which will make them appear as a normal file in the debugger and in stack traces reported by Error.prototype.stack. See this post: http://fitzgeraldnick.com/weblog/59/ for much more information. (36, development notes,  more development notes)

    Console statements now include the column number they originated from. (36, development notes)

    WebIDE

    You are now able to connect to Firefox for Android from the WebIDE.  See documentation for debugging firefox for android on MDN.  (36, development notes).

    We also made some changes to improve the user experience in the WebIDE:

    • Bring up developer tools by default when I select a runtime app / tab (35, development notes)
    • Re-select project on connect if last project is runtime app (35, development notes)
    • Auto-select and connect to last used runtime if available (35, development notes)
    • Font resizing (36, development notes)
    • You can now adding a hosted app project by entering the base URL (eg: “http://example.com”) instead of requiring the full path to the manifest.webapp file (35, development notes)

    Network Monitor

    We added a plain request/response headers view to make it easier to view and copy the raw headers on a request. (35, development notes)

    net-headers-raw

    Here is a list of all the bugs resolved for Firefox 35 and all the bugs resolved for Firefox 36.

    Do you have feedback, bug reports, feature requests, or questions? As always, you can comment here, add/vote for ideas on UserVoice or get in touch with the team at @FirefoxDevTools on Twitter.

  8. Introducing the JavaScript Internationalization API

    Firefox 29 issued half a year ago, so this post is long overdue. Nevertheless I wanted to pause for a second to discuss the Internationalization API first shipped on desktop in that release (and passing all tests!). Norbert Lindenberg wrote most of the implementation, and I reviewed it and now maintain it. (Work by Makoto Kato should bring this to Android soon; b2g may take longer due to some b2g-specific hurdles. Stay tuned.)

    What’s internationalization?

    Internationalization (i18n for short — i, eighteen characters, n) is the process of writing applications in a way that allows them to be easily adapted for audiences from varied places, using varied languages. It’s easy to get this wrong by inadvertently assuming one’s users come from one place and speak one language, especially if you don’t even know you’ve made an assumption.

    function formatDate(d)
    {
      // Everyone uses month/date/year...right?
      var month = d.getMonth() + 1;
      var date = d.getDate();
      var year = d.getFullYear();
      return month + "/" + date + "/" + year;
    }
     
    function formatMoney(amount)
    {
      // All money is dollars with two fractional digits...right?
      return "$" + amount.toFixed(2);
    }
     
    function sortNames(names)
    {
      function sortAlphabetically(a, b)
      {
        var left = a.toLowerCase(), right = b.toLowerCase();
        if (left > right)
          return 1;
        if (left === right)
          return 0;
        return -1;
      }
     
      // Names always sort alphabetically...right?
      names.sort(sortAlphabetically);
    }

    JavaScript’s historical i18n support is poor

    i18n-aware formatting in traditional JS uses the various toLocaleString() methods. The resulting strings contained whatever details the implementation chose to provide: no way to pick and choose (did you need a weekday in that formatted date? is the year irrelevant?). Even if the proper details were included, the format might be wrong e.g. decimal when percentage was desired. And you couldn’t choose a locale.

    As for sorting, JS provided almost no useful locale-sensitive text-comparison (collation) functions. localeCompare() existed but with a very awkward interface unsuited for use with sort. And it too didn’t permit choosing a locale or specific sort order.

    These limitations are bad enough that — this surprised me greatly when I learned it! — serious web applications that need i18n capabilities (most commonly, financial sites displaying currencies) will box up the data, send it to a server, have the server perform the operation, and send it back to the client. Server roundtrips just to format amounts of money. Yeesh.

    A new JS Internationalization API

    The new ECMAScript Internationalization API greatly improves JavaScript’s i18n capabilities. It provides all the flourishes one could want for formatting dates and numbers and sorting text. The locale is selectable, with fallback if the requested locale is unsupported. Formatting requests can specify the particular components to include. Custom formats for percentages, significant digits, and currencies are supported. Numerous collation options are exposed for use in sorting text. And if you care about performance, the up-front work to select a locale and process options can now be done once, instead of once every time a locale-dependent operation is performed.

    That said, the API is not a panacea. The API is “best effort” only. Precise outputs are almost always deliberately unspecified. An implementation could legally support only the oj locale, or it could ignore (almost all) provided formatting options. Most implementations will have high-quality support for many locales, but it’s not guaranteed (particularly on resource-constrained systems such as mobile).

    Under the hood, Firefox’s implementation depends upon the International Components for Unicode library (ICU), which in turn depends upon the Unicode Common Locale Data Repository (CLDR) locale data set. Our implementation is self-hosted: most of the implementation atop ICU is written in JavaScript itself. We hit a few bumps along the way (we haven’t self-hosted anything this large before), but nothing major.

    The Intl interface

    The i18n API lives on the global Intl object. Intl contains three constructors: Intl.Collator, Intl.DateTimeFormat, and Intl.NumberFormat. Each constructor creates an object exposing the relevant operation, efficiently caching locale and options for the operation. Creating such an object follows this pattern:

    var ctor = "Collator"; // or the others
    var instance = new Intl[ctor](locales, options);

    locales is a string specifying a single language tag or an arraylike object containing multiple language tags. Language tags are strings like en (English generally), de-AT (German as used in Austria), or zh-Hant-TW (Chinese as used in Taiwan, using the traditional Chinese script). Language tags can also include a “Unicode extension”, of the form -u-key1-value1-key2-value2..., where each key is an “extension key”. The various constructors interpret these specially.

    options is an object whose properties (or their absence, by evaluating to undefined) determine how the formatter or collator behaves. Its exact interpretation is determined by the individual constructor.

    Given locale information and options, the implementation will try to produce the closest behavior it can to the “ideal” behavior. Firefox supports 400+ locales for collation and 600+ locales for date/time and number formatting, so it’s very likely (but not guaranteed) the locales you might care about are supported.

    Intl generally provides no guarantee of particular behavior. If the requested locale is unsupported, Intl allows best-effort behavior. Even if the locale is supported, behavior is not rigidly specified. Never assume that a particular set of options corresponds to a particular format. The phrasing of the overall format (encompassing all requested components) might vary across browsers, or even across browser versions. Individual components’ formats are unspecified: a short-format weekday might be “S”, “Sa”, or “Sat”. The Intl API isn’t intended to expose exactly specified behavior.

    Date/time formatting

    Options

    The primary options properties for date/time formatting are as follows:

    weekday, era
    "narrow", "short", or "long". (era refers to typically longer-than-year divisions in a calendar system: BC/AD, the current Japanese emperor’s reign, or others.)
    month
    "2-digit", "numeric", "narrow", "short", or "long"
    year
    day
    hour, minute, second
    "2-digit" or "numeric"
    timeZoneName
    "short" or "long"
    timeZone
    Case-insensitive "UTC" will format with respect to UTC. Values like "CEST" and "America/New_York" don’t have to be supported, and they don’t currently work in Firefox.

    The values don’t map to particular formats: remember, the Intl API almost never specifies exact behavior. But the intent is that "narrow", "short", and "long" produce output of corresponding size — “S” or “Sa”, “Sat”, and “Saturday”, for example. (Output may be ambiguous: Saturday and Sunday both could produce “S”.) "2-digit" and "numeric" map to two-digit number strings or full-length numeric strings: “70” and “1970”, for example.

    The final used options are largely the requested options. However, if you don’t specifically request any weekday/year/month/day/hour/minute/second, then year/month/day will be added to your provided options.

    Beyond these basic options are a few special options:

    hour12
    Specifies whether hours will be in 12-hour or 24-hour format. The default is typically locale-dependent. (Details such as whether midnight is zero-based or twelve-based and whether leading zeroes are present are also locale-dependent.)

    There are also two special properties, localeMatcher (taking either "lookup" or "best fit") and formatMatcher (taking either "basic" or "best fit"), each defaulting to "best fit". These affect how the right locale and format are selected. The use cases for these are somewhat esoteric, so you should probably ignore them.

    Locale-centric options

    DateTimeFormat also allows formatting using customized calendaring and numbering systems. These details are effectively part of the locale, so they’re specified in the Unicode extension in the language tag.

    For example, Thai as spoken in Thailand has the language tag th-TH. Recall that a Unicode extension has the format -u-key1-value1-key2-value2.... The calendaring system key is ca, and the numbering system key is nu. The Thai numbering system has the value thai, and the Chinese calendaring system has the value chinese. Thus to format dates in this overall manner, we tack a Unicode extension containing both these key/value pairs onto the end of the language tag: th-TH-u-ca-chinese-nu-thai.

    For more information on the various calendaring and numbering systems, see the full DateTimeFormat documentation.

    Examples

    After creating a DateTimeFormat object, the next step is to use it to format dates via the handy format() function. Conveniently, this function is a bound function: you don’t have to call it on the DateTimeFormat directly. Then provide it a timestamp or Date object.

    Putting it all together, here are some examples of how to create DateTimeFormat options for particular uses, with current behavior in Firefox.

    var msPerDay = 24 * 60 * 60 * 1000;
     
    // July 17, 2014 00:00:00 UTC.
    var july172014 = new Date(msPerDay * (44 * 365 + 11 + 197));

    Let’s format a date for English as used in the United States. Let’s include two-digit month/day/year, plus two-digit hours/minutes, and a short time zone to clarify that time. (The result would obviously be different in another time zone.)

    var options =
      { year: "2-digit", month: "2-digit", day: "2-digit",
        hour: "2-digit", minute: "2-digit",
        timeZoneName: "short" };
    var americanDateTime =
      new Intl.DateTimeFormat("en-US", options).format;
     
    print(americanDateTime(july172014)); // 07/16/14, 5:00 PM PDT

    Or let’s do something similar for Portuguese — ideally as used in Brazil, but in a pinch Portugal works. Let’s go for a little longer format, with full year and spelled-out month, but make it UTC for portability.

    var options =
      { year: "numeric", month: "long", day: "numeric",
        hour: "2-digit", minute: "2-digit",
        timeZoneName: "short", timeZone: "UTC" };
    var portugueseTime =
      new Intl.DateTimeFormat(["pt-BR", "pt-PT"], options);
     
    // 17 de julho de 2014 00:00 GMT
    print(portugueseTime.format(july172014));

    How about a compact, UTC-formatted weekly Swiss train schedule? We’ll try the official languages from most to least popular to choose the one that’s most likely to be readable.

    var swissLocales = ["de-CH", "fr-CH", "it-CH", "rm-CH"];
    var options =
      { weekday: "short",
        hour: "numeric", minute: "numeric",
        timeZone: "UTC", timeZoneName: "short" };
    var swissTime =
      new Intl.DateTimeFormat(swissLocales, options).format;
     
    print(swissTime(july172014)); // Do. 00:00 GMT

    Or let’s try a date in descriptive text by a painting in a Japanese museum, using the Japanese calendar with year and era:

    var jpYearEra =
      new Intl.DateTimeFormat("ja-JP-u-ca-japanese",
                              { year: "numeric", era: "long" });
     
    print(jpYearEra.format(july172014)); // 平成26年

    And for something completely different, a longer date for use in Thai as used in Thailand — but using the Thai numbering system and Chinese calendar. (Quality implementations such as Firefox’s would treat plain th-TH as th-TH-u-ca-buddhist-nu-latn, imputing Thailand’s typical Buddhist calendar system and Latin 0-9 numerals.)

    var options =
      { year: "numeric", month: "long", day: "numeric" };
    var thaiDate =
      new Intl.DateTimeFormat("th-TH-u-nu-thai-ca-chinese", options);
     
    print(thaiDate.format(july172014)); // ๒๐ 6 ๓๑

    Calendar and numbering system bits aside, it’s relatively simple. Just pick your components and their lengths.

    Number formatting

    Options

    The primary options properties for number formatting are as follows:

    style
    "currency", "percent", or "decimal" (the default) to format a value of that kind.
    currency
    A three-letter currency code, e.g. USD or CHF. Required if style is "currency", otherwise meaningless.
    currencyDisplay
    "code", "symbol", or "name", defaulting to "symbol". "code" will use the three-letter currency code in the formatted string. "symbol" will use a currency symbol such as $ or £. "name" typically uses some sort of spelled-out version of the currency. (Firefox currently only supports "symbol", but this will be fixed soon.)
    minimumIntegerDigits
    An integer from 1 to 21 (inclusive), defaulting to 1. The resulting string is front-padded with zeroes until its integer component contains at least this many digits. (For example, if this value were 2, formatting 3 might produce “03”.)
    minimumFractionDigits, maximumFractionDigits
    Integers from 0 to 20 (inclusive). The resulting string will have at least minimumFractionDigits, and no more than maximumFractionDigits, fractional digits. The default minimum is currency-dependent (usually 2, rarely 0 or 3) if style is "currency", otherwise 0. The default maximum is 0 for percents, 3 for decimals, and currency-dependent for currencies.
    minimumSignificantDigits, maximumSignificantDigits
    Integers from 1 to 21 (inclusive). If present, these override the integer/fraction digit control above to determine the minimum/maximum significant figures in the formatted number string, as determined in concert with the number of decimal places required to accurately specify the number. (Note that in a multiple of 10 the significant digits may be ambiguous, as in “100” with its one, two, or three significant digits.)
    useGrouping
    Boolean (defaulting to true) determining whether the formatted string will contain grouping separators (e.g. “,” as English thousands separator).

    NumberFormat also recognizes the esoteric, mostly ignorable localeMatcher property.

    Locale-centric options

    Just as DateTimeFormat supported custom numbering systems in the Unicode extension using the nu key, so too does NumberFormat. For example, the language tag for Chinese as used in China is zh-CN. The value for the Han decimal numbering system is hanidec. To format numbers for these systems, we tack a Unicode extension onto the language tag: zh-CN-u-nu-hanidec.

    For complete information on specifying the various numbering systems, see the full NumberFormat documentation.

    Examples

    NumberFormat objects have a format function property just as DateTimeFormat objects do. And as there, the format function is a bound function that may be used in isolation from the NumberFormat.

    Here are some examples of how to create NumberFormat options for particular uses, with Firefox’s behavior. First let’s format some money for use in Chinese as used in China, specifically using Han decimal numbers (instead of much more common Latin numbers). Select the "currency" style, then use the code for Chinese renminbi (yuan), grouping by default, with the usual number of fractional digits.

    var hanDecimalRMBInChina =
      new Intl.NumberFormat("zh-CN-u-nu-hanidec",
                            { style: "currency", currency: "CNY" });
     
    print(hanDecimalRMBInChina.format(1314.25)); // ¥ 一,三一四.二五

    Or let’s format a United States-style gas price, with its peculiar thousandths-place 9, for use in English as used in the United States.

    var gasPrice =
      new Intl.NumberFormat("en-US",
                            { style: "currency", currency: "USD",
                              minimumFractionDigits: 3 });
     
    print(gasPrice.format(5.259)); // $5.259

    Or let’s try a percentage in Arabic, meant for use in Egypt. Make sure the percentage has at least two fractional digits. (Note that this and all the other RTL examples may appear with different ordering in RTL context, e.g. ٤٣٫٨٠٪ instead of ٤٣٫٨٠٪.)

    var arabicPercent =
      new Intl.NumberFormat("ar-EG",
                            { style: "percent",
                              minimumFractionDigits: 2 }).format;
     
    print(arabicPercent(0.438)); // ٤٣٫٨٠٪

    Or suppose we’re formatting for Persian as used in Afghanistan, and we want at least two integer digits and no more than two fractional digits.

    var persianDecimal =
      new Intl.NumberFormat("fa-AF",
                            { minimumIntegerDigits: 2,
                              maximumFractionDigits: 2 });
     
    print(persianDecimal.format(3.1416)); // ۰۳٫۱۴

    Finally, let’s format an amount of Bahraini dinars, for Arabic as used in Bahrain. Unusually compared to most currencies, Bahraini dinars divide into thousandths (fils), so our number will have three places. (Again note that apparent visual ordering should be taken with a grain of salt.)

    var bahrainiDinars =
      new Intl.NumberFormat("ar-BH",
                            { style: "currency", currency: "BHD" });
     
    print(bahrainiDinars.format(3.17)); // د.ب.‏ ٣٫١٧٠

    Collation

    Options

    The primary options properties for collation are as follows:

    usage
    "sort" or "search" (defaulting to "sort"), specifying the intended use of this Collator. (A search collator might want to consider more strings equivalent than a sort collator would.)
    sensitivity
    "base", "accent", "case", or "variant". This affects how sensitive the collator is to characters that have the same “base letter” but have different accents/diacritics and/or case. (Base letters are locale-dependent: “a” and “ä” have the same base letter in German but are different letters in Swedish.) "base" sensitivity considers only the base letter, ignoring modifications (so for German “a”, “A”, and “ä” are considered the same). "accent" considers the base letter and accents but ignores case (so for German “a” and “A” are the same, but “ä” differs from both). "case" considers the base letter and case but ignores accents (so for German “a” and “ä” are the same, but “A” differs from both). Finally, "variant" considers base letter, accents, and case (so for German “a”, “ä, “ä” and “A” all differ). If usage is "sort", the default is "variant"; otherwise it’s locale-dependent.
    numeric
    Boolean (defaulting to false) determining whether complete numbers embedded in strings are considered when sorting. For example, numeric sorting might produce "F-4 Phantom II", "F-14 Tomcat", "F-35 Lightning II"; non-numeric sorting might produce "F-14 Tomcat", "F-35 Lightning II", "F-4 Phantom II".
    caseFirst
    "upper", "lower", or "false" (the default). Determines how case is considered when sorting: "upper" places uppercase letters first ("B", "a", "c"), "lower" places lowercase first ("a", "c", "B"), and "false" ignores case entirely ("a", "B", "c"). (Note: Firefox currently ignores this property.)
    ignorePunctuation
    Boolean (defaulting to false) determining whether to ignore embedded punctuation when performing the comparison (for example, so that "biweekly" and "bi-weekly" compare equivalent).

    And there’s that localeMatcher property that you can probably ignore.

    Locale-centric options

    The main Collator option specified as part of the locale’s Unicode extension is co, selecting the kind of sorting to perform: phone book (phonebk), dictionary (dict), and many others.

    Additionally, the keys kn and kf may, optionally, duplicate the numeric and caseFirst properties of the options object. But they’re not guaranteed to be supported in the language tag, and options is much clearer than language tag components. So it’s best to only adjust these options through options.

    These key-value pairs are included in the Unicode extension the same way they’ve been included for DateTimeFormat and NumberFormat; refer to those sections for how to specify these in a language tag.

    Examples

    Collator objects have a compare function property. This function accepts two arguments x and y and returns a number less than zero if x compares less than y, 0 if x compares equal to y, or a number greater than zero if x compares greater than y. As with the format functions, compare is a bound function that may be extracted for standalone use.

    Let’s try sorting a few German surnames, for use in German as used in Germany. There are actually two different sort orders in German, phonebook and dictionary. Phonebook sort emphasizes sound, and it’s as if “ä”, “ö”, and so on were expanded to “ae”, “oe”, and so on prior to sorting.

    var names =
      ["Hochberg", "Hönigswald", "Holzman"];
     
    var germanPhonebook = new Intl.Collator("de-DE-u-co-phonebk");
     
    // as if sorting ["Hochberg", "Hoenigswald", "Holzman"]:
    //   Hochberg, Hönigswald, Holzman
    print(names.sort(germanPhonebook.compare).join(", "));

    Some German words conjugate with extra umlauts, so in dictionaries it’s sensible to order ignoring umlauts (except when ordering words differing only by umlauts: schon before schön).

    var germanDictionary = new Intl.Collator("de-DE-u-co-dict");
     
    // as if sorting ["Hochberg", "Honigswald", "Holzman"]:
    //   Hochberg, Holzman, Hönigswald
    print(names.sort(germanDictionary.compare).join(", "));

    Or let’s sort a list Firefox versions with various typos (different capitalizations, random accents and diacritical marks, extra hyphenation), in English as used in the United States. We want to sort respecting version number, so do a numeric sort so that numbers in the strings are compared, not considered character-by-character.

    var firefoxen =
      ["FireFøx 3.6",
       "Fire-fox 1.0",
       "Firefox 29",
       "FÍrefox 3.5",
       "Fírefox 18"];
     
    var usVersion =
      new Intl.Collator("en-US",
                        { sensitivity: "base",
                          numeric: true,
                          ignorePunctuation: true });
     
    // Fire-fox 1.0, FÍrefox 3.5, FireFøx 3.6, Fírefox 18, Firefox 29
    print(firefoxen.sort(usVersion.compare).join(", "));

    Last, let’s do some locale-aware string searching that ignores case and accents, again in English as used in the United States.

    // Comparisons work with both composed and decomposed forms.
    var decoratedBrowsers =
      [
       "A\u0362maya",  // A͢maya
       "CH\u035Brôme", // CH͛rôme
       "FirefÓx",
       "sAfàri",
       "o\u0323pERA",  // ọpERA
       "I\u0352E",     // I͒E
      ];
     
    var fuzzySearch =
      new Intl.Collator("en-US",
                        { usage: "search", sensitivity: "base" });
     
    function findBrowser(browser)
    {
      function cmp(other)
      {
        return fuzzySearch.compare(browser, other) === 0;
      }
      return cmp;
    }
     
    print(decoratedBrowsers.findIndex(findBrowser("Firêfox"))); // 2
    print(decoratedBrowsers.findIndex(findBrowser("Safåri")));  // 3
    print(decoratedBrowsers.findIndex(findBrowser("Ãmaya")));   // 0
    print(decoratedBrowsers.findIndex(findBrowser("Øpera")));   // 4
    print(decoratedBrowsers.findIndex(findBrowser("Chromè")));  // 1
    print(decoratedBrowsers.findIndex(findBrowser("IË")));      // 5

    Odds and ends

    It may be useful to determine whether support for some operation is provided for particular locales, or to determine whether a locale is supported. Intl provides supportedLocales() functions on each constructor, and resolvedOptions() functions on each prototype, to expose this information.

    var navajoLocales =
      Intl.Collator.supportedLocalesOf(["nv"], { usage: "sort" });
    print(navajoLocales.length > 0
          ? "Navajo collation supported"
          : "Navajo collation not supported");
     
    var germanFakeRegion =
      new Intl.DateTimeFormat("de-XX", { timeZone: "UTC" });
    var usedOptions = germanFakeRegion.resolvedOptions();
    print(usedOptions.locale);   // de
    print(usedOptions.timeZone); // UTC

    Legacy behavior

    The ES5 toLocaleString-style and localeCompare functions previously had no particular semantics, accepted no particular options, and were largely useless. So the i18n API reformulates them in terms of Intl operations. Each method now accepts additional trailing locales and options arguments, interpreted just as the Intl constructors would do. (Except that for toLocaleTimeString and toLocaleDateString, different default components are used if options aren’t provided.)

    For brief use where precise behavior doesn’t matter, the old methods are fine to use. But if you need more control or are formatting or comparing many times, it’s best to use the Intl primitives directly.

    Conclusion

    Internationalization is a fascinating topic whose complexity is bounded only by the varied nature of human communication. The Internationalization API addresses a small but quite useful portion of that complexity, making it easier to produce locale-sensitive web applications. Go use it!

    (And a special thanks to Norbert Lindenberg, Anas El Husseini, Simon Montagu, Gary Kwong, Shu-yu Guo, Ehsan Akhgari, the people of #mozilla.de, and anyone I may have forgotten [sorry!] who provided feedback on this article or assisted me in producing and critiquing the examples. The English and German examples were the limit of my knowledge, and I’d have been completely lost on the other examples without their assistance. Blame all remaining errors on me. Thanks again!)

  9. Videos and Firefox OS

    Before HTML5

    Those were dark times Harry, dark times – Rubeus Hagrid

    Before HTML5, displaying video on the Web required browser plugins and Flash.

    Luckily, Firefox OS supports HTML5 video so we don’t need to support these older formats.

    Video support on the Web

    Even though modern browsers support HTML5, the video formats they support vary:

    In summary, to support the most browsers with the fewest formats you need the MP4 and WebM video formats (Firefox prefers WebM).

    Multiple sizes

    Now that you have seen what formats you can use, you need to decide on video resolutions, as desktop users on high speed wifi will expect better quality videos than mobile users on 3G.

    At Rormix we decided on 720p for desktop, 360p for mobile connections, and 180p specially for Firefox OS to reduce the cost in countries with high data charges.

    There are no hard and fast rules — it depends on who your market audience is.

    Streaming?

    The best streaming solution would be to automatically serve the user different videos sizes depending on their connection status (adaptive streaming) but support for this technology is poor.

    HTTP live streaming works well on Apple devices, but has poor support on Android.

    At the time of writing, the most promising technology is MPEG DASH, which is an international standard.

    In summary, we are going to have to wait before we get an adaptive streaming technology that is widely accepted (Firefox does not support HLS or MPEG DASH).

    DIY Adaptive streaming

    In the absence of adaptive streaming we need to try to work out the best video quality to load at the outset. The following is a quick guide to help you decide:

    Wifi or 3G

    Using a certified Firefox OS app you can check to see if the user is on wifi or not.

    var lock    = navigator.mozSettings.createLock();
    var setting = lock.get('wifi.enabled');
     
    setting.onsuccess = function () {
      console.log('wifi.enabled: ' + setting.result);
    }
     
    setting.onerror = function () {
      console.warn('An error occured: ' + setting.error);
    }

    https://developer.mozilla.org/en-US/docs/Web/API/Settings_API

    There is some more information at the W3C Device API.

    Detecting screen size

    There is no point sending a 720p video to a user with a screen smaller than 720p. There are many ways to get the different bounds of a user’s screen; innerWidth and width allow you to get a good idea:

    function getVidSize()
    {
      //Get the width of the phone (rotation independent)
      var min = Math.min($(window).innerHeight(),$(window).innerWidth());
      //Return a video size we have
      if(min < 320)      return '180';
      else if(min < 550) return '360';
      else               return '720';
    }

    http://www.quirksmode.org/m/tests/widthtest.html

    Determining internet speed

    It is difficult to get an accurate read of a user’s internet speed using web technologies — usually they involve loading a large image onto the user’s device and timing it. This has the disadvantage of having to send more data to the user. Some services such as: http://speedof.me/api.html exist, but still require data downloads to the user’s device. (Stackoverflow has some more options.)

    You can be slightly more clever by using HTML5, and checking the time it takes between the user starting the video and a set amount of the video loading. This way we do not need to load any extra data on the user’s device. A quick VideoJS example follows:

    var global_speedcount = 0;
    var global_video = null;
    global_video = videojs("video", {}, function(){
    //Set up video sources
    });
     
    global_video.on('play',function(){
      //User has clicked play
      global_speedcount = new Date().getTime();
    });
     
    function timer()
    {
      var diff = new Date().getTime() - global_speedcount;
      //Remove this handler as it is run multiple times per second!
      global_video.off('timeupdate',timer);
    }
     
    global_video.on('timeupdate',timer);

    This code starts timing when the user clicks play, and when the browser starts to play the video it sends timing information to timeupdate. You can also use this function to detect if lots of buffering is happening.

    Detect high resolution devices

    One final thing to determine is whether or not a user has a high pixel density screen. In this case even if they have a small screen it can still have a large number of pixels (and therefore require a higher resolution video).

    Modernizr has a plugin for detecting hi-res screens.

    if (Modernizr.highresdisplay)
    {
      alert('Your device has a high resolution screen');
    }

    WebP Thumbnails

    Not to get embroiled in an argument, but at Rormix we have seen an average decrease of 30% in file size (WebP vs JPEG) with no loss of quality (in some cases up to 50% less). And in countries with expensive data plans, the less data the better.

    We encode all of our thumbnails in multiple resolutions of WebP and send them to every device that supports them to reduce the amount of data being sent to the user.

    Mobile considerations

    If you are playing HTML5 videos on mobile devices, their behavior differs. On iOS it automatically goes to full screen on iPhones/iPods, but not on tablets.

    Some libraries such as VideoJS have removed the controls from mobile devices until their stability increases.

    Useful libraries

    There are a few useful HTML5 video libraries:

    Mozilla links

    Mozilla has some great articles on web video:

    Other useful Links

  10. Mozilla Introduces the First Browser Built For Developers: Firefox Developer Edition

    Developers are critical to the continued success of the Web. The content and apps they create compel us to come back to the Web every day, whether on a computer or mobile phone.

    In celebration of the 10th anniversary of Firefox, we’re excited to unveil Firefox Developer Edition, the first browser created specifically for developers.

    Ten years ago, we built Firefox for early adopters and developers to give them more choice and control. Firefox integrated WebAPIs and Add-ons to enable people to get the most out of the Web. Now we’re giving developers the whole browser as a hard-hat area, allowing us to bring front and center the features most relevant to them. Having a dedicated developer browser means we can tailor the browsing experience to what developers do every day.

    Because Firefox is part of an open-source, independent community and not part of a proprietary ecosystem, we’re able to offer features other browsers can’t by applying our tools everywhere the Web goes, regardless of platform or device.

    One of the biggest pain points for developers is having to use numerous siloed development environments in order to create engaging content or for targeting different app stores. For these reasons, developers often end up having to bounce between different platforms and browsers, which decreases productivity and causes frustration.

    Firefox Developer Edition solves this problem by creating a focal point to streamline your development workflow. It’s a stable developer browser which is not only a powerful authoring tool but also robust enough for everyday browsing. It also adds new features that simplify the process of building for the entire Web, whether targeting mobile or desktop across many different platforms.

    If you’re an experienced developer, you’ll already be familiar with the installed tools so you can focus on developing your content or app as soon as you open the browser. There’s no need to download additional plugins or applications to debug mobile devices. If you’re a new Web developer, the streamlined workflow and the fact that everything is already set up and ready to go makes it easier to get started building sophisticated applications.

    So what’s under the hood?

    The first thing you’ll notice is the distinctive dark design running through the browser. We applied the developer tools theme to the entire browser. It’s trim and sharp and focused on saving space for the content on your screen. It also fits in with the darker look common among creative app development tools.

    We’ve also integrated two powerful new features, Valence and WebIDE that improve workflow and help you debug other browsers and apps directly from within Firefox Developer Edition.

    Valence (previously called Firefox Tools Adapter) lets you develop and debug your app across multiple browsers and devices by connecting the Firefox dev tools to other major browser engines. Valence also extends the awesome tools we’ve built to debug Firefox OS and Firefox for Android to the other major mobile browsers including Chrome on Android and Safari on iOS. So far these tools include our Inspector, Debugger and Console and Style Editor.

    WebIDE allows you to develop, deploy and debug Web apps directly in your browser, or on a Firefox OS device. It lets you create a new Firefox OS app (which is just a web app) from a template, or open up the code of an existing app. From there you can edit the app’s files. It’s one click to run the app in a simulator and one more to debug it with the developer tools.

    Firefox Developer Edition also includes all the tools experienced Web developers are familiar with, including:

    • Responsive Design Mode – see how your website or Web app will look on different screen sizes without changing the size of your browser window.
    • Page Inspector– examine the HTML and CSS of any Web page and easily modify the structure and layout of a page.
    • Web Console – see logged information associated with a Web page and use Web Console and interact with a Web page using JavaScript.
    • JavaScript Debugger – step through JavaScript code and examine or modify its state to help track down bugs.
    • Network Monitor – see all the network requests your browser makes, how long each request takes and details of each request.
    • Style Editor – view and edit CSS styles associated with a Web page, create new ones and apply existing CSS stylesheets to any page.
    • Web Audio Editor – inspect and interact with Web Audio API in real time to ensure that all audio nodes are connected in the way you expect.

    Give it a try and let us know what you think. We’re keen to hear your feedback.

    More Information: