Featured Articles

Sort by:


  1. Cross-browser camera capture with getUserMedia/WebRTC


    With Firefox adding support for getUserMedia, three of the major desktop browsers now have the ability to get data from cameras without the use of plugins. As it’s still early days, however, the implementations differ slightly between browsers. Below is an example of how to work around these differences and a script to do the heavy lifting for you, but first, an overview of how the three browsers stack up.

    Comparison of getUserMedia behaviour in browsers as of February 2013
    Firefox 18 Opera 12 Chrome 24
    Requires vendor prefix Yes (moz) No Yes (webkit)
    Triggered with autoplay attribute No Yes Yes
    Requires enabling by user Yes 1 No No
    Firing of playing event Repeatedly Once Once
    Supports file:// protocol Yes Yes No
    Tab playing notification None Icon Animated icon
    Permission request Each page load First page load only Each page load

    getUserMedia has to be enabled in Firefox by setting the media.peerconnection.enabled option to true in about:config.

    There are a few more differences once we start coding so let’s walk through it. Our recipe for getUserMedia success will be broken down into the following easy steps:

    1. A helping of HTML5
    2. A dollop of feature detection
    3. A spoonful of streaming
    4. Ready for serving
    5. A final tip

    Deep breath – here we go…

    A helping of HTML5

    Our main task in this short tutorial is just to get a moving image displaying in a page. In that respect, it’s no different to regular video so the first step is a simple <video> element in our HTML:


    That’s it. No controls, no src, no nothing.

    Over to the JavaScript, and obviously we need to get a reference to the <video> element, which we can do like so (or alternatively with an id):

    var video = document.querySelector('video');

    A dollop of feature detection

    Now it gets interesting as we check for getUserMedia support. We’re definitely not going to use unreliable user agent sniffing for this — no, we’ll do it the easy way by checking for the navigator.getUserMedia object. This is prefixed in Firefox and Chrome so first it’s handy to assign it to a common object for all browsers. While we’re at it, let’s do it for the window.URL object as well which we’ll use later on.

    navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia;
    window.URL = window.URL || window.webkitURL || window.mozURL || window.msURL;

    And next, the actual existence checking:

    if (navigator.getUserMedia) {
        // Call the getUserMedia method here
    } else {
        console.log('Native device media streaming (getUserMedia) not supported in this browser.');
        // Display a friendly "sorry" message to the user.

    If getUserMedia is supported, we need to pass it three arguments — an options object, a success callback function and an error callback function. Note that the error callback is required in Firefox but optional in Opera and Chrome. The options argument is a JSON-style object that specifies whether audio, video or both are to be used. The following example code is for video only:

    navigator.getUserMedia({video: true}, successCallback, errorCallback);

    Dialogs requesting camera access

    Dialog in Firefox
    Dialog in Google Chrome
    Dialog in Opera

    A spoonful of streaming

    So far so good, so let’s define what happens next. The success callback function receives an argument containing the video stream from the camera and we want to send that stream to our <video> element. We do this by setting its src attribute but there are a couple of things to bear in mind:

    • Firefox uses the mozSrcObject attribute whereas Opera and Chrome use src.
    • Chrome uses the createObjectURL method whereas Firefox and Opera send the stream directly.

    With Firefox, video.mozSrcObject is initially null rather than undefined so we can rely on this to detect for Firefox’s support (hat tip to Florent). Once the stream knows where to go we can tell the video stream to play.

    function successCallback(stream) {
        if (video.mozSrcObject !== undefined) {
            video.mozSrcObject = stream;
        } else {
            video.src = (window.URL && window.URL.createObjectURL(stream)) || stream;

    Ready for serving

    And there you have it. Add a simple error callback function and we have a working cross-browser script which looks something like this:

    window.addEventListener('DOMContentLoaded', function() {
        'use strict';
        var video = document.querySelector('video');
        function successCallback(stream) {
            // Set the source of the video element with the stream from the camera
            if (video.mozSrcObject !== undefined) {
                video.mozSrcObject = stream;
            } else {
                video.src = (window.URL && window.URL.createObjectURL(stream)) || stream;
        function errorCallback(error) {
            console.error('An error occurred: [CODE ' + error.code + ']');
            // Display a friendly "sorry" message to the user
        navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia;
        window.URL = window.URL || window.webkitURL || window.mozURL || window.msURL;
        // Call the getUserMedia method with our callback functions
        if (navigator.getUserMedia) {
            navigator.getUserMedia({video: true}, successCallback, errorCallback);
        } else {
            console.log('Native web camera streaming (getUserMedia) not supported in this browser.');
            // Display a friendly "sorry" message to the user
    }, false);

    Available on GitHub

    To get started with accessing getUserMedia in a cross web browser fashion, we have also put a working example on GitHub: GumWrapper.

    A final tip

    If you want to do anything fancy with the camera’s stream like capture a still image or add fancy effects, you’ll probably want to send its data to a canvas context. You can use drawImage() for this, in which case you’ll need the dimensions of the video. These are available through the video.videoWidth and video.videoHeight properties but beware — they’re only set when the browser has information about the stream. This means you have to listen for certain events before you can get these properties. There are a few relevant events, always fired in the following order:

    1. play
    2. loadedmetadata
    3. loadeddata
    4. playing

    The play event is fired after the method is called but there may be a slight delay before the video actually starts playing. That’s where the playing event comes in but note that it’s fired repeatedly in Firefox while the stream or video is playing. Before that are a couple of data events, the first of which is just for metadata, however in Firefox this doesn’t include video dimensions. Consequently, the most reliable event to listen for is the loadeddata event — you can then be sure of knowing the width and height of the video stream. You could code it up like this:

    video.addEventListener('loadeddata', function() {
    console.log('Video dimensions: ' + video.videoWidth + ' x ' + video.videoHeight);
    }, false);

    Incidentally, you could also use the stream’s dimensions as a further error check, for example checking whether the width and height are above 0. This would avoid problems such as the user’s webcam being broken or simply not plugged in.

    And there you have it. I’m sure the differences between browsers will disappear as the technology matures but for the time being, the above code should help you on your way.

  2. Using WebAPIs to make the web layer more capable

    Part of making both Firefox OS and the web as a platform a stronger layer and alternative for developers, we are working on a number of WebAPIs. I’d like to introduce you them here!

    Many things covered in this blog post are also available in a talk I’ve given on this topic.


    When you talk about new technology, ideas and solutions, people have a tendency to be wary. They might see it as the Not Invented Here syndrome, but that is not our intent (as we’ve written here before, and as mentioned by our CTO, Brendan Eich).

    Our intention is that everything we do is, or will be, standardized. Developed in the open.

    This blog post focuses on various WebAPIs, their security level and how to use them. In a later post, we will follow up specifically on standardization and progress in that area.

    Types of APIs

    There are basically three types of WebAPIs:

    Regular APIs
    APIs available from any app, hosted or packaged.
    Privileged APIs
    Only available in privileged apps. Privileged apps are apps that are:

    Certified APIs
    Certified APIs are only available to system itself (Mozilla in the case of Firefox OS), meaning they are so sensitive and need to be strictly controlled.

    Regular APIs

    The complete list of regular APIs is:

    • Vibration API
    • Screen Orientation
    • Geolocation API
    • Mouse Lock API
    • Open WebApps
    • Network Information API
    • Battery Status API
    • Alarm API
    • Web Activities
    • Push Notifications API
    • WebFM API
    • WebPayment
    • IndexedDB
    • Ambient light sensor
    • Proximity sensor
    • Notification
    • FMRadio

    Here’s how to use a number of them:

    Battery Status API

    This API is about detecting the current battery level of the computer/device, how long battery life there’s left and whether it’s being charged or not. Works in all three of desktop, Android and Firefox OS.

    var battery = navigator.battery;if (battery) {var batteryLevel = Math.round(battery.level * 100) + "%",

            charging = (battery.charging)? "" : "not ",

            chargingTime = parseInt(battery.chargingTime / 60, 10),

            dischargingTime = parseInt(battery.dischargingTime / 60, 10);
            // Set events

            battery.addEventListener("levelchange", showStatus);
            battery.addEventListener("chargingchange", showStatus);
            battery.addEventListener("chargingtimechange", showStatus);
            battery.addEventListener("dischargingtimechange", showStatus);}

    Vibration API

    Making the device vibrate, either once or with a certain pattern.

    // Vibrate for one second

    // Vibration pattern [vibrationTime, pause,…]

    navigator.vibrate([200, 100, 200, 100]);
    // Vibrate for 5 seconds

    // Turn off vibration


    Screen Orientation

    Gives you as a developer the chance to lock the orientation, or to specify what you want the primary orientation experience to be.

        Possible values:

    var portraitLock = screen.mozLockOrientation("portrait");
    if (portraitLock) {
        console.log("Orientation locked to portrait");

    Geolocation API

    Finding out where the user is at the moment – approved by the user to share this.

    navigator.geolocation.getCurrentPosition(function (position) {
            Getting latitude and longitude:

    Mouse Lock API

    Locking down the mouse movements and controlling the experience yourself. Especially good when you don’t want the mouse pointer/interaction to end when it hits the end of the web browser window, but rather to continue to scroll – like doing complete 360 degrees spin of an environment or similar.

    var docElm = document.documentElement;// Requesting Pointer Lock

    docElm.requestPointerLock = elem.requestPointerLock || 
                                elem.mozRequestPointerLock || 
    document.addEventListener("mousemove", function(e) {
        var movementX = e.movementX       ||
                        e.mozMovementX    ||
                        e.webkitMovementX ||
            movementY = e.movementY       ||
                        e.mozMovementY    ||
                        e.webkitMovementY ||
        // Get the mouse movement delta values
        console.log("movementX=" + movementX, "movementY=" + movementY);}

    Open WebApps

    In general, an APIs needed to install and handle Open Web Apps.

    var installApp = navigator.mozApps.install(manifestURL);
    // Successful install
    installApp.onsuccess = function(data) {
        console.log("Success, app installed!");
    // Install failed
    installApp.onerror = function() {
        console.log("Install failed\n:" +;

    Network Information API

    Used to get information about network connectivity.

    var connection = window.navigator.mozConnection,
        online = connection.bandwidth,
        metered = connection.metered;
    // online can return:
        0 when offline
        Infinity when the bandwidth isn't known
        Estimation of MB/s
    // metered implies if the connection is being metered,
       meaning limited in some way from the ISP

    Alarm API

    Making it possible to set alarms.

    request = navigator.mozAlarms.add(
        new Date("May 15, 2012 16:20:00"), 
            mydata: "my event"

    Web Activities

    Through Web Activities you can specify intents on what kind of action you want to do, or that your app can handle.

    var pick = new MozActivity({
         name: "pick",
         data: {
             type: ["image/png", "image/jpg", "image/jpeg"]}
    pick.onsuccess = function () {var img = document.createElement("img");
        img.src = window.URL.createObjectURL(this.result.blob);

    I’ve covered Web Activities in more detail in another blog post here.

    Push Notifications API

    A way for web sites to send messages to suers when aren’t on their web site. API is in a draft, not implemented yet. More can be read in the PushAPI documentation.

    function getPushURL() {
        var push = navigator.push ||
                    navigator.mozPush ||
        // Ask the user to allow notifications
        var request = push.requestURL(watoken, PbK);
        request.onsuccess = function () {
            var url = request.result.url;
            console.log('Push URL: ' + url);

    WebFM API

    Implementing a FM Radio in the web layer.

    var fmRadio = || navigator.mozFMRadio || navigator.mozFM;
    fmRadio.frequency = 106.7;


    Used to enable in-app payments, through JSON Web Token (JWT).

    var request = navigator.mozPay(JWTs);
    request.onsuccess = function () {
        // Money!


    Structured client-side storage with high performance search support. Covered in detail in Using IndexedDB and Storing images and files in IndexedDB.

    // Create/open database
    var request ="elephantFiles", 1),
         createObjectStore = function (dataBase) {
            console.log("Creating objectStore")
    request.onsuccess = function (event) {
        console.log("Success creating/accessing IndexedDB database");
        db = request.result;
        db.onerror = function (event) {
            console.log("Error creating/accessing IndexedDB database");
    // Needed for creating database/upgrading to a new version
    request.onupgradeneeded = function (event) {

    Ambient light sensor

    Detecting the level of ambient light, to be able to serve different versions depending on the environment the user currently is in.

    window.addEventListener("devicelight", function (event) {/* The level of the ambient light in lux

           A lux value for "dim" typically begin below 50,
           and a value for "bright" begin above 10000
    window.addEventListener("lightlevel", function (event) {
        // Possible values: "normal", "bright", "dim"

    Proximity sensor

    Getting an indication of how close the device is to another object.

    window.addEventListener("deviceproximity", function (event) {// Current device proximity, in centimeters



        // The maximum sensing distance the sensor is 

        // able to report, in centimeters



        // The minimum sensing distance the sensor is 

        // able to report, in centimeters



    Being able to show notifications to the user.

    var notification = navigator.mozNotification;
        "See this", 
        "This is a notification"
    // You could send an optional third parameter,
    // which would be the URL to an icon

    Privileged APIs

    These APIs can only be used in a privileged app.

    The complete list of privileged APIs is:

    • Device Storage API
    • Browser API
    • TCP Socket API
    • Contacts API
    • systemXHR

    Device Storage API

    Accessing files stored on the device.

    var storage = navigator.getDeviceStorage("videos"),

        cursor = storage.enumerate();


    cursor.onerror = function() {
        console.error("Error in DeviceStorage.enumerate()",;};

cursor.onsuccess = function() {if (!cursor.result)return;}
    var file = cursor.result;
        // If this isn't a video, skip it

        if (file.type.substring(0, 6) !== "video/") {

        // If it isn't playable, skip it

        var testplayer = document.createElement("video");if (!testplayer.canPlayType(file.type)) {

        // Show file

    Browser API

    Implementing your own web browser, completely with web technologies. Described more in BrowserAPI.

    iframe.addEventListener("mozbrowserlocationchange", function(e) {
    iframe.addEventListener("mozbrowsersecuritychange", function(e) {
        // "secure", "insecure", or "broken".  "broken" indicates mixed content.
    iframe.addEventListener("mozbrowsercontextmenu", function(e) {
        // Show context menu

    TCP Socket API

    A low-level TCP socket API, that will also include SSL support.

    var TCPSocket =
            useSSL: crypto,
            binaryType: "arraybuffer"

    Contacts API

    Accessing contacts on the device – adding, reading or modifying.

    var contact = new mozContact();
contact.init({name: "Tom"});

    var request =;
request.onsuccess = function() {

    request.onerror = function() {


    Making it possible to allow cross-domain XMLHTTPRequests. Specify in the manifest file in the permissions part that you will want to access it – "systemXHR":{} – and then just do the request.

    var xhr = new XMLHttpRequest();"GET", anyURL, true);

    Certified APIs

    The certified APIs are only available to the system itself/pre-installed apps. In the fcase of Firefox OS, this means only Mozilla.

    The complete list of certified APIs is:

    • WebTelephony
    • WebSMS
    • Idle API
    • Settings API
    • Power Management API
    • Mobile Connection API
    • WiFi Information API
    • WebBluetooth
    • Permissions API
    • Network Stats API
    • Camera API
    • Time/Clock API
    • Attention screen
    • Voicemail

    I’ll list a few of them here, in case you are interested in testing or contributing to Gaia, the UI of Firefox OS.


    For placing, receiving and dealing with calls.

    // Telephony object
    var tel = navigator.mozTelephony;
    // Check if the phone is muted (read/write property)
    // Check if the speaker is enabled (read/write property)
    // Place a call
    var call = tel.dial("123456789");
    // Events for that call
    call.onstatechange = function (event) {
            Possible values for state:
            "dialing", "ringing", "busy", "connecting", "connected", 
            "disconnecting", "disconnected", "incoming"
    // Above options as direct events
    call.onconnected = function () {
        // Call was connected
    call.ondisconnected = function () {
        // Call was disconnected
    // Receiving a call
    tel.onincoming = function (event) {
        var incomingCall =;
        // Get the number of the incoming call
        // Answer the call
    // Disconnect a call


    For sending and receiving SMS messages.

    // SMS object
    var sms = navigator.mozSMS;
    // Send a message
    sms.send("123456789", "Hello world!");
    // Receive a message
    sms.onreceived = function (event) {
        // Read message


    To be able to access certain APIs, in an Open Web App context, you need to specify permissions for the APIs you want to access in the manifest file.

    "permissions": {
        "contacts": {
            "description": "Required for autocompletion in the share screen",
            "access": "readcreate"
        "alarms": {
            "description": "Required to schedule notifications"

    For all APIs that are considered privileged or certified, this applies.

    For regular APIs, only geolocation and notification are affected (and for geolocation, in a regular web browser context, the user will be presented with a dialog to approve/decline).

    Additionally, the Camera API is restricted as a certified API at this time, but the long-term goal is to make it available to all apps. At the moment, if you want to access the camera, do it through Web Activities.

    Please take a look at the list of all APIs requiring permissions.

    Platform support

    As I’m sure you understand, a number of the WebAPIs are going through a process filled with progress, iterations and improvements. Some of the APIs above will work as described and intended, while others might not at this time.

    To be able to follow the current implementation state, please look at the list of APIs planned for the initial release of Firefox OS.

    The three columns imply:

    • D = Desktop support in Firefox
    • A = Android support in Firefox
    • B = Firefox OS support
    • Green = Implemented and enabled
    • Orange = Implemented, but needs to explicitly be turned on
    • Red = Not implemented
    • Blue = Only available as a Certified API
    • Grey = Not planned for this platform

    APIs planned for the future

    While not being worked on at the moment, I’d like to list APIs that are planned for the future. To show you our intent, and things we want to implement and support, and also to show in which direction we are moving.

    • Resource lock API
    • UDP Datagram Socket API
    • Peer to Peer API
    • WebNFC
    • WebUSB
    • HTTP-cache API
    • Calendar API
    • Spellcheck API
    • LogAPI
    • Keyboard/IME API
    • WebRTC
    • FileHandle API
    • Sync API

    Testing out these new APIs

    Feel free to just copy and paste code from this blog post to test the API(s) that you are interested in. I’ve also implemented support for some of them in the Firefox OS Boilerplate App, available in the webapp.js file (and I plan to add more in the future).

  3. Getting started with Open Web Apps – why and how

    We’ve been talking a lot about Open Web Apps, Firefox OS and more here lately, and I wanted to cover both how to get started, and, maybe more importantly, why.

    Why a web app?

    If we look at the climate for mobile development, it has usually come down to a choice where developers had to pick their platform and skill. Maybe do iOS and Objective-C, or Android and Java.

    The big challenge here, of course, is that if you want to deliver your content on several platforms, you’ve had a few choices:

    • Pick one platform, and ignore the others
    • Learn a number of programming languages
    • Have separate development teams for each platform

    For major organizations, having multiple teams has been something they could do, while many other have struggled with that. And naturally, many mobile apps are in addition to a company/service web site, just adding more and more on top of the things that need to be maintained, supported and developed.

    Therefore, for reasons like saving costs, simplicity of one development language and more, many developers have jumped on the options like PhoneGap, Titanium and more, thus developing with HTML5 and JavaScript, and then packaging it for various mobile operating systems.

    This has been a good and interesting approach, but probably far from optimal in most cases. We at Mozilla want you to be able to use your existing skills as a web developer, but instead of having you jump through hoops to make it work, we want the platforms to evolve and give you more possibilities and power.

    This is accomplished by giving you as a developer access to a large amount of WebAPIs, Web Activities and more, to make the web layer as powerful a platform as it deserves to be.

    The idea with Open Web Apps is not to make you choose a new platform or, worse, excluding others – on the contrary, it’s about giving you means to reuse your existing code and, if desired, make small additions to make it installable as an app.

    Should I build an app?

    While many other platforms have a strong interest in getting you tied into their platform, delivering to their app store etc, I would rather say that the first question you need to ask yourself is:

    Do I really need to make an app out of this?

    In some cases, yes, definitely! But in other cases, you need to be professional and come to the conclusion that it is probably not likely to add any extra value – a few use cases where you don’t need to do an app have been outlined in articles like No, I’m not going to download your bullshit app and Packaged HTML5 Apps: Are we emulating failure?.

    I don’t want to trick you into making an app just for the sake of it, or that you’ll do it just because others do. I would rather see you make a fair assessment, and if your app idea has something additional to offer to the end user and overall user experience, then you should consider doing an app.

    So, what could those cases be? A few of them might be:

    • You want to offer a richer experience than you could offer from a web page, e.g. accessing some WebAPIs specific to the platform/device
    • You need to store a lot of information in localStorage or IndexedDB
    • The user wants to have a real installation
    • The user wants a nice icon on their home screen/desktop for easier accessibility

    Types of Open Web Apps

    There are basically two types of Open Web Apps you can install:

    • Hosted apps
    • Packaged apps

    Hosted apps

    Hosted apps are apps running from a URL, but in an app context. This means that you need to be online to run it, and all resources (e.g. files, images etc) will reside on your server and where you will host it.

    One option to avoid the need for connectivity is make sure your app works offline. This is done by adding an appcache file with listings of assets to be made available offline, and then refer to it from your main page:

    HTML file

    <html manifest="manifest.appcache">

    Appcache file

    # Version 1.0
    / fallback.html

    For learning more about offline support and caveats, I strongly recommend looking into these resources:


    • You completely control the update process
    • Just run/reuse currently existing code


    • Requires connectivity (if offline support isn’t implemented)
    • Doesn’t have as much access to APIs as a packaged app

    Packaged apps

    Packaged apps are where you put all your assets in a ZIP file, and then offer that as a complete package to be installed. This makes these files available at all times, and it also gives you elevated privileges – i.e. you can access more APIs – since all the code can be security cleared before install.


    • Available offline by default
    • More API access


    • More difficult to maintain
    • Update process for getting new versions out

    At the end of the day, you need to evaluate your needs, workflow, APIs you need to work with and more to make a good decision whether you want to do a hosted or a packaged app.

    Getting started with Open Web Apps

    After all that talk, what do you actually need to build an Open Web App? Not much, as it turns out. We’ve documented it on MDN in Getting started with making apps but I’d like to give you a quick run-down here as well.


    Basically, all you need to do is take an existing web site/service you have and add a manifest file. Voilà, that’s it! Yes, really. And to get it installed, of course.

    The manifest file

    A manifest file describes your app, with things like name, icons, developer and such, but also localization support, path to launch, permission request for certain APIs and more. All manifest fields are available in App manifest on MDN.

    A simple manifest could look like this:

        "version": "1",
        "name": "Firefox OS Boilerplate App",
        "launch_path": "/Firefox-OS-Boilerplate-App/index.html",
        "description": "Boilerplate Firefox OS app with example use cases to get started",
        "icons": {
            "16": "/Firefox-OS-Boilerplate-App/images/logo16.png",
            "32": "/Firefox-OS-Boilerplate-App/images/logo32.png",
            "48": "/Firefox-OS-Boilerplate-App/images/logo48.png",
            "64": "/Firefox-OS-Boilerplate-App/images/logo64.png",
            "128": "/Firefox-OS-Boilerplate-App/images/logo128.png"
        "developer": {
            "name": "Robert Nyman",
            "url": ""
        "installs_allowed_from": ["*"],
        "default_locale": "en"

    Save this file with a .webapp extension, for instance, manifest.webapp. One very important thing to note here is that this file needs to be served with the Content-type: application/x-web-app-manifest+json.

    This is something you need to set up on your server, e.g. through an .htaccess file in Apache:

    AddType application/x-web-app-manifest+json .webapp

    Once you have your manifest, make sure to validate your app to see that it has the correct format.


    Installing the app

    Now that your manifest is in order and served with the right Content-type, let’s offer a way to install it. In your web page, you can add an install button that calls this code:

    var installApp = navigator.mozApps.install(manifestURL);
    // Successful install
    installApp.onsuccess = function(data) {
        console.log("Success, app installed!");
    // Install failed
    installApp.onerror = function() {
        console.log("Install failed\n\n:" +;

    Make sure that the URL to the manifest is absolute – a simple way to do this is to extract the URL from the current page with the install button, and to have the manifest file in the same location:

    var manifestURL = location.href.substring(0, location.href.lastIndexOf("/")) + "/manifest.webapp";

    Optionally, you can also provide a second parameter to the install method, receipts, which is a JSON object. More on that in the documentation for the install method.

    Installing a packaged app

    The above solution, with a manifest file and an install call, works well with hosted apps. With Packaged apps, you need to go through some extra steps:

    ZIP all app content

    Make sure to ZIP all the files (not the containing folder), including the regular manifest file. The manifest file has to be named manifest.webapp

    Create a mini manifest

    Create another manifest file, for instance named package.webapp, and make sure the package_path is absolute to where the ZIP file is located.

    Also, developer name and info has to match between mini manifest and the regular one in the ZIP file.

        "name": "Firefox OS Boilerplate App",
        "package_path" : "http://localhost/Firefox-OS-Boilerplate-App/",
        "version": "1",
        "developer": {
            "name": "Robert Nyman",
            "url": ""

    Installing a package

    Instead of using the regular install method, you now call the installPackage method, which points to the mini manifest, which in turn points to the ZIP file/package:

    var manifestURL = location.href.substring(0, location.href.lastIndexOf("/")) + "/package.webapp";
    var installApp = navigator.mozApps.installPackage(manifestURL);

    Turning on Developer Mode

    For this to work in the Firefox OS Simulator, you need to turn on the Developer Mode:

    Settings > Device Information > More Information > 
    Developer > Developer mode

    Note: this is a work in progress, and the availability of this option might vary, and not be available in your version of the Simulator or an actual Firefox OS device.

    All the releases and pre-releases of the Firefox OS Simulator are available on Mozilla’s FTP.

    Specifying permissions

    If you plan on using some APIs that only Packaged apps have access to, you need to add a couple of things to your regular (manifest.webapp) file:

    • Add type property (e.g. “type” : “privileged”)
    • Specify permission access
    "permissions": {
        "contacts": {
            "description": "Required for autocompletion in the share screen",
            "access": "readcreate"
        "alarms": {
            "description": "Required to schedule notifications"

    There is also an interesting option, in the form of packaged-app-server, which offers zipping the files as a package when they get requested at run time.

    $ cd ~/myapp
    $ python ~/

    An example app

    As an example app – if you want to check out something easy to dissect, tweak and get started with – feel free to test the Firefox OS Boilerplate App. It supports:

    • An install button, offering you to install it as a hosted app
    • Web Activities – lots of examples and use cases
    • WebAPIs in action
    • Offline support (disabled by default)
    • Packaged apps – install your app as a ZIP file


    Which platforms are supported?

    Let’s look at where we are right now with Open Web Apps. They are supported in:

    Firefox OS

    You can install an Open Web App in Firefox OS (the Simulator or on a device) and most WebAPIs and Web Activities will work.


    Firefox on Android

    In Firefox on Android you can install an app, and have it installed on your home screen with the correct icon. However, it doesn’t support the WebAPIs and Web Activities.


    Nightly/Aurora versions on Desktop

    You can install and run a stand-alone app in Firefox Nightly/Firefox Aurora, but it doesn’t have access to many WebAPIs and no Web Activities.


    The initial strong focus at the moment is support on mobile, but the hope and goal is that Open Web Apps will work on all platforms and devices, by adding support to needed APIs and more.


    When it comes to Open Web Apps, you can use or install them from anywhere. Completely up to you.However, if you are interested in being listed, hosted and much more, I recommend taking a look at the Firefox Marketplace.


    You can also visit the Developer Hub for a lot more information about app development.



    Open Web Apps aren’t here to change your developing ways – they are here to give you the option to install your existing web solutions as apps, access device-specific APIs and more.

    Don’t reinvent the wheel – just make it a bit more powerful!

  4. Hello Chrome, it’s Firefox calling!

    Mozilla is excited to announce that we’ve achieved a major milestone in WebRTC development: WebRTC RTCPeerConnection interoperability between Firefox and Chrome. This effort was made possible because of the close collaboration between the open Web community and engineers from both Mozilla and Google.

    RTCPeerConnection (also known simply as PeerConnection or PC) interoperability means that developers can now create Firefox WebRTC applications that make direct audio/video calls to Chrome WebRTC applications without having to install a third-party plugin. Because the functionality is now baked into the browser, users can avoid problems with first-time installs and buggy plugins, and developers can deploy their apps much more easily and universally.

    To help celebrate this momentous milestone, we thought it would be fun to call up our friends at Google to discuss it with them. Check out this Firefox-Chrome demonstration call between Mozilla’s Chief Innovation Officer, Todd Simpson, and Google’s Director of Product Management, Hugh Finnan, and read what Google had to say about this momentous occasion in their blog post.

    This milestone builds on an earlier demo we showed late last year of WebRTC integrated with Social API. There we demonstrated an industry first with our implementation of DataChannels, a powerful component of WebRTC that can combined with an audio/video chat to allow users to share almost anything on their computer or device. Send vacation photos, memorable videos, links news stories etc., simply by dragging the item into your video chat window. Look out for more on this to come.

    The purpose of WebRTC, an open standard being defined jointly at the W3C and IETF standards organizations, is to provide a common platform for all user devices to communicate and share audio, video and data in real-time. This is a first step toward that vision of interoperability and true, open, real-time communication on the web.

    Posted by:
    Serge Lachapelle, Chrome Product Manager and Maire Reavy, Firefox Media Product Lead

    Start Developing Using RTCPeerConnection in Firefox

    For JavaScript developers who haven’t tried RTCPeerConnection in Firefox yet (since it is a brand new feature for us), you can try this out using the most recent Firefox Nightly by setting the media.peerconnection.enabled pref to “true” (browse to about:config and search for the media.peerconnection.enabled pref in the list of prefs). Here is a snippet of code from a sample app that shows off how to initiate, accept, and end a WebRTC call in Firefox using RTCPeerConnection:

    function initiateCall(user) {
      document.getElementById("main").style.display = "none";
      document.getElementById("call").style.display = "block";
      // Here's where you ask user permission to access the camera and microphone streams
      navigator.mozGetUserMedia({video:true, audio:true}, function(stream) {
        document.getElementById("localvideo").mozSrcObject = stream;
        document.getElementById("localvideo").muted = true;
        // Here's where you set up a Firefox PeerConnection
        var pc = new mozRTCPeerConnection();
        pc.onaddstream = function(obj) {
          log("Got onaddstream of type " + obj.type);
          document.getElementById("remotevideo").mozSrcObject =;
          document.getElementById("dialing").style.display = "none";
          document.getElementById("hangup").style.display = "block";
        pc.createOffer(function(offer) {
          log("Created offer" + JSON.stringify(offer));
          pc.setLocalDescription(offer, function() {
            // Send offer to remote end.
            log("setLocalDescription, sending to remote");
            peerc = pc;
              "offer", {
                to: user,
                from: document.getElementById("user").innerHTML,
                offer: JSON.stringify(offer)
              function() { console.log("Offer sent!"); }
          }, error);
        }, error);
      }, error);
    function acceptCall(offer) {
      log("Incoming call with offer " + offer);
      document.getElementById("main").style.display = "none";
      document.getElementById("call").style.display = "block";
      // Here's where you ask user permission to access the camera and microphone streams
      navigator.mozGetUserMedia({video:true, audio:true}, function(stream) {
        document.getElementById("localvideo").mozSrcObject = stream;
        document.getElementById("localvideo").muted = true;
        // Here's where you set up a Firefox PeerConnection
        var pc = new mozRTCPeerConnection();
        pc.onaddstream = function(obj) {
          document.getElementById("remotevideo").mozSrcObject =;
          document.getElementById("dialing").style.display = "none";
          document.getElementById("hangup").style.display = "block";
        pc.setRemoteDescription(JSON.parse(offer.offer), function() {
          log("setRemoteDescription, creating answer");
          pc.createAnswer(function(answer) {
            pc.setLocalDescription(answer, function() {
              // Send answer to remote end.
              log("created Answer and setLocalDescription " + JSON.stringify(answer));
              peerc = pc;
                "answer", {
                  to: offer.from,
                  answer: JSON.stringify(answer)
                function() { console.log("Answer sent!"); }
            }, error);
          }, error);
        }, error);
      }, error);
    function endCall() {
      log("Ending call");
      document.getElementById("call").style.display = "none";
      document.getElementById("main").style.display = "block";
      document.getElementById("localvideo").mozSrcObject = null;
      document.getElementById("remotevideo").mozSrcObject = null;
      peerc = null;

    You’ll notice that Firefox still prefixes the RTCPeerConnection API call as mozRTCPeerConnection because the standards committee is not yet done defining it. Chrome prefixes it as webkitRTCPeerConnection. Once the standards committee finishes its work, we will remove the prefixes and use the same API, but in the meantime, you’ll want to support both prefixes so that your app works in both browsers.

    Trying Interop Yourself

    For those eager to give interop a try, here are instructions and information about “trying this at home”.

    This is Firefox’s and Chrome’s first version of PeerConnection interoperability. As with most early releases, there are still bugs to fix, and interop isn’t supported yet in every network environment. But this is a major step forward for this new web feature and for the Web itself. We thank the standards groups and every contributor to the WebRTC community. While there’s more work to do, we hope you’ll agree that the Web is about to get a lot more awesome.

  5. Introducing the Firefox OS Boilerplate App

    When coming to a new platform or context, it’s always good to get a peek at some code and examples how to make things work. With Firefox OS and app development, it’s just the web with a few additions.

    Before here at Mozilla Hacks, we’ve covered a few ways to get started with building apps for Firefox OS:

    My experiences

    Lately I’ve been fortunate enough to give and take part in a number of workshops around Firefox OS, to see developers trying to build things for it, port their existing web apps and much more.

    This has been a fantastic learning lesson for me, and it’s been crucial to see where people might need pointers, help or examples!

    The Firefox OS Boilerplate App

    This led to me creating the Firefox OS Boilerplate App. As the name implies, it’s there to provide you with the most basic features to get started with building an app from scratch, or tools to port your existing web app.

    The idea is also to avoid any dependency on external libraries or resources, but rather be self-contained.

    It contains:

    • An install button, offering you to install it as a hosted app
    • Web Activities – lots of examples and use cases
    • WebAPIs in action
    • Offline support (disabled by default)
    • Packaged apps – install your app as a ZIP file

    It’s available on GitHub:

    How to use it

    The easiest way to get started, installing it and testing the various features, is to navigate to the Firefox OS Boilerplate App in the web browser on a Firefox OS device or in the Firefox OS Simulator.

    Alternatively, install it in the Firefox OS Simulator Dashboard by providing either of these URLs:

    Running it locally

    Once you’re ready to get started developing, download the code and run it on a web server, or point out your local version of the Firefox OS Boilerplate App in the Firefox OS Simulator.

    Note: make sure that the paths in the manifest file are valid on your localhost – bear in mind that these paths are relative to the root of the web site they are being served at.

    Also make sure to configure your server to send the manifest file with the right Content-type: application/x-web-app-manifest+json.

    This is, for instance, easy to set up in an .htaccess file in Apache:

    AddType application/x-web-app-manifest+json .webapp

    Offline support

    I’ve provided an .appcache file for enabling offline support (it’s disabled by default).

    To enable offline capabilities, just add this to the index.html file:

    <html manifest="manifest.appcache">

    Please make sure to do your homework before enabling offline support, to avoid possible initial issues:

    Remember that the .appcache file has to be served as a text/cache-manifest file:

    AddType text/cache-manifest .appcache

    Packaged apps

    When you develop web apps, by default they are being delivered from a server, thus needing online connectivity or offline support to be enabled, to work as expected.

    You do have another option, though, which is packaged apps. Basically, what this is, is putting all the files of your app into a ZIP file, making them available directly on the device itself.

    Packaged apps can also request an elevated access to certain WebAPIs in Firefox OS that aren’t available to hosted apps (we’ll go more into the differences in a later post here on Mozilla Hacks).

    There are a couple of files included in the Firefox Boilerplate OS App to help you get started, if you are interested in this.

    To create and install a packaged app, you need to go through a few steps:

    • ZIP all app content (not containing folder), including regular manifest
    • Create a mini manifest (the package.manifest file) and make sure the “package_path” is absolute to where the ZIP is located
    • Developer name and info has to match between mini manifest and the regular one in the ZIP file
    • Have an installPackage call in JavaScript pointing to the mini manifest (instead of the regular install one) – this is shown in comments in the base.js file
    • Turn on Developer Mode in the Firefox OS Simulator (Settings > Device Information > More Information > Developer > Developer mode)
    • Add type property (e.g. "type" : "privileged") in the manifest if you want access to certain APIs

    Work in progress

    The Firefox OS Boilerplate App is a work in progress, meaning that it’s likely to change over time. I believe, however, that it gives you a good head start and look into what’s possible with web apps in Firefox OS.

    Hope you like it, and please let me know what you think!

  6. Introducing Web Activities

    One of the more powerful things lately for apps on various mobile phones have been intents. Register your app for handling certain types of actions, or specify in your app what kind of support you are looking for, for the thing you are trying to do.

    This is especially important in the case of Firefox OS. No matter how good your web app is to begin with, what really takes it to the next level is interaction with other apps and activities on the device.

    This is where Web Activities come into place.

    It is basically one of the new WebAPIs we’ve been working on to make the web even more powerful as a platform, and the idea is to have a simple API to both specify intent with your activity, but also declare that your app will be able to handle actions from other apps.

    If you haven’t looked into web apps yet, by the way, the best thing is probably to read Getting started with making apps. In their simplest form, a web app is just HTML5, CSS and JavaScript, with an app manifest added.

    Getting started with Web Activities

    There are a few ways you can work with Web Activities:

    • Call an activity and get presented with apps that can handle that
    • Register your activity support for your web app in the manifest file
    • Register activity support on-the-fly
    • Attach a handler to your app for when that activity occurs

    Calling an activity

    Let’s say you have a button in your app, and you want to be able to get a picture – either from the Gallery, Camera or any other app in Firefox OS that supports that activity. You can then call the pick activity, like this:

    var pick = new MozActivity({
       name: "pick",
       data: {
           type: ["image/png", "image/jpg", "image/jpeg"]}

    In this example, we specify the name pick and the data to be PNG or JPEG images. You will then be presented with a menu of available activities in Firefox OS:

    As a user, you choose the app you want to pick an image from – or take a picture with the Camera – and once you’ve done so, the result will be posted back to the requesting app (Note: it is chosen from within the app handling the activity what, and if, something will be returned).

    Handling the response

    For most WebAPIs, including Web Activities, you will have onsuccess and onerror event handlers. In the case of an image/file you will get a blob back. You can then represent that returned image visually directly in your app:

    pick.onsuccess = function () {// Create image and set the returned blob as the src
        var img = document.createElement("img");
        img.src = window.URL.createObjectURL(this.result.blob);
        // Present that image in your app
        var imagePresenter = document.querySelector("#image-presenter");
    pick.onerror = function () {// If an error occurred or the user canceled the activity
        alert("Can't view the image!");

    Register your app for an activity

    As mentioned above, you can also set your app as a handler for certain activities. There are two ways to do that:

    Through the manifest file – declaration registration

        "name": "My App",
        "description": "Doing stuff",
        "activities": {
           "view": {
                "filters": {
                    "type": "url",
                    "url": {
                        "required": true, 

    Register an activity handler – dynamic registration

    var register = navigator.mozRegisterActivityHandler({
        name: "view", 
        disposition: "inline", 
        filters: {
            type: "image/png"

register.onerror = function () {
        console.log("Failed to register activity");}

    and then handle the activity:

    navigator.mozSetMessageHandler("activity", function (a) {var img = getImageObject();

        img.src = a.source.url;/*
          Call a.postResult() or a.postError() if 

          the activity should return a value


    Available activities

    The available activities to choose from at this time are:

    • configure
    • costcontrol/balance
    • costcontrol/data_usage
    • costcontrol/telephony
    • dial
    • new (e.g. type: “websms/sms”, “webcontacts/contact”)
    • open
    • pick (e.g. type: “image/png”)
    • record
    • save-bookmark
    • share
    • test
    • view

    A few examples:


    var call = new MozActivity({
        name: "dial",
        data: {
            number: "+46777888999"

    New SMS

    var sms = new MozActivity({
        name: "new",
        data: {
            type: "websms/sms",
            number: "+46777888999"

    New Contact

    var newContact = new MozActivity({
        name: "new",
        data: {
            type: "webcontacts/contact",
            params: { // Will possibly move to be direct properties under "data"
                giveName: "Robert",
                familyName: "Nyman",
                tel: "+44789",
                email: "",
                address: "Sweden",
                note: "This is a note",
                company: "Mozilla"

    View URL

    var openURL = new MozActivity({
        name: "view",
        data: {
            type: "url", // Possibly text/html in future versions
            url: ""

    Save bookmark

    var savingBookmark = new MozActivity({
        name: "save-bookmark",
        data: {
            type: "url",
            url: "",
            name: "Robert's talk",
            icon: ""}

    Try it out now!

    You can try this out right now, by putting together a web app and calling web activities. You can then test it in the Firefox OS Simulator!

    Work in progress

    Web Activities are a work in progress, and the activity names, data types etc are subject to change. However, currently most of the above works right now (with the exception for mozRegisterActivityHandler and mozSetMessageHandler, that haven’t been implemented yet).

    I hope you share my excitement with all the possibilities Web Activities are offering us, and can probably think of a number of use cases, making your app much more powerful through interaction with other apps!

  7. Announcing the Firefox OS Developer Preview Phone!

    Firefox OS is a new mobile operating system built entirely using open web standards. In Firefox OS, HTML5 apps can do “phone things” – they can make the phone vibrate, make a phone call or send a text message.

    This week we are announcing our new Firefox OS developer preview phones because we believe that developers will help bring the power of the web to mobile. These developer phones are being developed by Geeksphone in partnership with Telefonica. We are looking to make them available in February.

    Updated: Added the two phone models and their specifications below.

    The Keon model

    • CPU Qualcomm Snapdragon S1 1Ghz
    • UMTS 2100/1900/900 (3G HSPA)
    • GSM 850/900/1800/1900 (2G EDGE)
    • Screen 3.5″ HVGA Multitouch
    • 3 MP Camera
    • 4GB ROM, 512 MB RAM
    • MicroSD, Wifi N, Light and proximity Sensor, G-Sensor, GPS, MicroUSB
    • 1580 mAh battery
    • Over the air updates
    • Unlocked, add your own SIM card

    The Peak model

    • CPU Qualcomm Snapdragon S4 1.2Ghz x2.
    • UMTS 2100/1900/900 (3G HSPA).
    • GSM 850/900/1800/1900 (2G EDGE).
    • Screen 4.3″ qHD IPS Multitouch.
    • Camera 8 MP (back) + 2 MP (front).
    • 4 GB (ROM) and 512 (RAM).
    • MicroSD, Wifi N, Light and proximity Sensor, G-Sensor, GPS, MicroUSB, Flash (camera).
    • Battery 1800 mAh.

    Developer preview phones will help make the mobile web more accessible to more people. Developers are critical to the web and to Mozilla’s mission to make the web accessible to everyone. Hundreds of millions of people worldwide use Firefox to discover, experience and connect to the Web. A web based on open standards and open technologies. We couldn’t have done this without web developers. Now we are working on bringing the power of the web to mobile, through Firefox OS, along with all the power of open standards and an open community, and once again, we’d like to invite web developers to join us.

    Firefox OS App Days

    We’d also like to invite you to participate in Firefox OS App Days, a worldwide set of 20+ hack days happening this week. At each App Day event, you’ll have the opportunity to learn, hack, share and celebrate Firefox OS, Mozilla’s open source operating system for the mobile web. Technologists and developers from the Mozilla community will present tools and technology built to extend and support the Web platform, including mobile Web APIs to access device hardware features such as the accelerometer.

    If you can’t get to an App Day, you can still sign up for Firefox Apps & Hacks, our email newsletter for Firefox OS app developers. We’ll let you know as soon as the phones are available from Geeksphone.

    Try out Firefox OS

    If you’re a developer interested in web technologies and mobile, now is the time to try out Firefox OS. Come learn how easy it is to create an app based on HTML that has access to all of the APIs on your device. Using the same web technologies you are used to, you can make your app give alerts by vibrating the phone, take a picture with the camera or more. Play with it, experiment, try out your website, create an app.

    You can create an app for Firefox OS by making some small changes to your existing website.

    You can try out Firefox OS and your app (if you created one) in several different ways:

    • Install Marketplace for Android on your Android phone. (Your friends and family can also do this – it’s as easy as installing an app, so they can try out your app too.)
    • Use the browser-based Firefox OS Simulator to view and test your mobile app on the desktop.
    • Install Firefox OS on your own hardware.
    • Purchase a Firefox OS Developer Preview device! Working with Geeksphone and Telefonica we’ve created a Firefox OS Developer Preview phone especially for developers. Be the first to show off Firefox OS and your app! First phones will be available in February.

    Why develop an app for Firefox OS?

    • Keep the web open. Support the open web and help make sure the power of the web is available to everyone – even on mobile devices.
    • Simplicity. Develop on a single technology stack (HTML5/CSS/JavaScript/new WebAPIs) and deliver across the web and devices.
    • Freedom. You’re not locked in to a vendor-controlled ecosystem. You can distribute your app through the Firefox Marketplace, your own website, or any other store based on Mozilla’s open app store technology.

    If you have any more questions, please read the Firefox OS FAQ.

    Join us in bringing the web to mobile!

  8. Building A Node.JS Server That Won’t Melt – A Node.JS Holiday Season, part 5

    This is episode 5, out of a total 12, in the A Node.JS Holiday Season series from Mozilla’s Identity team. For this post, we bring the discussion back to scaling Node.JS applications.

    How can you build a Node.JS application that keeps running, even under impossible load?

    This post presents a technique and a library that implements that technique, all which is distilled into the following five lines of code:

    var toobusy = require('toobusy');
    app.use(function(req, res, next) {
      if (toobusy()) res.send(503, "I'm busy right now, sorry.");
      else next();

    Why Bother?

    If your application is important to people, then it’s worth spending a moment thinking about disaster scenarios. These are the good kind of disasters where your project becomes the apple of social media’s eye and you go from ten thousand users a day to a million. With a bit of preparation you can build a service that can persevere during traffic bursts that exceed capacity by orders of magnitude. If you forego this preparation, then your service will become completely unusable at precisely the wrong time – when everyone is watching.

    Another great reason to think about legitimate bursts traffic, is malicious bursts of traffic: the first step to mitigating DoS attacks is building servers that don’t melt.

    Your Server Under Load

    To illustrate how applications with no considerations for burst behave, I built an application server with an HTTP API that consumes 5ms of processor time spread over five asynchronous function calls. By design, a single instance of this server is capable of handling 200 requests per second.

    This roughly approximates a typical request handler that perhaps does some logging, interacts with the database, renders a template, and streams out the result. What follows is a graph of server latency and TCP errors as we linearly increase connection attempts:

    Analysis of the data from this run tells a clear story:

    This server is not responsive: At 6x maximum capacity (1200 requests/second) the server is hobbled with 40 seconds of average request latency.

    These failures suck: With over 80% TCP failures and high latency, users will see a confusing failure after up to a minute of waiting.

    Failing Gracefully

    Next, I instrumented the same application with the code from the beginning of this post. This code causes the server to detect when load exceeds capacity and preemptively refuse requests. The following graph depicts the performance of this version of the server as we linearly increase connections attempts:

    Your server with limits

    One thing not depicted on the graph is the volume of 503 (server too busy) responses returned during this run which steadily increases proportional to connection attempts. So what do we learn from this graph and the underlying data?

    Preemptive limiting adds robustness: Under load that exceeds capacity by an order of magnitude the application continues to behave reasonably.

    Success and Failure is fast: Average response time stays for the most part under 10 seconds.

    These failures don’t suck: With preemptive limiting we effectively convert slow clumsy failures (TCP timeouts), into fast deliberate failures (immediate 503 responses).

    To be clear, building a server that returns HTTP 503 responses (“server is too busy”), requires that your interface render a reasonable message to the user. Typically this is a pretty simple task, and should be familiar as it’s done by many popular sites.

    How To Use It

    node-toobusy is available on npm and github. After installation (npm install toobusy), simply require it:

    var toobusy = require('toobusy');

    At the moment the library is included, it will begin actively monitoring the process, and will determine when the process is “too busy”. You can then check if the process is toobusy at key points in your application:

    // The absolute first piece of middle-ware we would register, to block requests
    // before we spend any time on them.
    app.use(function(req, res, next) {
      // check if we're toobusy() - note, this call is extremely fast, and returns
      // state that is cached at a fixed interval
      if (toobusy()) res.send(503, "I'm busy right now, sorry.");
      else next();

    This application of node-toobusy gives you a basic level of robustness at load, which you can tune and customize to fit the design of your application.

    How It Works

    How do we reliably determine if a Node application is too busy?

    This turns out to be more interesting that you might expect, especially when you consider that node-toobusy attempts to work for any node application out of the box. In order to understand the approach taken, let’s review some approaches that don’t work:

    Looking at processor usage for the current process: We could use a number like that which you see in top – the percentage of time that the node process has been executing on the processor. Once we had a way of determining this, we could say usage above 90% is “too busy”. This approach fails when you have multiple processes on the machine that are consuming resources and there is not a full single processor available for your node application. In this scenario, your application would never register as “too busy” and would fail terribly – in the way explained above.

    Combining system load with current usage: To resolve this issue we could retrieve current system load as well and consider that in our “too busy” determination. We could take the system load and consider the number of available processing cores, and then determine what percentage of a processor is available for our node app! Very quickly this approach becomes complex, requires system specific extensions, and fails to take into account things like process priority.

    What we want is a simpler solution that Just Works. This solution should conclude that the node.js process is too busy when it is unable to serve requests in a timely fashion – a criteria that is meaningful regardless of the details of other processes running on the server.

    The approach taken by node-toobusy is to measure event loop lag. Recall that Node.JS is at its core an event loop. Work to be done is enqueued, and on each loop iteration is processed. As a node.js process becomes over-loaded, the queue grows and there is more work to be done than can be done. The degree to which a node.js process is overloaded can be understood by determining how long it takes a tiny bit of work to get through the event queue. The node-toobusy library provides libuv with a callback that should be invoked every 500 milliseconds. Subtracting 500ms from the actual time elapsed between invocations gives us a simple measure of event loop lag.

    In short, node-toobusy measures event loop lag to determine how busy the host process is, which is a simple and robust technique that works regardless of whatever else is running on the host machine.

    Current State

    node-toobusy is very new library that makes it easy to build servers that don’t melt by measuring event loop lag: attempting to solve the general problem of determining if a node.js application is too busy. All of the test servers described here as well as the load generation tools used in the post are available on github.

    At Mozilla we’re currently evaluating applying this approach to the Persona service, and expect to refine it as we learn. I look forward to your feedback – as a comment on this post, on the identity mailing list, or in github issues.

    Previous articles in the series

    This was part five in a series with a total of 12 posts about Node.js. The previous ones are:

  9. Koalas to the Max – a case study

    One day I was browsing reddit when I came across this peculiar link posted on it:

    The game was addictive and I loved it but I found several design elements flawed. Why did it start with four circles and not one? Why was the color split so jarring? Why was it written in flash? (What is this, 2010?) Most importantly, it was missing a golden opportunity to split into dots that form an image instead of just doing random colors.

    Creating the project

    This seemed like a fun project, and I reimplemented it (with my design tweaks) using D3 to render with SVG.

    The main idea was to have the dots split into the pixels of an image, with each bigger dot having the average color of the four dots contained inside of it recursively, and allow the code to work on any web-based image.
    The code sat in my ‘Projects’ folder for some time; Valentines day was around the corner and I thought it could be a cute gift. I bought the domain name, found a cute picture, and thus “ (KttM)” was born.


    While the user-facing part of KttM has changed little since its inception, the implementation has been revisited several times to incorporate bug fixes, improve performance, and bring support to a wider range of devices.

    Notable excerpts are presented below and the full code can be found on GitHub.

    Load the image

    If the image is hosted on (same) domain then loading it is as simple as calling new Image()

    var img = new Image();
    img.onload = function() {
     // Awesome rendering code omitted
    img.src = the_image_source;

    One of the core design goals for KttM was to let people use their own images as the revealed image. Thus, when the image is on an arbitrary domain, it needs to be given special consideration. Given the same origin restrictions, there needs to be a image proxy that could channel the image from the arbitrary domain or send the image data as a JSONP call.

    Originally I used a library called $.getImageData but I had to switch to a self hosted solution after KttM went viral and brought the $.getImageData App Engine account to its limits.

    Extract the pixel data

    Once the image loads, it needs to be resized to the dimensions of the finest layer of circles (128 x 128) and its pixel data can be extracted with the help of an offscreen HTML5 canvas element.

    koala.loadImage = function(imageData) {
     // Create a canvas for image data resizing and extraction
     var canvas = document.createElement('canvas').getContext('2d');
     // Draw the image into the corner, resizing it to dim x dim
     canvas.drawImage(imageData, 0, 0, dim, dim);
     // Extract the pixel data from the same area of canvas
     // Note: This call will throw a security exception if imageData
     // was loaded from a different domain than the script.
     return canvas.getImageData(0, 0, dim, dim).data;

    dim is the number of smallest circles that will appear on a side. 128 seemed to produce nice results but really any power of 2 could be used. Each circle on the finest level corresponds to one pixel of the resized image.

    Build the split tree

    Resizing the image returns the data needed to render the finest layer of the pixelization. Every successive layer is formed by grouping neighboring clusters of four dots together and averaging their color. The entire structure is stored as a (quaternary) tree so that when a circle splits it has easy access to the dots from which it was formed. During construction each subsequent layer of the tree is stored in an efficient 2D array.

    // Got the data now build the tree
    var finestLayer = array2d(dim, dim);
    var size = minSize;
    // Start off by populating the base (leaf) layer
    var xi, yi, t = 0, color;
    for (yi = 0; yi < dim; yi++) {
     for (xi = 0; xi < dim; xi++) {
       color = [colorData[t], colorData[t+1], colorData[t+2]];
       finestLayer(xi, yi, new Circle(vis, xi, yi, size, color));
       t += 4;

    Start by going through the color data extracted in from the image and creating the finest circles.

    // Build up successive nodes by grouping
    var layer, prevLayer = finestLayer;
    var c1, c2, c3, c4, currentLayer = 0;
    while (size < maxSize) {
     dim /= 2;
     size = size * 2;
     layer = array2d(dim, dim);
     for (yi = 0; yi < dim; yi++) {
       for (xi = 0; xi < dim; xi++) {
         c1 = prevLayer(2 * xi    , 2 * yi    );
         c2 = prevLayer(2 * xi + 1, 2 * yi    );
         c3 = prevLayer(2 * xi    , 2 * yi + 1);
         c4 = prevLayer(2 * xi + 1, 2 * yi + 1);
         color = avgColor(c1.color, c2.color, c3.color, c4.color);
         c1.parent = c2.parent = c3.parent = c4.parent = layer(xi, yi,
           new Circle(vis, xi, yi, size, color, [c1, c2, c3, c4], currentLayer, onSplit)
     splitableByLayer.push(dim * dim);
     splitableTotal += dim * dim;
     prevLayer = layer;

    After the finest circles have been created, the subsequent circles are each built by merging four dots and doubling the radius of the resulting dot.

    Render the circles

    Once the split tree is built, the initial circle is added to the page.

    // Create the initial circle
    Circle.addToVis(vis, [layer(0, 0)], true);

    This employs the Circle.addToVis function that is used whenever the circle is split. The second argument is the array of circles to be added to the page.

    Circle.addToVis = function(vis, circles, init) {
     var circle = vis.selectAll('.nope').data(circles)
     if (init) {
       // Setup the initial state of the initial circle
       circle = circle
         .attr('cx',   function(d) { return d.x; })
         .attr('cy',   function(d) { return d.y; })
         .attr('r', 4)
         .attr('fill', '#ffffff')
     } else {
       // Setup the initial state of the opened circles
       circle = circle
         .attr('cx',   function(d) { return d.parent.x; })
         .attr('cy',   function(d) { return d.parent.y; })
         .attr('r',    function(d) { return d.parent.size / 2; })
         .attr('fill', function(d) { return String(d.parent.rgb); })
         .attr('fill-opacity', 0.68)
     // Transition the to the respective final state
       .attr('cx',   function(d) { return d.x; })
       .attr('cy',   function(d) { return d.y; })
       .attr('r',    function(d) { return d.size / 2; })
       .attr('fill', function(d) { return String(d.rgb); })
       .attr('fill-opacity', 1)
       .each('end',  function(d) { d.node = this; });

    Here the D3 magic happens. The circles in circles are added (.append('circle')) to the SVG container and animated to their position. The initial circle is given special treatment as it fades in from the center of the page while the others slide over from the position of their “parent” circle.

    In typical D3 fashion circle ends up being a selection of all the circles that were added. The .attr calls are applied to all of the elements in the selection. When a function is passed in it shows how to map the split tree node onto an SVG element.

    .attr('cx', function(d) { return d.parent.x; }) would set the X coordinate of the center of the circle to the X position of the parent.

    The attributes are set to their initial state then a transition is started with .transition() and then the attributes are set to their final state; D3 takes care of the animation.

    Detect mouse (and touch) over

    The circles need to split when the user moves the mouse (or finger) over them; to be done efficiently the regular structure of the layout can be taken advantage of.

    The described algorithm vastly outperforms native “onmouseover” event handlers.

    // Handle mouse events
    var prevMousePosition = null;
    function onMouseMove() {
     var mousePosition = d3.mouse(vis.node());
     // Do nothing if the mouse point is not valid
     if (isNaN(mousePosition[0])) {
       prevMousePosition = null;
     if (prevMousePosition) {
       findAndSplit(prevMousePosition, mousePosition);
     prevMousePosition = mousePosition;
    // Initialize interaction
     .on('mousemove.koala', onMouseMove)

    Firstly a body wide mousemove event handler is registered. The event handler keeps track of the previous mouse position and calls on the findAndSplit function passing it the line segments traveled by the user’s mouse.

    function findAndSplit(startPoint, endPoint) {
     var breaks = breakInterval(startPoint, endPoint, 4);
     var circleToSplit = []
     for (var i = 0; i < breaks.length - 1; i++) {
       var sp = breaks[i],
           ep = breaks[i+1];
       var circle = splitableCircleAt(ep);
       if (circle && circle.isSplitable() && circle.checkIntersection(sp, ep)) {

    The findAndSplit function splits a potentially large segment traveled by the mouse into a series of small segments (not bigger than 4px long). It then checks each small segment for a potential circle intersection.

    function splitableCircleAt(pos) {
     var xi = Math.floor(pos[0] / minSize),
         yi = Math.floor(pos[1] / minSize),
         circle = finestLayer(xi, yi);
     if (!circle) return null;
     while (circle && !circle.isSplitable()) circle = circle.parent;
     return circle || null;

    The splitableCircleAt function takes advantage of the regular structure of the layout to find the one circle that the segment ending in the given point might be intersecting. This is done by finding the leaf node of the closest fine circle and traversing up the split tree to find its visible parent.

    Finally the intersected circle is split (circle.split()).

    Circle.prototype.split = function() {
     if (!this.isSplitable()) return;;
     delete this.node;
     Circle.addToVis(this.vis, this.children);

    Going viral

    Sometime after Valentines day I meet with Mike Bostock (the creator of D3) regarding D3 syntax and I showed him KttM, which he thought was tweet-worthy – it was, after all, an early example of a pointless artsy visualization done with D3.

    Mike has a twitter following and his tweet, which was retweeted by some members of the Google Chrome development team, started getting some momentum.

    Since the koala was out of the bag, I decided that it might as well be posted on reddit. I posted it on the programing subreddit with the tile “A cute D3 / SVG powered image puzzle. [No IE]” and it got a respectable 23 points which made me happy. Later that day it was reposted to the funny subreddit with the title “Press all the dots :D” and was upvoted to the front page.

    The traffic went exponential. Reddit was a spike that quickly dropped off, but people have picked up on it and spread it to Facebook, StumbleUpon, and other social media outlets.

    The traffic from these sources decays over time but every several months KttM gets rediscovered and traffic spikes.

    Such irregular traffic patterns underscore the need to write scalable code. Conveniently KttM does most of the work within the user’s browser; the server needs only to serve the page assets and one (small) image per page load allowing KttM to be hosted on a dirt-cheap shared hosting service.

    Measuring engagement

    After KttM became popular I was interested in exploring how people actually interacted with the application. Did they even realize that the initial single circle can split? Does anyone actually finish the whole image? Do people uncover the circles uniformly?

    At first the only tracking on KttM was the vanilla GA code that tracks pageviews. This quickly became underwhelming. I decided to add custom event tracking for when an entire layer was cleared and when a percentage of circles were split (in increments of 5%). The event value is set to the time in seconds since page load.

    As you can see such event tracking offers both insights and room for improvement. The 0% clear event is fired when the first circle is split and the average time for that event to fire seems to be 308 seconds (5 minutes) which does not sound reasonable. In reality this happens when someone opens KttM and leaves it open for days then, if a circle is split, the event value would be huge and it would skew the average. I wish GA had a histogram view.

    Even basic engagement tracking sheds vast amounts of light into how far people get through the game. These metrics proved very useful when the the mouse-over algorithm was upgraded. I could, after several days of running the new algorithm, see that people were finishing more of the puzzle before giving up.

    Lessons learned

    While making, maintaining, and running KttM I learned several lessons about using modern web standards to build web applications that run on a wide range of devices.

    Some native browser utilities give you 90% of what you need, but to get your app behaving exactly as you want, you need to reimplement them in JavaScript. For example, the SVG mouseover events could not cope well with the number of circles and it was much more efficient to implement them in JavaScript by taking advantage of the regular circle layout. Similarly, the native base64 functions (atob, btoa) are not universally supported and do not work with unicode. It is surprisingly easy to support the modern Internet Explorers (9 and 10) and for the older IEs Google Chrome Frame provides a great fallback.

    Despite the huge improvements in standard compliance it is still necessary to test the code on a wide variety of browsers and devices, as there are still differences in how certain features are implemented. For example, in IE10 running on the Microsoft Surface html {-ms-touch-action: none; } needed to be added to allow KttM to function correctly.

    Adding tracking and taking time to define and collect the key engagement metrics allows you to evaluate the impact of changes that get deployed to users in a quantitative manner. Having well defined metrics allows you to run controlled tests to figure out how to streamline your application.

    Finally, listen to your users! They pick up on things that you miss – even if they don’t know it. The congratulations message that appears on completion was added after I received complaints that is was not clear when a picture was fully uncovered.

    All projects are forever evolving and if you listen to your users and run controlled experiments then there is no limit to how much you can improve.

  10. Writing Web Apps Quickly With Mortar

    With the introduction of Firefox OS, development of an open apps marketplace, and a push to implement powerful web APIs for closer hardware integration, Mozilla is serious about web apps. We believe that the web can deliver an experience similar to native apps, even on mobile.

    We can’t forget about the most important piece, however: a good developer ecosystem. We need to provide various tools to make it easy for both beginners and experts to write web apps and deploy them.

    This is why we’ve created mortar, a collection of app templates that provide starting points for different kinds of apps and tools to manage and deploy them.

    A few weeks ago, Pierre Richard wrote a post here about his experiences getting started writing apps for Firefox OS. There’s some good stuff in his post, including his critiques of mortar. He felt mortar was too complex and introduced a different starting point for writing web apps for Firefox OS.

    This post will loosely respond to some of his points, introduce mortar, and explain why I think it is a valuable starting point for writing web apps for both beginners and experts. A detailed walkthrough of mortar is provided at the end in a screencast.

    Why Are Templates Useful?

    Mortar is more than just a bunch of HTML, CSS, and JavaScript. As described later in this post, it comes with various other tools for automating tasks, such as deployment and minifying and concatenating JavaScript and CSS. These tools are necessary when you want to release your app and maintain it.

    If you’re someone who really enjoys simplicity, you might initially find mortar too complex. I’m definitely someone who wants to boil things down to its simplest form, rejecting unnecessary complexity. We’ve worked hard to make sure that we only include things in mortar that prove to be very useful when building and deploying a web app.

    So I encourage you to learn about the tools mortar comes with, especially if you think you don’t need it. Mortar is our expression of good web development practices, so it’s also a chance to learn about modern tools and good ways to build web apps. Just as Django or Ruby on Rails may seem complex at first, you pay off your initial investment later on.

    The Technology Within Mortar

    Mortar templates are pre-built to help you write web apps quickly. These aren’t specifically Firefox OS apps. We want you to build web apps that can run anywhere the web is. Firefox OS is a good place for a web app, but the mortar templates aren’t specifically targeting Firefox OS (even if some of the more advanced templates borrow a few styles from it).

    All of the mortar templates come with the following:

    • A project structure (folders for css, js, etc)
    • Some initial well-formed HTML, JavaScript, and files like manifest.webapp
    • require.js to manage javascript
    • volo for a development server, css/js optimizations, deployment, and other tasks
    • Some initial js libs: zepto, mozilla’s receipt verifier library, and a cross-platform library to install web apps

    In addition, an installation button is provided (but commented out by default) in case you want to test your app as an installed app (or let users install it outside of an app store). All you have to do is uncomment one line of HTML. This, in combination with the pre-built manifest.webapp file, makes it easy to build an app.

    Here’s what your app looks like when you start it from a template:

    Mortar-app-stub repo on Github

    All of the static files, like HTML, CSS, and JavaScript, are under the www directory. We live one level up from this so that we can use tools like volo and grunt, and if you need to add a server-side component it’s easier. The node_modules and tools directories are only for volo, and you can ignore them.

    We chose to use require.js, a JavaScript module system, because it’s good practice to write JavaScript in modules. It’s not an obscure library; it implements the Asynchronous Module Definition standard for writing modules which is extremely well-known in the javascript community. The module you should start writing JavaScript in is at www/js/app.js.

    Besides managing your Javacript dependencies, modules let us easily compress and concatenate all your JavaScript into one minified file, with zero extra work on your part. In fact, it’s so easy that it’s wrapped up in a volo command for you. Just type volo build in your project and all your JavaScript is concatenated into one file and minified (it also does this with your CSS).

    Volo is a tool much like grunt which simply automates tasks at the command line. We use volo because it integrates nicely with require.js. An example command is volo serve, which fires up a development server for your app. View this documentation for more volo commands available.

    Volo lets you easily deploy your site to github. After customizing www/manifest.webapp for your app, simply do:

    1. volo build
    2. volo ghdeploy

    Your optimized app is now running live on github, and is ready to submit to the marketplace!

    Games and Layouts

    Everything described above is applicable to all apps, but we also want to help people who want to write specific kinds of apps. This is why we’ve created multiple templates, each catering to a specific need:

    • mortar-app-stub: a minimal template that only has a little pre-built HTML (a blank canvas for any type of app)
    • mortar-game-stub: a minimal template that comes with some basic game components
    • mortar-list-detail: a template like app-stub but also includes a layouts library and a basic list-detail type interface
    • mortar-tab-view: a template like list-detail but the default interface is a tabbed view

    We have a blank canvas (app-stub), a game template (game-stub), and two other templates that come with pre-built layouts.

    This is what the game template looks like. It looks pretty boring, but it’s rendered using requestAnimationFrame and you can control the box with the arrow keys or WASD. In the future, we might include image loading and basic collision detection.

    mortar-game-stub template

    The list-detail template is below. It comes with a pre-built layout that implements a list-detail interaction where you can touch a list item and drill down into the details. You can also add new items. This data-driven functionality is powered by backbone.js. We are still improving the design. (The tab-view template looks similar to the list-view shown here.)

    List detail in Mortar (screenshot)

    The layouts library is mortar-layouts, and it’s something we came up with to make it really easy to build application UIs. To me, this might be the most exciting part of mortar. I won’t go into too much detail here, but it uses x-tags and other new HTML5/CSS3 features to bring a native-feeling application UI to the web. View documentation and a live demo (and the source for the demo).

    The layouts library and the two templates that come with layouts are in beta, so keep in mind there are minor bugs. The biggest problem is the lack of design, and we will be working on integrating better styles that reflect Firefox OS (but are easily customizable with CSS!).

    Future Work for Mortar

    Mortar is relatively new, and it needs time to be fully documented and fleshed out. We are working on this actively.

    We are also fine-tuning our templates to figure out what the best starting points are for each one. Admittedly, having the big blue install button be the default screen for the app-stub was a mistake, which has been fixed. Now you get a short helpful message with links of what else you can do.

    Most of the work we have left is with the list-detail and tab-view templates, and figuring out how to deliver useful layouts to developers. We don’t want to force a big framework on them, but feel strongly that we need to help them develop application UIs.

    We love feedback, so feel free to file an issue to get a conversation started!


    I made a screencast which walks through mortar in detail and shows you how to use require.js, volo, and other things that it comes with. Watch this if you are interested in more details!