Mozilla

Found 448 results for “html5”

Sort by:

View:

  1. How to make a browser app for Firefox OS

    Firefox OS is an operating system built on top of the Firefox web browser engine, which is called Gecko. A browser app on Firefox OS provides a user interface written with HTML5 technology and manages web page browsing using the Browser API. It also manages tabbing, browsing history, bookmarks, and so on depending on the implementation.

    While Firefox OS already includes a browser, you can use the browser API to create your own browser or add browser functionality to your app. This article shows you how to build a browser app for Firefox OS devices. Following the steps, you’ll get a basic Firefox OS browser app with an address bar and back/forward buttons to browse web pages.

    The source code can be downloaded at https://github.com/begeeben/firefox-os-browser-sample.

    Prerequisites: WebIDE

    The WebIDE is available with Firefox 34 or later. A Firefox OS device is not required to develop, build or test a browser app. By using the WebIDE, we can easily bootstrap a web app, make HTML/CSS/JS modifications and run the app on one of the Firefox OS simulators. To open the WebIDE within Firefox, select Tools > Web Developer > WebIDE from the top menu:

    open webide

    Create an app from the template

    First, bootstrap a web app using the WebIDE empty template. The app type needs to be privileged so the browser permission can be set. This permission is required in order to use the Browser API. The Browser API provides additional methods and events to manage iframes.

    new priviledged app

    Below is a screenshot of what the app should look like. Next we’ll start to add code to make a simple browser app.

    mybrowser empty

    Edit the manifest.webapp

    Let’s start to make some code changes. The template has already set the app type to privileged in the manifest.webapp.

      "type": "privileged",

    To allow our app to use the Browser API, we need to specify the ‘browser‘ permission.

      "permissions": {
        "browser": {}
      },

    HTML structure

    The browser app we’re building will have a toolbar on top and a div as the iframe container which displays ‘Hello myBrowser!’ by default. The browser iframe will be created later using JavaScript. We could have added the iframe to the HTML, but in this example a new browser iframe will be created dynamically.

    <!-- browser toolbar -->
    <div id="toolbar">
        <button id="goback-button" type="button" disabled><</button>
        <button id="goforward-button" type="button" disabled>></button>
        <!-- url bar displays page url or page title -->
        <form id="url-bar">
            <input id="url-input" type="text"></input>
        </form>
        <button id="url-button" form="url-bar">GO</button>
    </div>
    <!-- the container for the browser frame -->
    <div id="frame-container" class="empty">
        <div id="empty-frame">
            <h1>Hello myBrowser!</h1>
        </div>
    </div>

    Below is a screenshot of the simple browser app:

    browser startup

    Manage the browser iframe

    The browser app has to handle mozbrowser events which are accessible when the app has the browser permission. In order to separate the UI and mozbrowser event handling and preserve the flexibility of supporting multiple tabs in the future, we have added a tab.js file. Following is the code in tab.js which creates an iframe with the ‘mozbrowser‘ attribute to enable the use of the Browser API.

    /**
     * Returns an iframe which runs in a child process with Browser API enabled
     * and fullscreen is allowed
     *
     * @param  {String} [url] Optional URL
     * @return {iframe}       An OOP mozbrowser iframe
     */
    function createIFrame (url) {
      var iframe = document.createElement('iframe');
      iframe.setAttribute('mozbrowser', true);
      iframe.setAttribute('mozallowfullscreen', true);
      iframe.setAttribute('remote', true);
     
      if (url) {
        iframe.src = url;
      }
     
      return iframe;
    }

    The mozallowfullscreen attribute enables the embedded web page of the iframe to use fullscreen mode. A web page can request fullscreen mode by calling Element.mozRequestFullscreen().

    The ‘remote‘ attribute separates the embedded iframe into another child process. This is needed for security reasons to prevent malicious web sites from compromising the browser app. Currently, the attribute does nothing in this sample browser app because Nested OOP has not been implemented. See Bug 1020135 - (nested-oop) [meta] Allow nested oop <iframe mozbrowser>.

    Next the ‘Tab‘ object constructor is added to tab.js. This constructor handles browser iframe creation and browser event handling. When a Tab is constructed, it creates a browser iframe and attaches the mozbrowser event listeners to it:

    /**
     * The browser tab constructor.
     *
     * Creates an iframe and attaches mozbrowser events for web browsing.
     *
     * Implements EventListener Interface.
     *
     * @param {String} url An optional plaintext URL
     */
    function Tab (url) {
      this.iframe = createIFrame(url);
      this.title = null;
      this.url = url;
     
      this.iframe.addEventListener('mozbrowserloadstart', this);
      this.iframe.addEventListener('mozbrowserlocationchange', this);
      this.iframe.addEventListener('mozbrowsertitlechange', this);
      this.iframe.addEventListener('mozbrowserloadend', this);
      this.iframe.addEventListener('mozbrowsererror', this);
    };

    There are additional mozbrowser events that are not used in this simple browser app. The events used in the sample provide access to useful information about the iframe, and our browser app uses them to provide web page details, like the title, URL, loading progress, contextmenu, etc.

    For example, to display the page title on the toolbar, we use the mozbrowsertitlechange event to retrieve the title of the web page. Once the title is updated, a CustomEvent 'tab:titlechange' with the Tab itself as the detail is dispatched to notify other components to update the title.

    Tab.prototype.mozbrowsertitlechange = function _mozbrowsertitlechange (e) {
      if (e.detail) {
        this.title = e.detail;
      }
     
      var event = new CustomEvent('tab:titlechange', { detail: this });
      window.dispatchEvent(event);
    };

    Code can now be added to the main app.js to handle this custom event, which updates the title.

    /**
     * Display the title of the currentTab on titlechange event.
     */
    window.addEventListener('tab:titlechange', function (e) {
      if (currentTab === e.detail) {
        urlInput.value = currentTab.title;
      }
    });

    Browse a web page on submit event

    Using the address bar as a search bar is a common and useful practice among browsers. When a user submits the input, first we check if it is a valid URL. If not, the input is regarded as a search term and it will be appended to a search engine URI. Note that the validation of the URL can be handled in multiple ways. The sample app uses an input element with its type attribute set to URL. A URL object could also have been used with the URL string passed as the first parameter to the constructor. A DOM exception will be thrown if the URL is not valid. This exception can then be caught and the URL string assumed to be a search term.

    /**
     * The default search engine URI
     *
     * @type {String}
     */
    var searchEngineUri = 'https://search.yahoo.com/search?p={searchTerms}';
     
    /**
     * Using an input element to check the validity of the input URL. If the input
     * is not valid, returns a search URL.
     *
     * @param  {String} input           A plaintext URL or search terms
     * @param  {String} searchEngineUri The search engine to be used
     * @return {String}                 A valid URL
     */
    function getUrlFromInput(input, searchEngineUri) {
      var urlValidate = document.createElement('input');
      urlValidate.setAttribute('type', 'url');
      urlValidate.setAttribute('value', input);
     
      if (!urlValidate.validity.valid) {
        var uri = searchEngineUri.replace('{searchTerms}', input);
        return uri;
      }
     
      return input;
    }

    Code can then be added to the main app.js to handle the submit button. This code checks to see if a currently active Tab object exists, and either loads the URL or reloads the URL if it has not changed. If one does not exist, it is created and the URL is loaded.

    /**
     * Check the input and browse the address with a Tab object on url submit.
     */
    window.addEventListener('submit', function (e) {
      e.preventDefault();
     
      if (!currentUrlInput.trim()) {
        return;
      }
     
      if (frameContainer.classList.contains('empty')) {
        frameContainer.classList.remove('empty');
      }
     
      var url = getUrlFromInput(currentUrlInput.trim(), searchEngineUri);
     
      if (!currentTab) {
        currentTab = new Tab(url);
        frameContainer.appendChild(currentTab.iframe);
      } else if (currentUrlInput === currentTab.title) {
        currentTab.reload();
      } else {
        currentTab.goToUrl(url);
      }
    });

    The currentTab.reload function uses the iframe.reload function provided by the Browser API to reload the page. The code below is added to the tab.js file to handle the reload.

    /**
     * Reload the current page.
     */
    Tab.prototype.reload = function _reload () {
      this.iframe.reload();
    };

    Enable/disable back and forward buttons

    Finally the Browser API can be used to check if the page can go backward or forward in the navigation history. The iframe.getCanGoBack method returns a DOMRequest object which indicates if the page can go backward in its onsuccess callback. We use a Promise to wrap this function for easier access. This code is added to the tab.js file.

    /**
     * Check if the iframe can go backward in the navigation history.
     *
     * @return {Promise} Resolve with true if it can go backward.
     */
    Tab.prototype.getCanGoBack = function _getCanGoBack () {
      var self = this;
     
      return new Promise(function (resolve, reject) {
        var request = self.iframe.getCanGoBack();
     
        request.onsuccess = function () {
          if (this.result) {
            resolve(true);
          } else {
            resolve(false);
          }
        };
      });
    };

    The check is performed in the app.js file when a page is loaded. This occurs when the iframe receives a mozbrowserloadend event.

    /**
     * Enable/disable goback and goforward buttons accordingly when the
     * currentTab is loaded.
     */
    window.addEventListener('tab:loadend', function (e) {
      if (currentTab === e.detail) {
        currentTab.getCanGoBack().then(function(canGoBack) {
          gobackButton.disabled = !canGoBack;
        });
     
        currentTab.getCanGoForward().then(function(canGoForward) {
          goforwardButton.disabled = !canGoForward;
        });
      }
    });

    Run the app

    One of the Firefox OS simulators can now be used to test the sample browser app.

    Firefox_WebIDE_installing_simulator

    On the top right of the WebIDE, click Select Runtime to choose a Firefox OS phone or simulator:

    Firefox_WebIDE_choose_simulator

    Once a Firefox OS runtime is selected and launched, you can install and run the app by clicking the play button within the WebIDE. See the WebIDE documentation for more information on building, debugging, and deploying Firefox OS apps.

    mybrowser finished

    Final thoughts

    There’s more functionality that can be added to this simple little browser to make it really useful including bookmarks, settings, search suggestions, and sharing. Feel free to expand on this code and have fun with it. Let us know what you think and be sure to spread the word about Firefox OS!

    Additional information

    https://hacks.mozilla.org/2014/08/building-the-firefox-browser-for-firefox-os/
    https://developer.mozilla.org/en-US/docs/Web/API/Using_the_Browser_API

  2. An analytics primer for developers

    There are three kinds of lies: lies, damned lies, and statistics – Mark Twain

    Deciding what to track (all the things)

    When you are adding analytics to a system you should try to log everything. At some point in the future if you need to pull information out of a system it’s much better to have every piece of information to hand, rather than realising that you need some data that you don’t yet track. Here are some guidelines and suggestion for collecting and analysing information about how people are interacting with your website or app.

    Grouping your stats as a best practice

    Most analytics platforms allow you to tag an event with metadata. This lets you analyse stats against each other and makes it easier to compare elements in a user interaction.

    For example, if you are logging clicks on a menu, you could track each menu item differently e.g.:

    track("Home pressed");
    track("Cart pressed");
    track("Logout pressed");

    Doing this makes it harder to answer questions such as which button is most popular etc. Using metadata you can make most analytics platforms perform calculations like this for you:

    track("Menu pressed","Home button");
    track("Menu pressed","Cart button");
    track("Menu pressed","Logout button");

    The analytics above mean you now have a total of all menu presses, and you can find the most/least popular of the menu items with no extra effort.

    Optimising your funnel

    A conversion funnel is a term of art derived from a consumer marketing model. The metaphor of the funnel describes the flow of steps a user goes through as they engage more deeply with your software. Imagine you want to know how many users clicked log in and then paid at the checkout? If you track events such as “Checkout complete” and “User logged in” you can then ask your analytics platform what percentage of users did both within a certain time frame (a day for instance).

    Imagine the answer comes out to be 10%, this tells you useful information about the behaviour of your users (bear in mind this funnel is not order sensitive, i.e., it does not matter in which order the events happen in login -> cart -> pay or cart -> login -> pay). Thus, you can start to optimise parts of your app and use this value to determine whether or not you are converting more of your users to make a purchase or otherwise engage more deeply.

    Deciding what to measure

    Depending on your business, different stats will have different levels of importance. Here are some common stats of interest to developers of apps or online services:

    Number of sessions
    The total number of sessions on your product (the user opening your product, using it, then closing it = 1 session)
    Session length
    How long each session lasts (can be mode, mean, median)
    Retention
    How many people come back to your product having used it before (there are a variety of metrics such as rolling retention, 30 day retention etc)
    MAU
    Monthly active users: how may users use the app once a month
    DAU
    Daily active users: how may users use the app once a day
    ARPU
    Average revenue per user: how much money you make per person
    ATV
    Average transaction value: how much money you make per sale
    CAC
    Customer acquisition cost: how much it costs to get one extra user (normally specified by the channel for getting them)
    CLV
    Customer lifetime value: total profit made from a user (usually projected)
    Churn
    The number of people who leave your product in a given time (usually given as a percentage of total user base)
    Cycle time
    The the time it takes for one user to refer another

    Choosing an analytics tool or platform

    There are plenty of analytics providers, listed below are some of the best known and most widely used:

    Google Analytics

    Website
    Developer documentation

    Quick event log example:

    ga('send', 'event', 'button', 'click');

    Pros:

    • Free
    • Easy to set up

    Cons:

    • Steep learning curve for using the platform
    • Specialist training can be required to get the most out of the platform

    Single page apps:

    If you are making a single page app/website, you need to keep Google informed that the user is still on your page and hasn’t bounced (gone to your page/app and left without doing anything):

    ga('set' , 'page', location.pathname + location.search + location.hash);
    ga('send', 'pageview');

    Use the above code every time a user navigates to a new section of your app/website to let Google know the user is still browsing your site/app.

    Flurry

    Website
    Developer documentation

    Quick event log example:

    FlurryAgent.logEvent("Button clicked");
    FlurryAgent.logEvent("Button clicked",{more : 'data'});

    Pros:

    • Free
    • Easy to set up

    Cons:

    • Data normally 24 hours behind real time
    • Takes ages to load the data

    Mixpanel

    Website
    Developer documentation

    Quick event log example:

    mixpanel.track("Button clicked");
    mixpanel.track("Button clicked",{more : 'data'});

    Pros:

    • Free trial
    • Easy to set up
    • Real-time data

    Cons:

    • Gets expensive after the free trial
    • If you are tracking a lot of points, the interface can get cluttered

    Speeding up requests

    When you are loading an external JS file you want to do it asynchronously if possible to speed up the page load.

    <script type="text/javascript" async> ... </script>

    The above code will cause the JavaScript to load asynchronously but assumes the user has a browser supporting HTML5.

    //jQuery example
    $.getScript('https://cdn.flurry.com/js/flurry.js', 
    function(){
       ...
    });

    This code will load the JavaScript asynchronously with greater browser support.

    The next problem is that you could try to add an analytic even though the framework does not exist yet, so you need to check to see if the variable framework first:

    if(typeof FlurryAgent != "undefined"){
       ...
    }

    This will prevent errors and will also allow you to easily disable analytics during testing. (You can just stop the script from being loaded – and the variable will never be defined.)

    The problem here is that you might be missing analytics whilst waiting for the script to load. Instead, you can make a queue to store the events and then post them all when the script loads:

    var queue = [];
     
    if(typeof FlurryAgent != "undefined"){
       ...
    }else{
       queue.push(["data",{more : data}]);
    }
     
    ...
     
    //jQuery example
    $.getScript('https://cdn.flurry.com/js/flurry.js', 
    function(){
       ...
     
       for(var i = 0;i < queue.length;i++)
       {
          FlurryAgent.logEvent(queue[i][0],queue[i][1]);
       }
       queue = [];
    });

    Analytics for your Firefox App

    You can use any of the above providers above with Firefox OS, but remember when you paste a script into your code they generally are protocol agnostic: they start //myjs.com/analytics.js and you need to choose either http: or https:https://myjs.com/analytics.js (This is required only if you are making a packaged app.)

    Let us know how it goes.

  3. Ruby support in Firefox Developer Edition 38

    It was a long-time request from East Asian users, especially Japanese users, to have ruby support in the browser.

    Formerly, because of the lack of native ruby support in Firefox, users had to install add-ons like HTML Ruby to make ruby work. However, in Firefox Developer Edition 38, CSS Ruby has been enabled by default, which also brings the support of HTML5 ruby tags.

    Introduction

    What is ruby? In short, ruby is an extra text, which is usually small, attached to the main text for indicating the pronunciation or meaning of the corresponding characters. This kind of annotation is widely used in Japanese publications. It is also common in Chinese for books for children, educational publications, and dictionaries.

    Ruby Annotation

    Basic Usage

    Basically, the ruby support consists of four main tags: <ruby>, <rb>, <rt>, and <rp>. <ruby> is the tag that wraps the whole ruby structure, <rb> is used to mark the text in the normal line, <rt> is for the annotation, and <rp> is a tag which is hidden by default. With the four tags, the result above can be achieved from the following code:

    <ruby>
      <rb>とある<rb>科学<rb><rb>超電磁砲</rb>
      <rp></rp><rt>とある<rt>かがく<rt><rt>レールガン</rt><rp></rp>
    </ruby>

    Since <rb> and <rt> can be auto-closed by themselves, we don’t bother to add code to close those tags manually.

    As shown in the image, the duplicate parts as well as <rp>s are hidden automatically. But why should we add content which is hidden by default?

    The answer is: it is more natural in semantics, and it helps conversion to the inline form, which is a more general form accepted by more software. For example, this allows the page to have a decent effect on a browser with no ruby support. It also enables the user agent to generate well-formed plain text when you want to copy text with ruby (though this feature hasn’t yet been landed on Firefox).

    In addition, the extra content makes it possible to provide inline style of the annotation without changing the document. You would only need to add a rule to your stylesheet:

    ruby, rb, rt, rp {
      display: inline;
      font-size: inherit;
    }

    Actually, if you don’t have those requirements, only <ruby> and <rt> are necessary. For simplest cases, e.g. a single ideographic character, you can use code like:

    <ruby><rt>Saki</rt></ruby>

    Advanced Support

    Aside from the basic usage of ruby, Firefox now provides support for more advanced cases.

    By default, if the width of an annotation does not match its base text, the shorter text will be justified as shown in the example above. However, this behavior can be controlled via the ruby-align property. Aside from the default value (space-around), it can also make the content align to both sides (space-between), centered (center), or aligned to the start side (start).

    Multiple levels of annotations are also supported via tag <rtc> which is the container of <rt>s. Every <rtc> represents one level of annotation, but if you leave out the <rtc>, the browser will do some cleanup to wrap consecutive <rt>s in a single anonymous <rtc>, forming one level.

    For example, we can extend the example above to:

    <ruby>
      <rb>とある<rb>科学<rb><rb>超電磁砲</rb>
      <rp></rp><rt>とある<rt>かがく<rt><rt>レールガン</rt><rp></rp>
      <rtc><rt>Toaru<rt>Kagaku<rt>no<rt>Rērugan</rt></rtc><rp></rp>
    </ruby>

    If you do not put any <rt> inside a <rtc>, this annotation will become a span across the whole base:

    <ruby>
      <rb>とある<rb>科学<rb><rb>超電磁砲</rb>
      <rp></rp><rt>とある<rt>かがく<rt><rt>レールガン</rt><rp></rp>
      <rtc lang="en">A Certain Scientific Railgun</rtc><rp></rp>
    </ruby>

    You can use ruby-position to place the given level of annotation on the side you want. For the examples above, if you want to put the second level under the main line, you can apply ruby-position: under; to the <rtc> tag. Currently, only under and the default value over is supported.

    (Note: The CSS Working Group is considering a change to the default value of ruby-position, so that annotations become double-sided by default. This change is likely to happen in a future version of Firefox.)

    In the end, an advanced example of ruby combining everything introduced above:

    Advanced Example for Ruby

    rtc:lang(en) {
      ruby-position: under;
      ruby-align: center;
      font-size: 75%;
    }
    <ruby>
      <rb>とある<rb>科学<rb><rb>超電磁砲</rb>
      <rp></rp><rt>とある<rt>かがく<rt><rt>レールガン</rt><rp></rp>
      <rtc lang="en">A Certain Scientific Railgun</rtc><rp></rp>
    </ruby>

    Got questions, comments, feedback on the implementation? Don’t hesitate to share your thoughts or issues here or via bugzilla.

  4. 2014: Mozilla Hacks looks back

    Wherever you live, it’s a season of work holidays, school vacations, year-end blog posts, and lists. The Hacks blog will be back in early January 2015 to continue writing for developers about the products and technologies created by Mozilla and by builders of the Open Web around the world.

    In the (chronological) list below, we celebrate some 2014 milestones and memorable moments for Mozilla, Firefox, and the web platform:

    • The new improved Firefox Sync launched early in 2014, built atop Firefox Accounts. Here’s a deeper look at the new Sync protocol.
    • Mozilla worked with the Apache Cordova team to integrate Firefox OS with Cordova’s open source device APIs, enabling mobile app developers to access native device functions from JavaScript, and release Cordova apps for the Firefox OS platform. Cordova is the underlying software in Adobe PhoneGap: the integration made it possible for PhoneGap developers to easily port their apps to the Firefox Marketplace.
    • Mozilla Research made great progress on Servo, a prototype browser engine. Servo is written in Rust, a systems programming language designed to support concurrency and parallelism for modern hardware. Another Big Milestone for Servo—Acid2 was reported in the spring.
    • Australis was the codename for our major redesign of the Firefox Desktop browser, which launched in late April. Designer Stephen Horlander wrote a post about the process, titled (Re)Designing Firefox.
    • The summer release of Firefox 31 brought new capabilities for game developers, including access to emscripten for compiling to JavaScript. In Resources for HTML5 game developers, we described some of the tools and techniques for coding and debugging sophisticated, performant HTML5 games. On the Mozilla blog, we featured the first commercial games to leverage asm.js, demonstrating the readiness of HTML5 as a game platform.
    • In October, we shared a Preview of Firefox Hello, a WebRTC-powered voice/video chat service.
    • As part of the festivities surrounding Firefox’s 10th birthday, Mozilla Research launched MozVR to help bring virtual reality to the Web. Grab your oculi and hold on to your hats.
    • At the same time, the Firefox Developer Tools team released Firefox Developer Edition, the first browser created specifically for developers. Though still in its early days, the Developer Edition features Valence, an integrated add-on to let you develop and debug your app across multiple browsers and devices, as well as WebIDE, providing an editing environment for app development and remote debugging.
    • The evolution of asm.js continued with the release of Massive, an open source benchmarking tool that measures asm.js performance specifically. You can run Massive here.
    • Mozilla and partners announced the formation of the Internet Security Research Group (ISRG) and launched the Let’s Encrypt initiative. Let’s Encrypt is a new Certificate Authority that’s free, automated, and open. It’s due to arrive in summer 2015.
    • Our friends at Telenor introduced Gonzo, an ongoing project which explores the use of Firefox OS as an embedded platform. Telenor engineer and Firefox OS evangelist Jan Jongboom is hacking together a wireless camera and sharing what he learns along the way.
    • Firefox OS Expands to Nearly 30 Countries – It’s been an expansive year. At last count, Firefox OS is now up and running on 16 smartphones offered in 29 countries. And in December news, Mozilla and KDDI announced the release of Japan’s first Firefox OS smartphone, which went on sale on December 25. The Fx0 is the first high-spec Firefox OS smartphone and it’s a beauty!

    If you’re interested in writing for the Hacks blog in 2015, we’d love to hear from you. Email us. In the meantime, have a safe and hacky holiday!

  5. You can’t go wrong watching JavaScript talks

    Late last week, I was collecting suggestions for year-end Hacks blog posts. As she headed out for the winter holidays, apps engineer Soledad Penadés gifted me “a bunch of cool talks I watched this year.”

    In fact, it’s a curated collection of presentations from JSConf, JSConf EU, and other recent developer conferences. Presenters include notable Mozillians and opinionated friends: hackers, JavaScripters, teachers, preachers, and hipsters. Many talks come with links to slides. They cover diverse topics: software development, developer culture and practices, developer tools, open source, JavaScript, CSS, new APIs.

    Part 1: Sole’s list

    Part 2: Making video better on Firefox

    Binge-watching JavaScript talks not your idea of a holiday break? If you’re watching video at all, Mozilla could use your help improving the video experience on Firefox.

    Here’s how you can contribute: Download Firefox Nightly, and help us test the implementation of Media Source Extensions (MSE) to support YouTube videos running in HTML5. Over recents weeks, Firefox engineers have been working hard addressing top bugs. To get this into the hands of users as quickly as possible, we need more help testing the experience. Here’s a detailed description of the testing task, thanks to Bugmaster Liz Henry from the QA team.

    File bugs using this link. Describe your platform, OS, and the steps needed to reproduce the problem.

    Thank you for helping to make the Firefox Desktop video experience awesome. See you in the new year.

  6. Videos and Firefox OS

    Before HTML5

    Those were dark times Harry, dark times – Rubeus Hagrid

    Before HTML5, displaying video on the Web required browser plugins and Flash.

    Luckily, Firefox OS supports HTML5 video so we don’t need to support these older formats.

    Video support on the Web

    Even though modern browsers support HTML5, the video formats they support vary:

    In summary, to support the most browsers with the fewest formats you need the MP4 and WebM video formats (Firefox prefers WebM).

    Multiple sizes

    Now that you have seen what formats you can use, you need to decide on video resolutions, as desktop users on high speed wifi will expect better quality videos than mobile users on 3G.

    At Rormix we decided on 720p for desktop, 360p for mobile connections, and 180p specially for Firefox OS to reduce the cost in countries with high data charges.

    There are no hard and fast rules — it depends on who your market audience is.

    Streaming?

    The best streaming solution would be to automatically serve the user different videos sizes depending on their connection status (adaptive streaming) but support for this technology is poor.

    HTTP live streaming works well on Apple devices, but has poor support on Android.

    At the time of writing, the most promising technology is MPEG DASH, which is an international standard.

    In summary, we are going to have to wait before we get an adaptive streaming technology that is widely accepted (Firefox does not support HLS or MPEG DASH).

    DIY Adaptive streaming

    In the absence of adaptive streaming we need to try to work out the best video quality to load at the outset. The following is a quick guide to help you decide:

    Wifi or 3G

    Using a certified Firefox OS app you can check to see if the user is on wifi or not.

    var lock    = navigator.mozSettings.createLock();
    var setting = lock.get('wifi.enabled');
     
    setting.onsuccess = function () {
      console.log('wifi.enabled: ' + setting.result);
    }
     
    setting.onerror = function () {
      console.warn('An error occured: ' + setting.error);
    }

    https://developer.mozilla.org/en-US/docs/Web/API/Settings_API

    There is some more information at the W3C Device API.

    Detecting screen size

    There is no point sending a 720p video to a user with a screen smaller than 720p. There are many ways to get the different bounds of a user’s screen; innerWidth and width allow you to get a good idea:

    function getVidSize()
    {
      //Get the width of the phone (rotation independent)
      var min = Math.min($(window).innerHeight(),$(window).innerWidth());
      //Return a video size we have
      if(min < 320)      return '180';
      else if(min < 550) return '360';
      else               return '720';
    }

    http://www.quirksmode.org/m/tests/widthtest.html

    Determining internet speed

    It is difficult to get an accurate read of a user’s internet speed using web technologies — usually they involve loading a large image onto the user’s device and timing it. This has the disadvantage of having to send more data to the user. Some services such as: http://speedof.me/api.html exist, but still require data downloads to the user’s device. (Stackoverflow has some more options.)

    You can be slightly more clever by using HTML5, and checking the time it takes between the user starting the video and a set amount of the video loading. This way we do not need to load any extra data on the user’s device. A quick VideoJS example follows:

    var global_speedcount = 0;
    var global_video = null;
    global_video = videojs("video", {}, function(){
    //Set up video sources
    });
     
    global_video.on('play',function(){
      //User has clicked play
      global_speedcount = new Date().getTime();
    });
     
    function timer()
    {
      var diff = new Date().getTime() - global_speedcount;
      //Remove this handler as it is run multiple times per second!
      global_video.off('timeupdate',timer);
    }
     
    global_video.on('timeupdate',timer);

    This code starts timing when the user clicks play, and when the browser starts to play the video it sends timing information to timeupdate. You can also use this function to detect if lots of buffering is happening.

    Detect high resolution devices

    One final thing to determine is whether or not a user has a high pixel density screen. In this case even if they have a small screen it can still have a large number of pixels (and therefore require a higher resolution video).

    Modernizr has a plugin for detecting hi-res screens.

    if (Modernizr.highresdisplay)
    {
      alert('Your device has a high resolution screen');
    }

    WebP Thumbnails

    Not to get embroiled in an argument, but at Rormix we have seen an average decrease of 30% in file size (WebP vs JPEG) with no loss of quality (in some cases up to 50% less). And in countries with expensive data plans, the less data the better.

    We encode all of our thumbnails in multiple resolutions of WebP and send them to every device that supports them to reduce the amount of data being sent to the user.

    Mobile considerations

    If you are playing HTML5 videos on mobile devices, their behavior differs. On iOS it automatically goes to full screen on iPhones/iPods, but not on tablets.

    Some libraries such as VideoJS have removed the controls from mobile devices until their stability increases.

    Useful libraries

    There are a few useful HTML5 video libraries:

    Mozilla links

    Mozilla has some great articles on web video:

    Other useful Links

  7. An easier way of using polyfills

    Polyfills are a fantastic way to enable the use of modern code even while supporting legacy browsers, but currently using polyfills is too hard, so at the FT we’ve built a new service to make it easier. We’d like to invite you to use it, and help us improve it.

    Image from https://www.flickr.com/photos/hamur0w0/6984884135

    More pictures, they said. So here’s a unicorn, which is basically a horse with a polyfill.

    The challenge

    Here are some of the issues we are trying to solve:

    • Developers do not necessarily know which features need to be polyfilled. You load your site in some old version of IE beloved by a frustratingly large number of your users, see that the site doesn’t work, and have to debug it to figure out which feature is causing the problem. Sometimes the culprit is obvious, but often not, especially when legacy browsers also lack good developer tools.
    • There are often multiple polyfills available for each feature. It can be hard to know which one most faithfully emulates the missing feature.
    • Some polyfills come as a big bundle with lots of other polyfills that you don’t need, to provide comprehensive coverage of a large feature set, such as ES6. It should not be necessary to ship all of this code to the browser to fix something very simple.
    • Newer browsers don’t need the polyfill, but typically the polyfill is served to all browsers. This reduces performance in modern browsers in order to improve compatibility with legacy ones. We don’t want to make that compromise. We’d rather serve polyfills only to browsers that lack a native implementation of the feature.

    Our solution: polyfills as a service

    To solve these problems, we created the polyfill service. It’s a similar idea to going to an optometrist, having your eyes tested, and getting a pair of glasses perfectly designed to correct your particular vision problem. We are doing the same for browsers. Here’s how it works:

    1. Developers insert a script tag into their page, which loads the polyfill service endpoint.
    2. The service analyses the browser’s user-agent header and a list of requested features (or uses a default list of everything polyfillable) and builds a list of polyfills that are required for this browser
    3. The polyfills are ordered using a graph sort to place them in the right dependency order.
    4. The bundle is minified and served through a CDN (for which we’re very grateful to Fastly for their support)

    Do we really need this solution? Well, consider this: Modernizr is a big grab bag of feature detects, and all sensible use cases benefit from a custom build, but a large proportion of Modernizr users just use the default build, often from cdnjs.com or as part of html5boilerplate. Why include Modernizr if you aren’t using its feature detects? Maybe you misunderstand the purpose of the library and just think that Modernizr “fixes stuff”? I have to admit, I did, when I first heard the name, and I was mildly disappointed to find that rather than doing any actual modernising, Modernizr actually just defines modernness.

    The polyfill service, on the other hand, does fix stuff. There’s really nothing wrong with not wanting to spend time gaining intimate knowledge of all the foibles of legacy browsers. Let someone figure it out once, and then we can all benefit from it without needing or wanting to understand the details.

    How to use it

    The simplest use case is:

    <script src="//cdn.polyfill.io/v1/polyfill.min.js" async defer></script>

    This includes our default polyfill set. The default set is a manually curated list of features that we think are most essential to modern web development, and where the polyfills are reasonably small and highly accurate. If you want to specify which features you want to polyfill though, go right ahead:

    <!-- Just the Array.from polyfill -->
    <script src="//cdn.polyfill.io/v1/polyfill.min.js?features=Array.from" async defer></script>
     
    <!-- The default set, plus the geolocation polyfill -->
    <script src="//cdn.polyfill.io/v1/polyfill.min.js?features=default,Navigator.prototype.geolocation" async defer></script>

    If it’s important that you have loaded the polyfills before parsing your own code, you can remove the async and defer attributes, or use a script loader (one that doesn’t require any polyfills!).

    Testing and documenting feature support

    This table shows the polyfill service’s effect for a number of key web technologies and a range of popular browsers:

    Polyfill service support grid

    The full list of features we support is shown on our feature matrix. To build this grid we use Sauce Labs’ test automation platform, which runs each polyfill through a barrage of tests in each browser, and documents the results.

    So, er, user-agent sniffing? Really?

    Yes. There are several reasons why UA analysis wins out over feature detection for us:

    • In some cases, we have multiple polyfills for the same feature, because some browsers offer a non-compliant implementation that just needs to be bashed into shape, while others lack any implementation at all. With UA detection you can choose to serve the right variant of the polyfill.
    • With UA detection, the first HTTP request can respond directly with polyfill code. If we used feature detection, the first request would serve feature-detect code, and then a second one would be needed to fetch specific polyfills.

    Almost all websites with significant scale do UA detection. This isn’t to say the stigma attached to it is necessarily bad. It’s easy to write bad UA detect rules, and hard to write good ones. And we’re not ruling out making a way of using the service via feature-detects (in fact there’s an issue in our tracker for it).

    A service for everyone

    The service part of the app is maintained by the FT, and we are working on expanding and improving the tools, documentation, testing and service features all the time. The source is freely available on GitHub so you can easily host it yourself, but we also host an instance of the service on cdn.polyfill.io which you can use for free, and our friends at Fastly are providing free CDN distribution and SSL.

    We’ve made a platform. We need the community’s help to populate it. We already serve some of the best polyfills from Jonathan Neal, Mathias Bynens and others, but we’d love to be more comprehensive. Bring your polyfills, improve our tests, and make this a resource that can help move the web forward!

  8. Massive: The asm.js Benchmark

    asm.js is a subset of JavaScript that is very easy to optimize. Most often it is generated by a compiler, such as Emscripten, from C or C++ code. The result can run at very high speeds, close to that of the same code compiled natively. For that reason, Emscripten and asm.js are useful for things like 3D game engines, which are usually large and complex C++ codebases that need to be fast, and indeed top companies in the game industry have adopted this approach, for example Unity and Epic, and you can see it in action in the Humble Mozilla Bundle, which recently ran.

    As asm.js code becomes more common, it is important to be able to measure performance on it. There are of course plenty of existing benchmarks, including Octane which contains one asm.js test, and JetStream which contains several. However, even those do not contain very large code samples, and massive codebases are challenging in particular ways. For example, just loading a page with such a script can take significant time while the browser parses it, causing a pause that is annoying to the user.

    A recent benchmark from Unity measures the performance of their game engine, which (when ported to the web) is a large asm.js codebase. Given the high popularity of the Unity engine among developers, this is an excellent benchmark for game performance in browsers, as real-world as it can get, and also it tests large-scale asm.js. It does however focus on game performance as a whole, taking into account both WebGL and JavaScript execution speed. For games, that overall result is often what you care about, but it is also interesting to measure asm.js on its own.

    Benchmarking asm.js specifically

    Massive is a benchmark that measures asm.js performance specifically. It contains several large, real-world codebases: Poppler, SQLite, Lua and Box2D; see the FAQ on the massive site for more details on each of those.

    Massive reports an overall score, summarizing it’s individual measurements. This score can help browser vendors track their performance over time and point to areas where improvements are needed, and for developers it can provide a simple way to get an idea of how fast asm.js execution is on a particular device and browser.

    Importantly, Massive does not only test throughput. As already mentioned, large codebases can affect startup time, and they can also affect responsiveness and other important aspects of the user experience. Massive therefore tests, in addition to throughput, how long it takes the browser to load a large codebase, and how responsive it is while doing so. It also tests how consistent performance is. Once again, see the FAQ for more details on each of those.

    Massive has been developed openly on github from day one, and we’ve solicited and received feedback from many relevant parties. Over the last few months Massive development has been in beta while we received comments, and there are currently no substantial outstanding issues, so we are ready to announce the first stable version, Massive 1.0.

    Massive tests multiple aspects of performance, in new ways, so it is possible something is not being measured in an optimal manner, and of course bugs always exist in software. However, by developing Massive in the open and thereby giving everyone the chance to inspect it and report issues, and by having a lengthy beta period, we believe we have the best possible chance of a reliable result. Of course, if you do find something wrong, please file an issue! General feedback is of course always welcome as well.

    Massive performance over time

    Massive is brand-new, but it is still interesting to look at how it performs on older browsers (“retroactively”), because if it measures something useful, and if browsers are moving in the right direction, then we should see Massive improve over time, even on browser versions that were released long before Massive existed. The graph below shows Firefox performance from version 14 (released 2012-07-17, over 2 years ago) and version 32 (which became the stable version in September 2014):

    Higher numbers are better, so we can indeed see that Massive scores do follow the expected pattern of improvement, with Firefox’s Massive score rising to around 6x its starting point 2 years ago. Note that the Massive score is not “linear” in the sense that 6x the score means 6x the performance, as it is calculated using the geometric mean (like Octane), however, the individual scores it averages are mostly linear. A 6x improvement therefore does represent a very large and significant speedup.

    Looking more closely at the changes over time, we can see which features landed in each of those versions of Firefox where we can see a significant improvement:

    There are three big jumps in Firefox’s Massive score, each annotated:

    • Firefox 22 introduced OdinMonkey, an optimization module for asm.js code. By specifically optimizing asm.js content, it almost doubled Firefox’s Massive score. (At the time, of course, Massive didn’t exist; but we measured speedups on other benchmarks.)
    • Firefox 26 parses async scripts off of the main thread. This avoids the browser or page becoming nonresponsive while the script loads. For asm.js content, not only parsing but also compilation happens in the background, making the user experience even smoother. Also in Firefox 26 are general optimizations for float32 operations, which appear in one of the Massive tests.
    • Firefox 29 caches asm.js code: The second time you visit the same site, previously-compiled asm.js code will just be loaded from disk, avoiding any compilation pause at all. Another speedup in this version is that the previous float32 optimizations are fully optimized in asm.js code as well.

    Large codebases, and why we need a new benchmark

    Each of those features is expected to improve asm.js performance, so it makes sense to see large speedups there. So far, everything looks pretty much as we would expect. However, a fourth milestone is noted on that graph, and it doesn’t cause any speedup. That feature is IonMonkey, which landed in Firefox 18. IonMonkey was a new optimizing compiler for Firefox, and it provided very large speedups on most common browser benchmarks. Why, then, doesn’t it show any benefit in Massive?

    IonMonkey does help very significantly on small asm.js codebases. But in its original release in Firefox 18 (see more details in the P.S. below), IonMonkey did not do well on very large ones – as a complex optimizing compiler, compilation time is not necessarily linear, which means that large scripts can take very large amounts of time to compile. IonMonkey therefore included a script size limit – over a certain size, IonMonkey simply never kicks in. This explains why Massive does not improve on Firefox 18, when IonMonkey landed – Massive contains very large codebases, and IonMonkey at the time could not actually run on them.

    That shows exactly why a benchmark like Massive is necessary, as other benchmarks did show speedups upon IonMonkey’s launch. In other words, Massive is measuring something that other benchmarks do not. And that thing – large asm.js codebases – is becoming more and more important.

    (P.S. IonMonkey’s script size limit prevented large codebases from being optimized when IonMonkey originally launched, but that limit has been relaxed over time, and practically does not exist today. This is possible through compilation on a background thread, interruptible compilation, and just straightforward improvements to compilation speed, all of which make it feasible to compile larger and larger functions. Exciting general improvements to JavaScript engines are constantly happening across the board!)

  9. Matchstick Brings Firefox OS to Your HDTV: Be the First to get a Developer Stick

    The first HDMI streaming stick powered by Firefox OS has arrived. It’s called Matchstick and we’re looking for your help to create apps for this new device.

    Background

    Matchstick stems from a group of coders that spent way too much time mired in the guts of platforms such as Boot to Gecko, XBMC, and Boxee. When Google introduced Chromecast we were excited about the possibilities but ultimately were disappointed when they pulled back on the device’s ultimate promise – any content on any HD screen, anywhere, anytime.

    We decided to make something better and more open, and to accomplish this we had to choose an operating system that would become the bedrock for the adaptable and open-sourced platform that is Matchstick. That platform is Firefox OS, which allows us to build the first streaming stick free of any walled garden ecosystem.

    Matchstick and Firefox OS combined offers a totally open platform (both software and hardware), that lets developers explore content and applications from video to games, and bring it right into the living room. That’s right Developers! An open SDK means you can build out your own personalized streaming and interactive experiences without the need for approval or review.

    Apps for Matchsticks

    We have opened up a full developer site with access to everything you need to begin working with Matchstick. Support for Firefox OS will be available at launch, and we look forward to adding TV applications to the Firefox OS Marketplace. For now, we have included a full API library, of sender apps with support for Android and iOS, as well as receiver apps that are compatible with the Matchstick Receiver.

    When we say Apps for Matchsticks, we mean both sender and receiver apps. You can use the sender APIs to enable your Firefox OS, Android or iOS device to discover a Matchstick device, then communicate with your receiver app. It’s not difficult to embed sender APIs into existing apps or create new sender apps, please refer to the sample code we have included with the SDK.

    Matchstick sender apps typically follow this execution flow:

    1. Scan for Matchstick
      The sender app searches for Matchstick devices residing in the same Wi-Fi network as the sender device. The scanning reveals a friendly display name, model and manufacturer, icon, and the device’s IP address. Presented with a list, a user may then select a target device from all those discovered.
    2. Connect to Matchstick
      We support both TLS and NON-TLS communication between sender and receiver.
    3. Launch Receiver App
      The sender initiates a negotiation with the target device, launching a receiver app either with the URL of a HTML5 receiver app, or even a Chromecast App ID.
    4. Establish Message Channels
      With the receiver app now launched, Matchstick establishes message channels between the sender and receiver. In addition to a media control channel common to all Matchstick and Chromecast apps, you may establish any number of application-specific channels to convey whatever customized data that your app might require.

    The receiver app is a combination of HTML5, CSS and Javascript, loaded into a “receiver container” which is a certified app of Firefox OS. To use the Matchstick receiver APIs, you need only include fling_receiver.js in your app.

    Here is an example of a simple video receiver app:

    <!DOCTYPE html>
    <html>
        <head>
         <title>Example simplest receiver</title>
         <meta charset="UTF-8">
         <!--(need) include matchstick receiver sdk-->
         <script src="//fling.matchstick.tv/sdk/libs/receiver/2.0.0/fling_receiver.js"></script>
        </head>
        <body>
         <!--(need) Give a video tag-->
         <video id="media"></video>
         <script>
             // (need) Get video element
             window.mediaElement = document.getElementById("media");
             // (need) For media applications, override/provide any event listeners on the MediaManager.
             window.mediaManager = new fling.receiver.MediaManager(window.mediaElement);
     
             // (need) Get and hold the provided instance of the FlingReceiverManager.
             // Override/provide any event listeners on the FlingReceiverManager.
             window.flingReceiverManager = fling.receiver.FlingReceiverManager.getInstance();
     
             // (need) Call start on the FlingReceiverManager to indicate the receiver application is ready to receive messages.
             window.flingReceiverManager.start();
         </script>
        </body>
    </html>

    Matchsticks for Apps

    Rather than relying on emulators, we want to be sure developers can get their hands on Matchstick prototypes and start coding without delay. As a result, we are inviting app developers who will commit to building and porting apps for Firefox OS on Matchstick to apply for a free developer-preview device through our Matchsticks for Apps program.

    Similar to the phones-for-apps program launched by Mozilla, our Matchsticks for Apps program is aimed at developers who have built apps for Firefox OS, Chrome, Android, iOS …. Even for Chromecast! Let’s bring those visions to the big screen.

    image

    Developer (pre-production) version of Matchstick (pic by Christian Heilmann)

    We are now looking for good ideas, video content, new channels, as well as games, tools, utilities, pictures, even skins for the UI. If you have a plan to build apps or do something for Matchstick, please share your plan and we’ll send you a stick so you can start coding ASAP.

    Who Should Apply:

    • Those interested in building apps on a big screen TV
    • Those with existing Web or mobile apps, who would like to expand to the big screen
    • HDMI dongle developers, who want to build their own Matchstick
    • Chromecast developers, who want to port their apps to an open platform
    • You!

    Matchstick workshop in November

    On Tuesday, November 18th, we plan to host an invitation-only Firefox OS App Workshop for Matchstick in the Mozilla office in San Francisco. The enrollment form is open and we are accepting applications from qualified developers.

    Apply now for the San Francisco workshop!

    If you don’t live near San Francisco, don’t worry! We plan to offer several other app workshops in the near future and we’ll announce them here, of course!

    Happy Hacking!

  10. 350 posts on Hacks in 2 years!

    Two years ago, we made a number of changes to the Mozilla Hacks blog. Since then we’ve had over three million unique visitors and 350 quality posts in just less than two years – almost one every second day!

    Part of these changes included:

    • A clear focus on learning about the Open Web & open source – more detail in What Mozilla Hacks is
    • A dedicated Editor, me, working with ensuring consistency, quality & versatility of the articles
    • Articles covering both interesting technologies and possibilities but also learning lessons of how to build exciting solutions and work with a number of different technical opportunities
    • To be a credible and independent go-to resource for developers

    Testimonials

    I’ve gathered a few testimonials from authors behind the most popular articles during this time, with their experiences writing for Mozilla Hacks.

    And you know what? You could write for Hacks too!

    If you have experience with a great solution or idea for the Open Web/open source, just let me know and Mozilla can help you share it with a lot of developers out there! Send an e-mail to robert [at] mozilla [dot] com or talk to me via @robertnyman on Twitter and we’ll talk!

    Here are the thoughts from Peter Cooper of HTML5 Weekly & JavaScript Weekly, Dave Camp, Director of Firefox Developer Tools and Thorben Bochenek from Opera on writing for Mozilla Hacks.

    Austin Hallock from Clay.io:

    My co-founder Zoli and I have both written articles for Mozilla Hacks and we agree that it was a very worthwhile experience. The process of getting our posts on the site was easy and both articles generated more response than we expected. Mozilla has a fantastic mission, and the Mozilla Hacks blog is a great extension of that.

    Their articles:

    Also, thanks to the authors on Mozilla Hacks for their great contributions! And those in that page are only authors with 2 posts and more – in total we have 230 authors all time who have published a post on Hacks!

    Most popular content

    During these two years, these are the most popular posts, written by Mozilla staff:

    The most popular posts, written by 3rd party developers/writers:

    Read & write

    If you like our articles, please spread the word about them! Ask your friends to read too and follow @mozhacks on Twitter for the latest articles and announcements.

    And if you have experiences and learnings to share, let us know and help you with sharing that knowledge!