Mozilla

Articles

Sort by:

View:

  1. HTML out of the Browser

    Amongst my friends, I’m known as something of a Star Wars nerd. My longtime nick has been cfjedimaster (a combination of two passions, the other being ColdFusion), I work in a room packed to the gills with Star Wars toys, and I’ve actually gotten inked up twice now with Star Wars tats. That being said, it was another movie that had the most influence on my current career – Tron.

    Tron_poster

    I had already discovered an interest in computers before then, but it was Tron that really crystallized the idea for me. All of sudden I imagined myself being the programmer – creating intelligent programs and dominating the grid. Yeah, I know, I was a nerd back then too.

    My dreams, though, ran into reality rather quickly during my time as a Comp Sci major in college. First – I discovered that “Hey, I’m good at math!” means crap all when you hit Calculus 3, and secondly, I discovered that I really wasn’t that interested in the performance of one sort versus another. I switched to English as a major (always a good choice) but kept messing around with computers. It was during this time that I was exposed to Mosaic and the early web.

    I quickly jumped into the web as – well – I’ll put it bluntly – easier programming than what I had been exposed to before. I can remember LiveScript. I can remember my first Perl CGI scripts. This wasn’t exactly light cycle coding but it was simpler, fun, and actually cutting edge. I’ve spent a good chunk of my adult life now as a web developer, and while I certainly have delusions of the web being a pristine environment, it has been good to me (and millions of others) and I’m loving to see how much it evolves over time.

    One of the most fascinating ways that web technologies have grown is outside of the web itself. In this article I’m going to look at all the various ways we can reuse our web-based technologies (HTML, JavaScript, and CSS) in non-web based environments. While it would be ludicrous to say that one shouldn’t learn other technologies, or that web standards work everywhere and in every situation, I feel strongly that the skills behind the web are ones open to a large variety of people in different disciplines – whether or not you got that PhD in computer science!

    Mobile

    This is typically the point where I’d discuss how important mobile is, but it’s 2014 and I think we’re past that now. Mobile development has typically involved either Java (Android) or Objective-C (iOS). Developers can also use web standards to build native applications. One solution is Apache Cordova (AKA PhoneGap).

    Cordova uses a web view wrapped in a native application to allow web developers to build what are typically referred to as hybrid applications. Along with providing an easy way to get your HTML into an app, Cordova provides a series of different plugins that let you do more than what a typical web page can do on a device. So for example, you have easy access to the camera:

    navigator.camera.getPicture(onSuccess, onFail, { quality: 50,
        destinationType: Camera.DestinationType.DATA_URL
    });
     
    function onSuccess(imageData) {
        var image = document.getElementById('myImage');
        image.src = "data:image/jpeg;base64," + imageData;
    }
     
    function onFail(message) {
        alert('Failed because: ' + message);
    }

    You can also work with the accelerometer, GPS, contacts, the file system, and more. Cordova provides a JavaScript API that handles the communication to native code in the back end. Best of all, the same code can be used to build native applications for multiple different platforms (including Firefox OS, now supported in Cordova & PhoneGap).

    To be clear, this isn’t a case of being able to take an existing web site and just package it up. Mobile applications are – by definition – completely different from a simple web site. But the fact that you can use your existing knowledge gives the developer a huge head start.

    Another example of this (and one that is hopefully well known to readers of this site) is Firefox OS. Unlike Cordova, developers don’t have to wrap their HTML inside a web view wrapper. The entire operating system is web standards based. What makes Firefox OS even more intriguing is support for hosted applications. Firefox OS is a new platform, and the chance that your visitors are using a device with it is probably pretty slim. But with a hosted application I can easily provide support for installation on the device while still running my application off a traditional URL.

    Consider a simple demo I built called INeedIt. If you visit this application in Chrome, it just plain works. If you visit it in a mobile browser on Android or iOS – it just works. But visit it with Firefox and code will trigger to ask if you want to install the application. Here is the block that handles this.

    if(!$rootScope.checkedInstall && ("mozApps" in window.navigator)) {
     
        var appUrl = 'http://'+document.location.host+'/manifest.webapp';
        console.log('havent checked and can check');
        var request = window.navigator.mozApps.checkInstalled(appUrl);
     
        //silently ignore
        request.onerror = function(e) {
            console.log('Error checking install '+request.error.name);
        };
     
        request.onsuccess = function(e) {
            if (request.result) {
                console.log("App is installed!");
            }
            else {
                console.log("App is not installed!");
                if(confirm('Would you like to install this as an app?')) {
                    console.log('ok, lets try to install'); 
                    var installRequest = window.navigator.mozApps.install(appUrl);
     
                    installRequest.onerror = function() {
                        console.log('install failure: '+this.error.name);
                        alert('Sorry, install failed.');    
                    };
     
                    installRequest.onsuccess = function() {
                        console.log('did it');
                        alert('Thanks, app installed!');    
                    };
                }
                $rootScope.checkedInstall=true;
            }
        };
     
    } else {
        console.log('either checked or non compat');                
    }

    Pretty simple, right? What I love about this is that the code is 100% ignored outside of Firefox OS but automatically enhanced for folks using that operating system. I risk nothing – but I get the benefit of providing them a way to download and have my application on their device screen.

    FF OS prompting you to install the app

    Desktop

    Of course, there are still a few people who sit in front of a gray box (or laptop) for their day to day work. Many desktop applications have been replaced by web pages, but there are still things that outside the scope of web apps. There are still times when a desktop app makes sense. And fortunately – there’s multiple ways of building them with web standards as well.

    So you know the code example I just showed you? The one where Firefox OS users would be given a chance to install the application from the web page? That exact same code works on the desktop as well. While still in development (in fact, the application I built doesn’t work due to a bug with Geolocation), it will eventually allow you to push your web based application both to a mobile Firefox OS user as well as the other billion or so desktop users. Here’s the application installed in my own Applications folder.

    INeedIt as a desktop app

    As I said though – this is still relatively new and needs to bake a bit longer before you can make active use of it. Something you can use right now is Node Webkit. This open source project allows you to wrap Node.js applications in a desktop shell. Executables can than be created for Windows, Mac, and Linux. You get all the power of a “real” desktop application with the ease of use of web standards as your platform. There’s already a growing list of real applications out there making use of the framework – some I had even used before what realizing they used Node Webkit behind the scenes.

    As an example, check out A Wizard’s Lizard, a RGP with random dungeons and great gameplay.

    Screen shot - Wizard's Lizard

    Native App Extensions

    In the previous section we covered applications built with web standards. There are also multiple applications out there today, built natively, that can be extended with web standards. As a web developer you are probably already familiar with Firefox Add-Ons and Chrome extensions. There is an incredibly rich ecosystem of browser extensions for just about any need you can think of. What interests me however is the move to use web standards to open other products as well.

    Did you know Photoshop, yes, Photoshop, now has the ability to extended with Node.js? Dubbed “Adobe Generator”, this extensibility layer allows for a script-based interface to the product. One example of this is the ability to generate web assets from layers based on a simple naming scheme. If you’ve ever had to manually create web assets, and update them, from a PSD, you will appreciate this. The entire feature though is driven by JavaScript and all makes use of a public API you can build upon. The code, and samples, are all available via GitHub.

    Generator running within Photoshop

    What Next?

    Coming from the perspective of someone who has been in this industry for way too long, I can say that I feel incredibly lucky that web standards have become such a driving force of innovation and creativity. But it is not luck that has driven this improvement. It is the hard work of many people, companies, and organizations (like Mozilla) that have created the fertile landscape we have before us today. To continue this drive requires all of us to become involved, evangelize to others, and become proponents of a web created by everyone – for everyone.

  2. Powerful tools for developing Web Apps

    In the recent years, web development changed drastically. The emergence of the mobile web and the new form factor of smart phones created the demand for different solutions than the former desktop-only web.

    Since then a lot of frameworks and tools have been created, with new ones being added almost weekly. Now, we web developers are faced with a different problem: for every development concern, there are multiple options to consider, without clear pros or cons. It is easy to feel intimidated not only by the choices available, but also by how similar those choices are.

    Every day, web developers have to successfully overcome this issue and turn this diversity from a daunting proposition into an empowering one.

    But why reinvent the wheel all the time?

    A set of solid recommendations for app developers

    Mozilla is putting together a core set of tools and recommendations that we believe are the most useful for making Web apps.

    The key considerations for what we might recommend are:

    • Sufficiently well-documented and straightforward to use for an average developer (we will concisely document the required knowledge one needs in order to engage with the technology).
    • Loosely coupled and as modular as possible (so you can follow one recommendation but not another if you are so inclined).
    • Tested on Mozilla products (i.e. the UI components will perform well on Firefox OS, etc), but with cross-platform apps in mind

    The upcoming, initial set of recommendations involves a toolchain that’s core to any modern web app, like a JavaScript framework, templating interface, UI framework and task runner. We will employ existing solutions wherever possible and write libraries or utilities to fill in the gaps.

    On an ongoing basis, we’ll expand this systematically across the different parts of the development experience, such as offline handling or the use of various Web APIs.

    All of this will be delivered in one central spot: The App Center on the Mozilla Developer Network (MDN), our established, renowned resource for web app development.

    But I already have it all figured out!

    We encourage you to share your success story with us. Heck, share your failures too. We need all the feedback we can get from everyday web developers. If you have a tool chain of your own, with your favorite JavaScript framework, etc., we’re not interested in converting you to something else. We’re trying to help developers who aren’t sure how to go about making these kinds of decisions.

    Come join us!

    In the Mozilla tradition, this is a community-driven process. This means your input is encouraged and appreciated, and we would like your help to make this a successful initiative!

    The main discussions around this will happen in the following places:

    For the initial tool chain recommendations, I started a thread on the mailing list already, go ahead and weigh in.

    If you have ideas for topics worth exploring in future iterations, don’t be shy and open a new thread to get the discussion started.

    What’s next?

    If this whets your appetite, then great! 2014 is an exciting year to be a web app developer. We’ll keep you updated here on Hacks, as well as the MDN App Center over the coming months.

  3. Introducing PredictionIO

    PredictionIO is an open source machine learning server for software developers to create predictive features, such as personalization, recommendation and content discovery. Building a production-grade engine to predict users’ preferences and personalize content for them used to be time-consuming. Not anymore with PredictionIO’s latest v0.7 release.

    We are going to show you how PredictionIO streamlines the data process and make it friendly for developers and production deployment. A movie recommendation case will be used for illustration purpose. We want to offer “Top 10 Personalized Movie Recommendation” for each user. In addition, we also offer “If you like this movie, you may also like these 10 other movies….”.

    Prerequisite

    First, let’s explain a few terms we use in PredictionIO.

    Apps

    Apps in PredictionIO are not apps with program code. Instead, they correspond to the software application that is using PredictionIO. Apps can be treated as a logical separation between different sets of data. An App can have many Engines, but each Engine can only be associated with one App.

    Engines

    Engines are logical identities that an external application can interact with via the API. There are two types of Engines as of writing: item-recommendation, and item-similarity. Each Engine provide unique settings to meet different needs. An Engine may have only one deployed Algorithm at any time.

    Algorithms

    Algorithms are actual computation code that generates prediction models. Each Engine comes with a default, general purpose Algorithm. If you have any specific needs, you can swap the Algorithm with another one, and even fine tune its parameters.

    Getting Hands-on

    Preparing the Environment

    Assuming a recent 64-bit Ubuntu Linux is installed, the first step is to install Java 7 and MongoDB. Java 7 can be installed by sudo apt-get install openjdk-7-jdk. MongoDB can be installed by following the steps described in Install MongoDB on Ubuntu.

    Note: any recent 64-bit Linux should work. Distributions such as ArchLinux, CentOS, Debian, Fedora, Gentoo, Slackware, etc should all work fine with PredictionIO. Mac OS X is also supported.

    Java 7 is required to run PredictionIO and its component Apache Hadoop. Apache Hadoop is optional since PredictionIO 0.7.0.

    MongoDB 2.4.x is required for PredictionIO to read and write behavioral data and prediction model respectively.

    Installing PredictionIO and its Components

    To install PredictionIO, simply download a binary distribution and extract it. In this article, we assume that PredictionIO 0.7.0 has been installed at /opt/PredictionIO-0.7.0. When relative paths are used later in this article, they will be relative to this installation path unless otherwise stated.

    The installation procedure is outlined in Installing PredictionIO on Linux.

    To start and stop PredictionIO, you can use bin/start-all.sh and bin/stop-all.sh respectively.

    Fine Tuning Apache Hadoop (Optional)

    PredictionIO relies on Apache Hadoop for distributed data processing and storage. It is installed at vendors/hadoop-1.2.1. The following outlines some optimization opportunities.

    vendors/hadoop-1.2.1/conf/hdfs-site.xml

    dfs.name.dir and dfs.data.dir can be set to point at a big and persistent storage. Default values usually point at a temporary storage (usually /tmp), which could be pruned by the OS periodically.

    vendors/hadoop-1.2.1/conf/mapred-site.xml

    mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum control the maximum number of mapper and reducer jobs (data processing jobs). It is a good start to set the first value to be the number of CPU cores, and the second value to be half the first value. These should be reduced slightly to provide margin when you serve prediction in production settings if you do not run Hadoop on a separate machine.

    mapred.child.java.opts controls the Java Virtual Machine options for each mapper and reducer job. It usually is a good idea to reserve a good amount of memory even when all mapper and reducer jobs are running. If you have a maximum of 4 mappers and 2 reducers, a 1GB heap space (-Xmx1g) for each of them (a total of 6GB max) would be reasonable if your machine has more than 10GB of RAM.

    io.sort.mb controls the size of the internal sort buffer and is usually set to half of the child’s heap space. If you have set to 1GB above, this could be set to 512MB.

    Creating an App in PredictionIO

    This can be done by logging in PredictionIO’s web UI located at port 9000 on the server.

    The following is the first screen after logging in and clicking on the “Add an App” button. To add an app, simply input the app’s name and click “Add”.

    Importing Data to PredictionIO

    Before importing data, the app key of the target app must be obtained. This was done by clicking on “Develop” next to an app. The screen below is what can be seen afterwards.

    The app key for the “ml100k” app is NO5FpSnGh1F3PUi7xiSGc910q63z6rxZ4BcYe039Jjf2sTrqyjOc1PmQ3MJowNwM, as shown above. The app key is unique and your app key should be different.

    To parse the MovieLens 100k data set and import to PredictionIO, one can use the import_ml.rb Gist. It requires the PredictionIO Ruby gem, which can be installed by sudo gem install predictionio.

    Importing the whole set of data is simple:

    ruby import_ml.rb NO5FpSnGh1F3PUi7xiSGc910q63z6rxZ4BcYe039Jjf2sTrqyjOc1PmQ3MJowNwM u.data
    

    u.data is a file from the MovieLens 100k data set, which can be obtained from the GroupLens web site.

    Adding Engines

    Two engines can be added by simply clicking on the “Add an Engine” button. In our case, we have added “itemrec” and “itemsim” engines. Once they are added, they will commence training automatically with a default hourly schedule.

    Getting Prediction Results

    At the moment, you may access prediction results from these two sample URLs (API server at port 8000):

    “Top 10 Personalized Movie Recommendation”:

    http://localhost:8000/engines/itemrec/itemrec/topn.json?pio_appkey=NO5FpSnGh1F3PUi7xiSGc910q63z6rxZ4BcYe039Jjf2sTrqyjOc1PmQ3MJowNwM&pio_n=10&pio_uid=1

    “If you like this movie, you may also like these 10 other movies….”:

    http://localhost:8000/engines/itemsim/itemsim/topn.json?pio_appkey=NO5FpSnGh1F3PUi7xiSGc910q63z6rxZ4BcYe039Jjf2sTrqyjOc1PmQ3MJowNwM&pio_n=10&pio_iid=1

    You may change the pio_uid and pio_iid parameters above to see results for other users/items. You are encouraged to take advantage of these endpoints to verify these results. You can also take advantage of our official and community-powered SDKs to simplify this process. Contributed SDKs are also available.

    What’s News in PredictionIO v0.7

    With the new extended support to GraphChi – a disk-based large-scale graph computation framework, developers can now evaluate and deploy algorithms in both GraphChi and Apache Mahout on one single platform!

    Enjoy!

  4. Measuring power consumption on phones

    While learning about and measuring what happens on phones, we’re learned a great deal around power consumption. Therefore we want to share some learnings and what have resulted in the FxOS Powertool!

    Introducing the FxOS Powertool!

    With the FxOS Powertool!, we can optimize apps for power consumption, but also verify and fix bugs related to that. It is a command line utility with a number of options:

    usage: powertool [-h] -d {yocto,mozilla} [-p PATH] -u {tk,cli} [-f FILE]
                     [-o OUT] [-s SHOW]
     
    Mozilla Powertool
     
    optional arguments:
      -h, --help            show this help message and exit
      -d {yocto,mozilla}, --device {yocto,mozilla}
                            specify ammeter device to use
      -p PATH, --path PATH  specify path to ammeter device (e.g. /dev/ttyACM0)
      -u {tk,cli}, --ui {tk,cli}
                            specify which UI to use
      -f FILE, --file FILE  test run config file
      -o OUT, --out OUT     output data file
      -s SHOW, --show SHOW  name of the sample source to display

    Get the code by cloning the repository:

    $ git clone git://github.com/JonHylands/fxos-powertool

    The FxOS Powertool! is written in Python and uses the Tkinter UI package and should be cross-platform. Then install the application and dependencies:

    $ cd fxos-powertool
    $ sudo python ./setup.py install

    The next step is to create a test suite description file. The FxOS Powertool! uses the description file to know what tests you plan on running and it organizes the collected data under each test. A test suite description file looks like:

    {
      "title": "My Tests",
      "tests": [
        "My first test",
        "My second test",
        "My third test"
      ]
    }

    Building a battery harness

    This is what you need:

    Next you need to set up your hardware:

    1. Pull the battery from your FxOS device & place it in the battery harness

    2. Put the battery harness into your device

    3. Plug in your ammeter

    Yocto Ammeter

    Running FxOS Powertool!

    Now you’re ready to start the FxOS Powertool! and begin collecting data.

    With the FxOS Powertool!, you can save the collected data in two different formats: JSON and comma separated values. The format is determined from the file extension of the file specified in the --out option. For example:

    • My test suite description file is called mytests.json
    • I’m using a Yoctopuce ammeter
    • I want the Tk GUI to show the current consumption graph
    • I want the collected data to be saved in .csv files

    Then my command line would look like this:

    $ powertool -d yocto -p /dev/ttyUSB0 -u tk -s current -f mytests.json -o mytests.csv

    If I were to use the Mozilla ammeter instead, with the same test parameters:

    $ powertool -d mozilla -p /dev/ttyACM0 -u tk -s current -f mytests.json -o mytests.csv

    The Tk GUI should start up and the title of your first test will be shown in the title bar of the app:

    Powertool Launch

    To control the collection of data and navigate through the tests, FxOS Powertool! uses keyboard shortcuts. With the GUI open, you can start/stop collecting data for the currently selected test by hitting the space bar. The first time you hit the spacebar, data collection begins; to stop data collection, just hit the spacebar again. Each data collection run will be recorded as a row of data in the output files:

    Powertool Test One Data Run

    You can repeat data runs for a given test as many times as you like, navigate to another test or tests, do data runs for those test, then come back to a previous test and continue doing those data runs. No data will be lost in the FxOS Powertool! will not lose any data and it will, for each test, aggregate the data runs into the correct data files. Use the ‘L’ and ‘H’ keys on the keyboard for navigating between tests. ‘L’ moves to the next test in the list, ‘H’ moves to the previous test.

    Powertool Test Two Data Run

    When you have collected all of your test data, the data files are written to disk when you quit the application: do so by either clicking on the close window button in the title bar or pressing the ‘Q’ button on the keyboard. In this example, the data was written out in CSV format by giving it “mytests.csv” as the –out paramter value. The application will create a “mytests” directory and then write out a CSV file for each test with all of the data gathered for the test.

    CSV File Open in a Spreadsheet Application

    Like we’ve gone through in this article, the FxOS Powertool! will do some very basic statistics calculations for you. Along with the raw data, it also calculates the minimum and maximum value, mean, and standard deviation for all of the data collected for the test.

  5. Coordinate Conversion Made Easy – the power of GeometryUtils

    In a previous post we introduced the GeometryUtils interface and the getBoxQuads() API for retrieving the CSS box geometry of a DOM node. GeometryUtils also takes care of another important problem: converting coordinates reliably from one DOM node to another. For example, you might want to find the bounding-box of one element relative to another element, or you might want to convert event coordinates from the viewport to some arbitrary element.

    Existing APIs

    Until now, simple cases could be handled using getBoundingClientRect() and some math, but complex cases (e.g. involving CSS transforms) were almost impossible to handle using standard APIs. The nonstandard APIs webkitConvertPointToPage and webkitConvertPageToPoint are a big improvement, but apart from not being standardized, they’re not as powerful as they need to be. In particular it’s more convenient and more robust to provide an API for directly converting coordinates from one element to another.[1]

    New APIs

    GeometryUtils introduces three new methods for coordinate conversion:

    • to.convertPointFromNode(point, from) converts a a point relative to the top-left of the first border-box of “from” to a point relative to the top-left of the first border-box of “to”. The point is a DOMPointInit, which means you can pass a DOMPoint or a JS object such as {x:0, y:0}.
    • to.convertRectFromNode(rect, from) converts a a DOMRect relative to the top-left of the first border-box of “from” to a DOMQuad relative to the top-left of the first border-box of “to” by converting the vertices of the DOMRect. It converts to a DOMQuad to ensure that the result is accurate even if it needs to be rotated or skewed by CSS transforms.
    • to.convertQuadFromNode(quad, from) converts a DOMQuad from “from” to “to”. It’s just like convertRectFromNode except for taking a DOMQuad.

    As with getBoxQuads, a node can be an Element, TextNode or Document; when a Document is used, the coordinates are relative to the document’s viewport.

    Example:

    <div id="d" style="position:absolute; transform:rotate(45deg); left:100px; top:100px; width:100px; height:100px;"></div>
    <div id="e" style="position:absolute; left:100px; top:100px; width:100px; height:100px;"></div>
    var p1 = document.convertPointFromNode({
        x:0, y:0
      }, document.getElementById("e")
    );
    // p1.x == 100, p1.y == 100
     
    var p2 = document.convertPointFromNode({
        x:0, y:0
      }, document.getElementById("d")
    );
    // p2.x == 150, p2.y == 150 - 50*sqrt(2) (approx)
     
    p2 = document.getElementById("e").convertPointFromNode({
        x:0, y:0
      }, document.getElementById("d")
    );
    // p2.x == 50, p2.y == 50 - 50*sqrt(2) (approx)
     
    var q1 = document.convertRectFromNode(
      new DOMRect(0, 0, 50, 50), 
      document.getElementById("e")
    );
    // q1.p1.x == 100, q1.p1.y == 100
    // q1.p2.x == 150, q1.p2.y == 100
    // q1.p3.x == 150, q1.p3.y == 150
    // q1.p4.x == 100, q1.p4.y == 150
     
    var q2 = document.convertQuadFromNode(
      new DOMQuad({
        x:60, y:50
      }, {
        x:90, y:50
      }, {
        x:100, y:100
      }, {
        x:50, y:100
      }), 
      document.getElementById("e")
    );
    // q2.p1.x == 100, q2.p1.y == 100
    // q2.p2.x == 150, q2.p2.y == 100
    // q2.p3.x == 140, q2.p3.y == 150
    // q2.p4.x == 110, q2.p4.y == 150
    p1
    p2

    Sometimes it’s useful to convert to or from an element’s CSS content-box, padding-box or margin-box. This is supported via an optional ConvertCoordinateOptions dictionary with the following options:

    • fromBox: one of "content", "padding", "border" or "margin", selecting which CSS box of the first fragment of the from node the input point(s) are relative to.
    • toBox: selects which CSS box of the first fragment of the to node the returned point(s) are relative to.

    As a special case, this makes it easy to convert points between different
    CSS box types of the same element. For example, to convert a point from an
    element’s border-box to be relative to its content-box, use
    element.convertPointFromNode(point, element, {toBox:"content"}).

    Example:

    <div id="e" style="position:absolute; padding:20px; left:100px; top:100px; width:60px; height:60px;"></div>
    var p1 = document.convertPointFromNode({
        x:0, y:0
      }, document.getElementById("e"), 
      {fromBox:"content"}
    );
    // p1.x == 120, p1.y == 120
     
    p1 = document.getElementById("e").convertPointFromNode({
        x:120, y:120
      }, document,
      {toBox:"content"}
    );
    // p1.x == 0, p1.y == 0
     
    p1 = document.getElementById("e").convertPointFromNode({
        x:0, y:0
      }, document.getElementById("e"), 
      {fromBox:"content"}
    );
    // p1.x == 20, p1.y == 20
     
    p1 = document.getElementById("e").convertPointFromNode({
        x:20, y:20
      }, document.getElementById("e"), 
      {toBox:"content"}
    );
    // p1.x == 0, p1.y == 0
    p1
    e content-box
    e border-box

    These APIs are available in Firefox nightly builds and should be released in Firefox 31. Firefox is the first browser to implement these APIs.

    Footnote

    [1] Consider the following example:

    <div style="transform:scale(0)">
      <div id="a">...<>
      <div id="b">...<>
    </div>

    In this case, converting a point relative to a to be relative to b by converting first to page coordinates and then back to b doesn’t work, because the scale(0) maps every point in a to a single point in the page.

  6. Rormix – Discover Emerging Music Videos with Firefox OS

    Rormix is a platform for discovering emerging music videos. Music videos are tagged by genre and similar commercial artists, making it easy to discover new music videos.

    The Rormix app was made using PhoneGap and released on iOS and Android. Development took just over a month from the first line of code, to the app submissions in the app stores. The Firefox OS port took one developer just one day!

    Listed below are a few things we learnt along the way:

    What screen sizes am I developing for?

    When you develop an open web app you can install it in the actual desktop browser, the Android Firefox browser or Firefox OS devices.

    If you want to support all of them in one app, responsive designs are a must (you can also select just the platform you want to support). The current crop of Firefox OS phones have a resolution of 320×480. They have a pixel density of 1 so no special graphics need to be produced.

    Back Button?

    iOS devices don’t have a back button, Android devices have a hardware back button, so where does Firefox OS stand? It has a software back button that you can optionally hide or show when building the manifest for the app. The back button can be hidden at the bottom of the screen however it can be hard to press.

    I recommend that you build a back button into your app and hide the default one to make the app easier to navigate.

    //jQuery example
    $('.backbutton').click(function(){
        history.go(-1);
    });

    Stateful design

    As a back button has a presence in Firefox OS you need to build a stateful application in order to go back in state when the user presses the back button.
    A simple way to implement this is using one of the various JS frameworks that use fragment identifiers to load different states (e.g. Sammy JS).

    //jQuery example
     
    //Sammy app
    var app;
     
    $(function(){
     
        app = Sammy(function() {
            this.get('#/', function() {
                //Load default content
            });
            this.get('#/trending', function() {
                //Get trending content
            });
            this.get('#/fresh', function() {
                //Get fresh content
            });
        });
    });
     
    //Load the default content on app load
    app.run('#/');
     
     
    //Go to fresh content
    $('.freshbutton').click(function(){
        app.setLocation('#/fresh');
    });

    Creating a menu

    The trick with making menus for Firefox OS is to use CSS3 transforms for speed, but also making them simple enough to limit the redraw cycle when the menu comes into play. Firefox OS phones have the same width in reference pixels as all iPhones (at the time of writing), and the same pixel height as iPhones previous to the iPhone 5, so if you have a design that works for iOS then you’re all set.

    Adding some Firefox OS flavour

    There are a set of design guidelines that give you an idea of the colour scheme etc of the Firefox OS platform. They also detail how to make the icon for your app, the fonts used etc.

    Submitting your app

    When you have finished building your app you have a choice of how to submit it. You can package it up in a zip file:

    zip -r package.zip *

    You can send this zip to the Marketplace or you can host it yourself.

    The other option is to simply host the code as a web page (rather than zip it), and with a little extra JS prompt the user to download the app to their phone.

    Aside: Using PhoneGap / Cordova and HTML5

    Building web apps allows you to quickly and easily build cross platform apps. Even better, with responsive designs it can all be in one project. Advancing tools and workflows (Sass and Yeoman for example) makes developing apps even easier.

    PhoneGap / Cordova supports Firefox OS from version 3.4 (more information in Building Cordova apps for Firefox OS). The biggest advantage of using PhoneGap is that you only need to support a single codebase for all your apps. We all know some browsers have niggles, and PhoneGap has a built in merge mechanism that allows you to put platform specific code aside from the main code and it will merge them when building the app.

    PhoneGap also has a bunch of libraries for accessing native properties of the phone (native dialogue boxes for example) and this code is the same across all platforms, minimising duplicate code.

    The best thing about PhoneGap is the ability for you to create your own plugins, harnessing the power of mobile devices in a really easy way, effortlessly switching between JS and native mobile code.

    Contact:
    @pixelcodeUK

  7. Inside the Party Bus: Building a Web App with Multiple Live Video Streams + Interactive Graphics

    Gearcloud Labs is exploring the use of open technologies to build new kinds of shared video experiences. Party Bus is a demo app that mixes multiple live video streams together with interactive graphics and synchronized audio. We built it using a combination of node.js, WebSockets, WebRTC, WebGL, and Web Audio. This article shares a few things we learned along the way.

    User experience

    First, take a ride on the Party Bus app to see what it does. You need Firefox or Chrome plus a decent GPU, but if that’s not handy you can get an idea of the app by watching the example video on YouTube.

    Since the app uses WebRTC getUserMedia(), you have to give permission for the browser to use your camera. After it starts, the first thing you’ll notice is your own video stream mounted to a seat on the 3D bus (along with live streams from any other concurrent riders). In most scenes, you can manipulate the bus in 3D using the left mouse (change camera angle), scroll wheel (zoom in/out), and right mouse (change camera position). Also try the buttons in the bottom control bar to apply effects to your own video stream: from changing your color, to flipping yourself upside down, bouncing in your seat, etc.

    How party bus uses WebRTC

    Party Bus uses WebRTC to set up P2P video streams needed for the experience. WebRTC does a great job supporting native video in the browser, and punching out firewalls to enable peer connections (via STUN). But with WebRTC, you also need to provide your own signaler to coordinate which endpoints will participate in a given application session.

    The Party Bus app uses a prototype platform we built called Mixology to handle signaling and support the use of dynamic peer topologies. Note that many apps can simply use peer.js, but we are using Mixology to explore new and scalable approaches for combining large numbers of streams in a variety of different connection graphs.

    For example, if a rider joins a bus that already has other riders, the system takes care of building the necessary connection paths between the new rider and peers on the same bus, and then notifying all peers through a WebSocket that the new rider needs to be assigned a seat.

    Specifically, clients interact with the Mixology signaling server by instantiating a Mixology object

    var m = new Mixology(signalerURL);

    and then using it to register with the signaler

    m.register(['mix-in'], ['mix-out']);

    The two arguments give specific input and output stream types supported by the client. Typing inputs and outputs in this way allows Mixology to assemble arbitrary graphs of stream connections, which may vary depending on application requirements. In the case of Party Bus, we’re just using a fully connected mesh among all peers. That is, all clients register with the same input and output types.

    The signaler is implemented as a node.js application that maintains a table of registered peers and the connections among them. The signaler can thus take care of handling peer arrivals, departures, and other events — updating other peers as necessary via callback functions. All communications between peers and the signaler are implemented internally using WebSockets, using socket.io.

    For example, when a new peer is registered, the server updates the topology table, and uses a callback function to notify other peers that need to know about the new connection.

    m.onPeerRegistered = function(peer) { ... }

    In this function, peers designated to send streams initiate the WebRTC offer code. Peers designated to receive streams initiate the WebRTC answer code (as well as provide a callback function onAddStream() to be used when the new input stream is ready).

    In the case of Party Bus, it’s then up to the app to map the new video stream to the right seat in the 3D bus model, and from then on, apply the necessary 3D transforms using three.js. Similarly, if a rider leaves the bus, the system takes care of notifying other clients that a peer has exited, so they can take appropriate action to remove what would otherwise be a dead video stream in the display.

    Party Bus organizes the “riders” on a bus using an array of virtual screens:

    var vsArray = new Array(NUM_SEATS);

    After registering itself with Mixology, the app receives a callback whenever a new peer video stream becomes available for its bus instance:

    function onAddStream(stream, peerId) {
        var i = getNextOpenSeat();
        vsArray[i] = new VScreen(stream, peerId);
    }

    The Party Bus app creates a virtual screen object for every video stream on the current bus. The incoming streams are associated with DOM video objects in the virtual screen constructor:

    function VScreen(stream, id) {
        var v = document.createElement(‘video’);
        v.setAttribute(“id”, “monitor:+ id);
        v.style.visibility = “hidden”;
        v.src = window.URL.createObjectURL(stream);  // binds stream to dom video object
        v.autoplay =true;
        document.body.appendChild(v);
    }

    Movie or app?

    Party Bus uses three.js to draw a double decker bus, along with virtual screens “riding” in the seats. The animation loop runs about two minutes, and consists of about a dozen director “shots”. Throughout the demo, the individual video screens are live, and can be manipulated by each rider. The overall sequence of shots is designed to change scene lighting and present other visual effects, such as bus thrusters which were created with the particle engine of Stemkoski.

    Party Bus is a web app, but the animation is programmed so the user can just let it run like a movie. The curious user may try to interact with it, and find that in most scenes it’s also possible to change the 3D view. However, in shots with a moving camera or bus, we found it necessary to block certain camera translations (movements in x, y, z position), or rotations (turning on x, y, z axis) — otherwise, the mouse will “fight” the program, resulting in a jerky presentation.

    But most of the fun in Party Bus is just hamming it up for the camera, applying visual effects to your own stream, and looking for other live riders on the bus.

    More info

    For more information on the Party Bus app, or to stay in the loop on development of the Mixology platform, please check out www.gearcloudlabs.com.

  8. Introducing the getBoxQuads API

    Web developers often need to determine where an element has been placed in the page, or more generally, where it is relative to another element. Existing APIs for doing this have significant limitations. The new GeometryUtils interface and its supporting interfaces DOMPoint, DOMRect and DOMQuad provide Web-standard APIs to address these problems. Firefox is the first browser to implement these APIs; they are available in Firefox 31 Nightly builds.

    Current best standardized APIs for retrieving element geometry

    Currently the best standardized DOM APIs for retrieving element geometry are element.getBoundingClientRect() and element.getClientRects(). These return the border-box rectangle(s) for an element relative to the viewport of the containing document. These APIs are supported cross-browser but have several limitations:

    • When complex CSS transforms are present, they return the smallest axis-aligned rectangle enclosing the transformed border-box. This loses information.
    • There is no way to obtain the coordinates of the content-box, padding-box or border-box. In simple cases you can add or subtract computed style values from the results of getBoundingClientRect()/getClientRects() but this is clumsy and difficult to get right. For example, when a <span> breaks into several fragments, its left border is only added to one of the fragments — either the first or the last, depending on the directionality of the text.
    • There is no way to obtain box geometry relative to another element.

    Introducing getBoxQuads()

    The GeometryUtils.getBoxQuads() method, implemented on Document, Element and TextNode, solves these problems. It returns a list of DOMQuads, one for each CSS fragment of the object (normally this list would just have a single
    DOMQuad).

    Example:

    <div id="d" 
      style="position:absolute; left:100px; top:100px; width:100px; height:100px;">
    </div>
    var quads = document.getElementById("d").getBoxQuads();
    // quads.length == 1
    // quads[0].p1.x == 100
    // quads[0].p1.y == 100
    // quads[0].p3.x == 200
    // quads[0].p3.y == 200
    p1
    p2
    p3
    p4

    Using bounds

    A DOMQuad is a collection of four DOMPoints defining the corners of an arbitrary quadrilateral. Returning DOMQuads lets getBoxQuads() return accurate information even when arbitrary 2D or 3D transforms are present. It has a handy bounds attribute returning a DOMRectReadOnly for those cases where you just want an axis-aligned bounding rectangle.

    For example:

    <div id="d" 
      style="transform:rotate(45deg); position:absolute; left:100px; top:100px; width:100px; height:100px;">
    </div>
    var quads = document.getElementById("d").getBoxQuads();
    // quads[0].p1.x == 150
    // quads[0].p1.y == 150 - 50*sqrt(2) (approx)
    // quads[0].p3.x == 150
    // quads[0].p3.y == 150 + 50*sqrt(2) (approx)
    // quads[0].bounds.width == 100*sqrt(2) (approx)
    p1
    p2
    p3
    p4
    bounds

    Passing in options

    By default getBoxQuads() returns border-boxes relative to the node’s document viewport, but this can be customized by passing in an optional
    options dictionary with the following (optional) members:

    • box: one of "content", "padding", "border" or "margin", selecting which CSS box type to return.
    • relativeTo: a Document, Element or TextNode; getBoxQuads() returns coordinates relative to the top-left of the border-box of that node (the border-box of the first fragment, if there’s more than one fragment). For documents, the origin of the document’s viewport is used.

    Example:

    <div id="d" 
      style="position:absolute; left:100px; top:100px; width:150px; height:150px;">
        <div id="e" 
          style="position:absolute; padding:20px; left:0; top:0; width:100px; height:100px;">
        </div>
    </div>
    var quads = document.getElementById("e").getBoxQuads({
      relativeTo:document.getElementById("d")
    });
    // quads[0].p1.x == 0
    // quads[0].p1.y == 0
     
    quads = document.getElementById("e").getBoxQuads({
      relativeTo:document.getElementById("d"), 
      box:"content"
    });
    // quads[0].p1.x == 20
    // quads[0].p1.y == 20
    d
    e content-box
    e border-box

    The relativeTo node need not be an ancestor of the node receiving getBoxQuads(). The nodes can even be in different documents, although they must be in the same toplevel browsing context (i.e. browser tab).

    Scratching the surface

    If you’ve read this far, you’re probably observant enough to have noticed additional methods in GeometryUtils — methods for coordinate conversion. These will be covered in a future blog post.

  9. Building a persistent Notes app for Firefox OS

    In this tutorial we will be building a notes app (like Evernote) from scratch and deploy it to Firefox OS! See a live demo.

    A persistent notes app needs a place to store all the notes for a user (so no one else can read it). For this we will use my own backend solution called Sproute. Sproute is a hosted web framework for quickly building dynamic webapps.

    First create an account with a unique subdomain then access the Dashboard through http://<mysubdomain>.sproute.io/admin. You must login with the same details used to signup to Sprotue.

    Models

    In Sproute there is a concept called Models. A model defines the dynamic data in our app with properties for data integrity. Create a new model called notes with the following fields and properties:

    • body: Text, Required, Min: 1
    • sharing: Text, Allowed Values: public, private, Default Value: private
    The model form filled out for notes

    The model form filled out for notes

    The body field will store the contents of the note. The sharing field will specify whether the note can be viewed by others (with a direct link) or just the owner. This is all the data we need to define, everything else is covered by built-in fields.

    List notes

    We need a way to list the available notes for a user. This can be done with Pages. A page is HTML with embedded template tags for retrieving and processing data.

    Open the index page. This is the home page when people view your space. We will list the notes here. Add the following:

    {{ get /data/notes?sort=_lastUpdated,desc as notes }}

    All data is retrieved through an HTTP interface so the {{ get }} tag must take a URL. The request above is retrieving all notes for the logged in user and stores the results in a variable called notes. We use a query parameter to sort the results by last modified first (An underscore denotes a built-in field).

    Let’s display each note in a list:

    <ul>
        {{ each notes as note }}
            <li><a href="/view/{{note._id}}">
            {{ word note.body 0 5 }}
            </a></li>
        {{ / }}
    </ul>

    The {{ word }} template tag will extract the first 5 words from the body content. We link to a not-yet-made page with the URL /view/<id>. note._id is a built-in unique identifier.

    Add the above code in the 'index' page

    Add the above code in the ‘index’ page

    Create a note

    Before creating the view page for a note, let’s make a new page for creating a note. Create a new page called create. Add the following HTML:

    <form action="/data/notes?goto=/" method="post">
        <div><textarea name="body"></textarea></div>
        <div><select name="sharing">
            <option value="private">Private</option>
            <option value="public">Public</option>
        </select></div>
        <button type="submit">Create Note</button>
    </form>

    Simple! As mentioned above, all data is retrieved and modified through an HTTP interface. This means we can use a simple HTML form to create a new note. The goto query parameter will redirect the user to that URL.

    Because this is a new page we need to create a new route so the page is accessible. A route is a pattern for a requested URL that will render a page if matched. There will already be a route for the index page. Click Routes and create a new one with the following:

    • Route: /create, Page: create.
    New route to 'create' page

    New route to ‘create’ page

    User login and registration

    User accounts are built into Sproute but we still need to create a Register and Login page. Thankfully this is also a simple HTML form so create a new page called users and add the following:

    <h2>Login</h2>
    <form action="/api/login?goto=/" method="post">
        <label>
            <div>Username:</div>
            <input type="text" name="name" />
        </label>
        <label>
            <div>Password:</div>
            <input type="password" name="pass" />
        </label>
        <button type="submit">Login</button>
    </form>

    You may add the Register form on the same page by copying the login form and replace the action attribute to /api/register. Create two new routes with the following:

    • Route: /login, Page: users
    • Route: /register, Page: users

    View and update a note

    Previously we created a link to view a note. This will also be where we let users modify the note. Create a new page called view and add the following:

    {{ get /data/notes/_id/:params.id?single=true admin as note }}
     
    {{ if note.sharing eq private }}
        {{ if note._creator neq :session.user._id }}
            {{ error This note is private }}
        {{ / }}
    {{ / }}
     
    <form action="/data/notes/_id/{{params.id}}?goto=/" method="post">
        <div><textarea name="body">{{note.body}}</textarea></div>
        <div><select name="sharing">
           <option value="private" selected>Private</option>
           <option value="public">Public</option>
        </select></div>
        <button type="submit">Update Note</button>
    </form>

    We make a request to get the note data for the note that was passed into the URL (through the params object). The query parameter single=true will return just the object instead of a one item collection. admin will run the request with the specified user-type. Next we check whether the note is private. If so throw an error if the user is not the creator.

    The update form is very similar to the create form. We only need to change the action to update notes where the _id matches the one in the URL.

    This page requires a slightly complex route. We need to use a placeholder in the route so that /view/anything will still match the view page and store anything in a variable.

    Create a route with the following:

    • Route: /view/:id, Page: view

    Now you can see where params.id comes from. params is an object for all placeholders in the route and the matched values.

    Permissions

    The last important concept of Sproute is permissions. Permissions are just like routes where a requested URL matches a pattern but instead of pointing to a page, it validates against a required user type.

    Click Permissions and add the following:

    • Method: GET, Route: /data/notes, User: Owner
    • Method: GET, Route: /data/notes/*, User: Owner

    This will make sure the only notes listed will be ones the user has created (i.e. the owner). Because of the second permission, we needed to run the {{ get }} request as admin otherwise it would fail for everyone (except for admins and the creator) even if the note is public.

    Adding a permission to retrieving notes

    Adding a permission to retrieving notes

    Firefox OS support

    Sproute can very easily support hosted apps on Firefox OS by creating a page with the manifest JSON data and the route: /manifest.webapp.

    Create a new page called manifest with the following:

    {
        "name": "{{self.name}}",
        "description": "A persistent notes app",
        "launch_path": "/",
        "icons": {
            "128": "/absolute/path/to/icon"
        }
    }

    Create a Route with the pattern /manifest.webapp pointing to the manifest page.

    We’re done! Test this in the app manager by using the link to the manifest (http://notes.sproute.io/manifest.webapp). Sproute is open-source and on GitHub under the Mozilla Public License v2 so you may clone the repo for private hosting.

    Notes app at mobile width

    Notes app at mobile width

  10. Box model highlighter, Web Console improvements, Firefox OS HUD + more – Firefox Developer Tools Episode 30

    Firefox 30 was just uplifted to the Aurora release channel, so let’s take a look at the most important DevTools changes in this release.

    Inspector

    One of our most requested features has been to highlight box model regions of elements on the page. We are happy to report that this feature has landed in Firefox 30. One of the great things is that the colors of the box model highlighter match the box model diagram found in the right pane of the inspector more clearly than before.

    Check out the inspector documentation to read more about the new functionality, or just see the screenshot and short video below:

    Inspector box model highlighter

    There is a new font family tooltip to the CSS rule view. Hover over a font-family value to see an inline preview of the font. (development notes)

    Inspector font family preview

    Web Console

    There are some big improvements in the web console to help view and navigate output.

    Sample console output

    Highlight and jump to nodes from console

    Running the cd() command in the console switches the scope between iframes. Read more in the cd command documentation. (development notes)

    Console cd() command

    You can read more from Mihai about the ongoing changes to the web console. He has also been documenting the web console API for extension authors.

    Firefox OS

    The network monitor is now working with Firefox OS. (development notes)

    There is now memory tracking (development notes) and jank tracking (development notes) in the Firefox OS Developer HUD. You can read much more about jank (aka “event loop lag”) in Paul’s Firefox OS: tracking reflows and event loop lags.

    Firefox OS Developer HUD

    Network Monitor

    The Network Monitor has a new look to go along with some new features:

    • The design of the network timeline has been updated, which has actually improved scroll performance on the panel. (development notes)
    • Hovering over a request with an image response now shows a popup with the image. (development notes)
    • Network requests with an image response now display a thumbnail near the file name. (development notes)

    Network Monitor Timeline UI

    Network requests with a JSON-like response will show an object preview, even if the response type is plain text. (development notes)

    JSON Response in Network Monitor

    Toolbox

    There is new behavior for console shortcut key (cmd+alt+k or ctrl+shift+k). It now focuses the input line in web console at all times, opening the toolbox if necessary but never closing it. There are more details about this change on robcee’s blog. (development notes)

    To save some space on the top toolbar, there are now options to hide command buttons, like Scratchpad. The only buttons enabled by default now are Inspect Element, Split Console, and Responsive Mode. More information about this change on the devtools mailing list. (development notes). To enable Scratchpad, Paint Flashing, or Tilt, just click on the checkbox in the options panel.

    Enabling Command Buttons on Toolbar

    We would like to give a special thanks to all 46 people who contributed patches to DevTools this release! Here is a list of all DevTools bugs resolved for Firefox 30.

    Do you have feedback, bug reports, feature requests, or questions? As always, you can comment here or get in touch with the team at @FirefoxDevTools.