Mozilla

HTML5 Articles

Sort by:

View:

  1. It’s a wrap! “App Basics for FirefoxOS” is out and ready to get you started

    A week ago we announced a series of video tutorials around creating HTML5 apps for Firefox OS. Now we released all the videos and you can watch the series in one go.

    wrap
    Photo by Olliver Hallmann

    The series is aimed at web developers who want to build their first HTML5 application. Specifically it is meant to be distributed in the emerging markets, where Firefox OS is the first option to get an affordable smartphone and start selling apps to the audiences there.

    Over the last week, we released the different videos of the series – one each day:

    Yesterday we announced the last video in the series. For all of you who asked for the whole series to watch in one go, you now got the chance to do so.

    There are various resources you can use:

    What’s next?

    There will be more videos on similar topics coming in the future and we are busy getting the videos dubbed in other languages. If you want to help us get the word out, check the embedded versions of the videos on Codefirefox.com, where we use Amara to allow for subtitles.

    Speaking of subtitles and transcripts, we are currently considering both, depending on demand. If you think this would be a very useful thing to have, please tell us in the comments.

    Thanks

    Many thanks to Sergi, Jan, Jakob, Ketil, Nathalie and Anne from Telenor, Brian Bondy from Khan Academy, Paul Jarrat and Chris Heilmann of Mozilla to make all of this possible. Technologies used to make this happen were Screenflow, Amazon S3, Vid.ly by encoding.com and YouTube.

  2. Flambe Provides Support For Firefox OS

    Flambe is a performant cross-platform open source game engine based on the Haxe programming language. Games are compiled to HTML5 or Flash and can be optimized for desktop or mobile browsers. The HTML5 Renderer uses WebGL, but provides fallback to the Canvas tag and functions nicely even on low-end phones. Flash Rendering uses Stage 3D and native Android and iOS apps are packaged using Adobe AIR.

    Flambe provides many other features, including:

    • simple asset loading
    • scene management
    • touch support
    • complete physics library
    • accelerometer access

    It has been used to create many of the Nickelodeon games available at nick.com/games and m.nick.com/games. To see other game examples, and some of the other well-known brands making use of the engine, have a look at the Flambe Showcase.

    In the last few weeks, the developers of the Flambe engine have been working to add support for Firefox OS. With the 4.0.0 release of Flambe, it is now possible to take Flambe games and package them into publication-ready Firefox OS applications, complete with manifest.

    Firefox Marketplace Games

    To get an idea of what is possible with the Flambe engine on the Firefox OS platform, take a look at two games that were submitted recently to the Firefox Marketplace. The first — The Firefly Game written by Mark Knol — features a firefly that must navigate through a flock of hungry birds. The game’s use of physics, sound and touch are very effective.
    firefly

    The second game, entitled Shoot’em Down, tests the player’s ability to dodge fire while shooting down as many enemy aircraft as possible. The game was written by Bruno Garcia, who is the main developer of the Flambe engine. The source for this game is available as one of the engine’s demo apps.
    shootemup

    Building a Firefox OS App using Flambe

    Before you can begin writing games using the Flambe engine, you will need to install and setup a few pieces of software:

    1. Haxe. Auto installers are available for OSX, Windows and Linux on the download page.
    2. Node.js for building projects. Version 0.8 or greater is required
    3. A Java runtime.

    Once those prerequisites are met, you can run the following command to install Flambe:

    # Linux and Mac may require sudo
    npm install -g flambe 
    flambe update

    This will install Flambe and you can begin writing apps with the engine.

    Create a Project

    To create a new project, run the following command.

    flambe new

    This will create a directory named whatever you supplied for ProjectName. In this directory you will have several files and other directories for configuring and coding your project. By default the new command creates a very simple project that illustrates loading and animating an image.

    A YAML (flambe.yaml) file within the project directory defines several characteristics of the project for build purposes. This file contains tags for developer, name and version of the app, and other project meta-data, such as description. In addition it contains the main class name that will be fired as the entry point to your application. This tag needs to be set to a fully qualified Haxe Class name. I.e., if you use a package name in your Haxe source file, you need to prepend the package name in this tag like this: packagename.Classname. (The default example uses urgame.Main.) You can also set the orientation for your app within the YAML file.

    Of specific note for Firefox OS developers, a section of the YAML file contains a partial manifest.webapp that can be altered. This data is merged into a complete manifest.webapp when the project is built.

    The main project folder also contains a directory for assets (images, sounds, animations, and particle effects files). The icons folder contains the icons that will be used with your app. The src folder contains the Haxe source code for your application.

    Build the Project

    Flambe provides a build method to compile your code to the appropriate output. To build the app run:

    flambe build <output>

    Where output is html, flash, android, ios, or firefox. Optionally you can add the –debug option to the build command, producing output more suitable for debugging. For Firefox OS this will produce non-minified JavaScript files. The build process will add a build directory to your application. Inside of the build directory a firefox directory will be created containing your Firefox OS app.

    Debug the Project

    You can debug your application in the Firefox App Manager. See Using the App Manager for details on installing and debugging using the App Manager. Within the App Manager you can add the built app using the Add Packaged App button and selecting the ProjectName/build/firefox directory. Debugging for other platforms is described in the Flambe documentation.
    appmanager
    The -debug option can provide additional insight for debugging and performance tuning. In addition to being able to step through the generated JavaScript, Flambe creates a source map that allows you to look look through the original Haxe files while debugging.
    debugsession
    To see the original Haxe files in the debugger, select the Debugger options icon in the far right corner of the debugger and choose Show Original Sources.
    sourcemap
    Also, when using the -debug option you can use a shortcut key (Ctrl + O) to initiate a view of your app that illustrates overdraw — this measures the number of times a pixel is being drawn in a frame. The brighter the pixel the more times it is being drawn. By reducing the amount of overdraw, you should be able to improve the performance of your game.
    overdraw

    A Bit about Haxe and Flambe

    Haxe is an object-oriented, class-based programing language that can be compiled to many other languages. In Flambe, your source code needs to be written using Haxe-specific syntax. Developers familiar with Java, C++ or JavaScript will find learning the language relatively straightforward. The Haxe website contains a reference guide that nicely documents the language. For editing, there are many options available for working with Haxe. I am using Sublime with the Haxe plugin.

    Flambe offers some additional classes that need to be used when building your app. To get a better understanding of these classes, let’s walk through the simple app that is created when you run the flambe new command. The Main.hx file created in the source directory contains the Haxe source code for the Main Class. It looks like this:

    package urgame;
     
    import flambe.Entity;
    import flambe.System;
    import flambe.asset.AssetPack;
    import flambe.asset.Manifest;
    import flambe.display.FillSprite;
    import flambe.display.ImageSprite;
     
    class Main
    {
      private static function main ()
      {
        // Wind up all platform-specific stuff
        System.init();
     
        // Load up the compiled pack in the assets directory named "bootstrap"
        var manifest = Manifest.fromAssets("bootstrap");
        var loader = System.loadAssetPack(manifest);
        loader.get(onSuccess);
      }
     
      private static function onSuccess (pack :AssetPack)
      {
        // Add a solid color background
        var background = new FillSprite(0x202020, System.stage.width, System.stage.height);
        System.root.addChild(new Entity().add(background));
     
        // Add a plane that moves along the screen
        var plane = new ImageSprite(pack.getTexture("plane"));
        plane.x._ = 30;
        plane.y.animateTo(200, 6);
        System.root.addChild(new Entity().add(plane));
      }
    }

    Haxe Packages and Classes

    The package keyword provides a way for classes and other Haxe data types to be grouped and addressed by other pieces of code, organized by directory. The import keyword is used to include classes and other Haxe types within the file you are working with. For example, import flambe.asset.Manifest will import the Manifest class, while import flambe.asset.* will import all types defined in the asset package. If you try to use a class that you have not imported into your code and run the build command, you will receive an error message stating that the particular class could not be found. All of the Flambe packages are documented on the Flambe website.

    Flambe Subsystem Setup and Entry point

    The main function is similar to other languages and acts as the entry point into your app. Flambe applications must have one main function and only one per application. In the main function the System.init() function is called to setup all the subsystems that will be needed by your code and the Flambe engine.

    Flambe Asset Management

    Flambe uses a dynamic asset management system that allows images, sound files, etc. to be loaded very simply. In this particular instance the fromAssets function defined in the Manifest class examines the bootstrap folder located in the assets directory to create a manifest of all the available files. The loadAssetPack System function creates an instance of the AssetPack based on this manifest. One of the functions of AssetPack is get, which takes a function parameter to call when the asset pack is loaded into memory. In the default example, the only asset is an image named plane.png.

    Flambe Entities and Components

    Flambe uses an abstract concept of Entities and Components to describe and manipulate game objects. An Entity is essentially just a game object with no defining characteristics. Components are characteristics that are attached to entities. For example an image component may be attached to an entity. Entities are also hierarchal and can be nested. For example, entity A can be created and an image could be attached to it. Entity B could then be created with a different image. Entity A could then be attached to the System root (top level Entity) and Entity B could then be attached to Entity A or the System root. The entity nest order is used for rendering order, which can be used to make sure smaller visible objects are not obscured by other game objects.

    Creating Entities and Components in the Sample App

    The onSuccess function in the default sample is called by the loader instance after the AssetPack is loaded. The function first creates an instance of a FillSprite Component, which is a rectangle defined by the size of the display viewport width and height. This rectangle is colored using the hex value defined in the first parameter. To actually have the FillSprite show up on the screen you first have to create an Entity and add the Component to it. The new Entity().add(background) method first creates the Entity and then adds the FillSprite Component. The entire viewport hierarchy starts at the System.root, so the addChild command adds this new Entity to the root. Note this is the first Entity added and it will be the first rendered. In this example this entity represents a dark background.

    Next the plane image is created. This is done by passing the loaded plane image to the ImageSprite Component constructor. Note that the AssetPack class’s getTexture method is being used to retrieve the loaded plane image. The AssetPack class contains methods for retrieving other types of Assets as well. For example, to retrieve and play a sound you would use pack.getSound("bounce").play();.

    Flambe Animated Data Types

    Flambe wraps many of the default Haxe data types in classes and introduces a few more. One of these is the AnimatedFloat class. This class essentially wraps a float and provides some utility functions that allow the float to be altered in a specific way. For example, one of the functions of the AnimatedFloat class is named animateTo, which takes parameters to specify the final float value and the time in which the animation will occur. Many components within the Flambe system use AnimatedFloats for property values. The plane that is loaded in the default application is an instance of the ImageSprite Component. Its x and y placement values are actually AnimatedFloats. AnimatedFloat values can be set directly but special syntax has to be used (value._).

    In the example, the x value for the ImageSprite is set to 30 using this syntax: plane.x._ = 30;. The y value for the ImageSprite is then animated to 200 over a 6 second period. The x and y values for an ImageSprite represent the upper left corner of the image when placed into the viewport. You can alter this using the centerAnchor function of the ImageSprite class. After this call, the x and y values will be in reference to the center of the image. While the default example does not do this, it could be done by calling plane.centerAnchor();. The final line of code just creates a new Entity, adds the plane Component to the Entity and then adds the new Entity to the root. Note that this is the second Entity added to the root and it will render after the background is rendered.

    Flambe Event Model

    Another area of Flambe that is important to understand is its event model. Flambe uses a signal system where the subsystems, Components and Entities have available signal properties that can be connected to in order to listen for a specific signal event. For example, resizing the screen fires a signal. This event can be hooked up using the following code.

    System.stage.resize.connect(function onResize() {
      //do something 
    });

    This is a very nice feature when dealing with other components within apps. For example, to do something when a user either clicks on or touches an ImageSprite within your app you would use the following code:

    //ImageSprite Component has pointerDown signal property
    myBasketBallEntity.get(ImageSprite).pointerDown.connect(function (event) {
        bounceBall();
    });

    In this case the pointerDown signal is fired when a user either uses a mouse down or touch gesture.

    Demo Apps

    The Flambe repository also contains many demo apps that can be used to further learn the mechanics and APIs for the engine. These demos have been tested on Firefox OS and perform very well. Pictured below are several screenshots taken on a Geeksphone Keon running Firefox OS.
    colla

    Of particular note in the demos are the physics and particles demos. The physics demo uses the Nape Haxe library and allows for some very cool environments. The Nape website contains documentation for all the packages available. To use this library you need to run the following command:

    haxelib install nape

    The particle demo illustrates using particle descriptions defined in a PEX file within a Flambe-based game. PEX files can be defined using a particle editor, like Particle Designer.

    Wrapping Up

    If you are a current Flambe game developer with one or more existing games, why not use the new version of the engine to compile and package them for Firefox OS? If you are a Firefox OS developer and are looking for a great way to develop new games for the platform, Flambe offers an excellent means for developing engaging, performant games for Firefox OS–and many other platforms besides!

    And, if you are interested in contributing to Flambe, we’d love to hear from you as well.

  3. App basics for Firefox OS – a screencast series to get you started

    Over the next few days we’ll release a series of screencasts explaining how to start your first Open Web App and develop for Firefox OS.

    Firefox OS - Intro and hello

    Each of the screencasts is terse enough to watch in a short break and the whole series should not take you more than an hour of your time. The series features Jan Jongboom (@janjongboom), Sergi Mansilla (@sergimansilla) of Telenor Digital and Chris Heilmann (@codepo8) of Mozilla and was shot in three days in Oslo, Norway at the offices of Telenor Digital in February 2014.

    Here are the three of us telling you about the series and what to expect:

    Firefox OS is an operating system that brings the web to mobile devices. Instead of being a new OS with new technologies and development environments it builds on standardised web technologies that have been in use for years now. If you are a web developer and you want to build a mobile app, Firefox OS gives you the tools to do so, without having to change your workflow or learn a totally new development environment. In this series of short videos, developers from Mozilla and Telenor met in Oslo, Norway to explain in a few steps how you can get started to build applications for FirefoxOS. You’ll learn:

    • how to build your first application for Firefox OS
    • how to debug and test your application both on the desktop and the real device
    • how to get it listed in the marketplace
    • how to use the APIs and special interfaces Firefox OS offers a JavaScript developer to take advantage of the hardware available in smartphones.

    In addition to the screencasts, you can download the accompanying code samples from GitHub . If you want to try the code examples out for yourself, you will need to set up a very simple development environment. All you need is:

    • A current version of Firefox (which comes out of the box with the developer tools you need) – we recommend getting Firefox Aurora or Nightly if you really want to play with the state-of-the-art technology.
    • A text editor – in the screencasts we used Sublime Text, but any will do. If you want to be really web native, you can try Adobe Brackets.
    • A local server or a server to push your demo files to. A few of the demo apps need HTTP connections instead of local ones.

    sergi and chris recording

    Over the next few days we’ll cover the following topics:

    In addition to the videos, you can also go to the Wiki page of the series to get extra information and links on the subjects covered.

    Come back here to see the links appear day by day or follow us on Twitter at @mozhacks to get information when the next video is out.

    jan recording his video

    Once the series is out, there’ll be a Wiki resource to get them all in one place. Telenor are also working on getting these videos dubbed in different languages. For now, stay tuned.

    Many thanks to Sergi, Jan, Jakob, Ketil, Nathalie and Anne from Telenor to make all of this possible.

  4. The Making of the Time Out Firefox OS app

    A rash start into adventure

    So we told our client that yes, of course, we would do their Firefox OS app. We didn’t know much about FFOS at the time. But, hey, we had just completed refactoring their native iOS and Android apps. Web applications were our core business all along. So what was to be feared?

    More than we thought, it turned out. Some of the dragons along the way we fought and defeated ourselves. At times we feared that we wouldn’t be able to rescue the princess in time (i.e. before MWC 2013). But whenever we got really lost in detail forest, the brave knights from Mozilla came to our rescue. In the end, it all turned out well and the team lived happily ever after.

    But here’s the full story:

    Mission & challenge

    Just like their iOS and Android apps, Time Out‘s new Firefox OS app was supposed to allow browsing their rich content on bars, restaurants, things to do and more by category, area, proximity or keyword search, patient zero being Barcelona. We would need to show results as illustrated lists as well as visually on a map and have a decent detail view, complete with ratings, access details, phone button and social tools.

    But most importantly, and in addition to what the native apps did, this app was supposed to do all of that even when offline.

    Oh, and there needed to be a presentable, working prototype in four weeks time.

    Cross-platform reusability of the code as a mobile website or as the base of HTML5 apps on other mobile platforms was clearly prio 2 but still to be kept in mind.

    The princess was clearly in danger. So we arrested everyone on the floor that could possibly be of help and locked them into a room to get the basics sorted out. It quickly emerged that the main architectural challenges were that

    • we had a lot of things to store on the phone, including the app itself, a full street-level map of Barcelona, and Time Out’s information on every venue in town (text, images, position & meta info),
    • at least some of this would need to be loaded from within the app; once initially and synchronizable later,
    • the app would need to remain interactively usable during these potentially lengthy downloads, so they’d need to be asynchronous,
    • whenever the browser location changed, this would be interrupted

    In effect, all the different functionalities would have to live within one single HTML document.

    One document plus hash tags

    For dynamically rendering, changing and moving content around as required in a one-page-does-all scenario, JavaScript alone didn’t seem like a wise choice. We’d been warned that Firefox OS was going to roll out on a mix of devices including the very low cost class, so it was clear that fancy transitions of entire full-screen contents couldn’t be orchestrated through JS loops if they were to happen smoothly.

    On the plus side, there was no need for JS-based presentation mechanics. With Firefox OS not bringing any graveyard of half-dead legacy versions to cater to, we could (finally!) rely on HTML5 and CSS3 alone and without fallbacks. Even beyond FFOS, the quick update cycles in the mobile environment didn’t seem to block the path for taking a pure CSS3 approach further to more platforms later.

    That much being clear, which better place to look for best practice examples than Mozilla Hacks? After some digging, Thomas found Hacking Firefox OS in which Luca Greco describes the use of fragment identifiers (aka hashtags) appended to the URL to switch and transition content via CSS alone, which we happily adopted.

    Another valuable source of ideas was a list of GAIA building blocks on Mozilla’s website, which has since been replaced by the even more useful Building Firefox OS site.

    In effect, we ended up thinking in terms of screens. Each physically a <div>, whose visibility and transitions are governed by :target CSS selectors that draw on the browser location’s hashtag. Luckily, there’s also the onHashChange event that we could additionally listen to in order to handle the app-level aspects of such screen changes in JavaScript.

    Our main HTML and CSS structure hence looked like this:

    And a menu

    We modeled the drawer menu very similarily, just that it sits in a <nav> element on the same level as the <section> container holding all the screens. Its activation and deactivation works by catching the menu icon clicks, then actively changing the screen container’s data-state attribute from JS, which triggers the corresponding CSS3 slide-in / slide-out transition (of the screen container, revealing the menu beneath).

    This served as our “Hello, World!” test for CSS3-based UI performance on low-end devices, plus as a test case for combining presentation-level CSS3 automation with app-level explicit status handling. We took down a “yes” for both.

    UI

    By the time we had put together a dummy around these concepts, the first design mockups from Time Out came in so that we could start to implement the front end and think about connecting it to the data sources.

    For presentation, we tried hard to keep the HTML and CSS to the absolute minimum. Mozilla’s GAIA examples being a very valuable source of ideas once more.

    Again, targeting Firefox OS alone allowed us to break free of the backwards compatibility hell that we were still living in, desktop-wise. No one would ask us Will it display well in IE8? or worse things. We could finally use real <section>, <nav>, <header>, and <menu> tags instead of an army of different classes of <div>. What a relief!

    The clear, rectangular, flat and minimalistic design we got from Time Out also did its part to keep the UI HTML simple and clean. After we were done with creating and styling the UI for 15 screens, our HTML had only ~250 lines. We later improved that to 150 while extending the functionality, but that’s a different story.

    Speaking of styling, not everything that had looked good on desktop Firefox even in its responsive design view displayed equally well on actual mobile devices. Some things that we fought with and won:

    Scale: The app looked quite different when viewed on the reference device (a TurkCell branded ZTE device that Mozilla had sent us for testing) and on our brand new Nexus 4s:

    After a lot of experimenting, tearing some hair and looking around how others had addressed graceful, proportional scaling for a consistent look & feel across resolutions, we stumbled upon this magic incantation:

    <meta name="viewport" content="user-scalable=no, initial-scale=1,
    maximum-scale=1, width=device-width" />

    What it does, to quote an article at Opera, is to tell the browser that there is “No scaling needed, thank you very much. Just make the viewport as many pixels wide as the device screen width”. It also prevents accidental scaling while the map is zoomed. There is more information on the topic at MDN.

    Then there are things that necessarily get pixelated when scaled up to high resolutions, such as the API based venue images. Not a lot we could do about that. But we could at least make the icons and logo in the app’s chrome look nice in any resolution by transforming them to SVG.

    Another issue on mobile devices was that users have to touch the content in order to scroll it, so we wanted to prevent the automatic highlighting that comes with that:

    li, a, span, button, div
    {
        outline:none;
        -moz-tap-highlight-color: transparent;
        -moz-user-select: none;
        -moz-user-focus:ignore
    }

    We’ve since been warned that suppressing the default highlighting can be an issue in terms of accessibility, so you might wanted to consider this carefully.

    Connecting to the live data sources

    So now we had the app’s presentational base structure and the UI HTML / CSS in place. It all looked nice with dummy data, but it was still dead.

    Trouble with bringing it to life was that Time Out was in the middle of a big project to replace its legacy API with a modern Graffiti based service and thus had little bandwidth for catering to our project’s specific needs. The new scheme was still prototypical and quickly evolving, so we couldn’t build against it.

    The legacy construct already comprised a proxy that wrapped the raw API into something more suitable for consumption by their iOS and Android apps, but after close examination we found that we better re-re-wrap that on the fly in PHP for a couple of purposes:

    • Adding CORS support to avoid XSS issues, with the API and the app living in different subdomains of timeout.com,
    • stripping API output down to what the FFOS app really needed, which we could see would reduce bandwidth and increase speed by magnitude,
    • laying the foundation for harvesting of API based data for offline use, which we already knew we’d need to do later

    As an alternative to server-side CORS support, one could also think of using the SystemXHR API. It is a mighty and potentially dangerous tool however. We also wanted to avoid any needless dependency on FFOS-only APIs.

    So while the approach wasn’t exactly future proof, it helped us a lot to get to results quickly, because the endpoints that the app was calling were entirely of our own choice and making, so that we could adapt them as needed without time loss in communication.

    Populating content elements

    For all things dynamic and API-driven, we used the same approach at making it visible in the app:

    • Have a simple, minimalistic, empty, hidden, singleton HTML template,
    • clone that template (N-fold for repeated elements),
    • ID and fill the clone(s) with API based content.
    • For super simple elements, such as <li>s, save the cloning and whip up the HTML on the fly while filling.

    As an example, let’s consider the filters for finding venues. Cuisine is a suitable filter for restaurants, but certainly not for museums. Same is true for filter values. There are vegetarian restaurants in Barcelona, but certainly no vegetarian bars. So the filter names and lists of possible values need to be asked of the API after the venue type is selected.

    In the UI, the collapsible category filter for bars & pubs looks like this:

    The template for one filter is a direct child of the one and only

    <div id="templateContainer">

    which serves as our central template repository for everything cloned and filled at runtime and whose only interesting property is being invisible. Inside it, the template for search filters is:

    <div id="filterBoxTemplate">
      <span></span>
      <ul></ul>
    </div>

    So for each filter that we get for any given category, all we had to do was to clone, label, and then fill this template:

    $('#filterBoxTemplate').clone().attr('id', filterItem.id).appendTo(
    '#categoryResultScreen .filter-container');
    ...
    $("#" + filterItem.id).children('.filter-button').html(
    filterItem.name);

    As you certainly guessed, we then had to to call the API once again for each filter in order to learn about its possible values, which were then rendered into <li> elements within the filter‘s <ul> on the fly:

    $("#" + filterId).children('.filter_options').html(
    '<li><span>Loading ...</span></li>');
    
    apiClient.call(filterItem.api_method, function (filterOptions)
    {
      ...
      $.each(filterOptions, function(key, option)
      {
        var entry = $('<li filterId="' + option.id + '"><span>'
          + option.name + '</span></li>');
    
        if (selectedOptionId && selectedOptionId == filterOptionId)
        {
          entry.addClass('filter-selected');
        }
    
        $("#" + filterId).children('.filter_options').append(entry);
      });
    ...
    });

    DOM based caching

    To save bandwidth and increase responsiveness in on-line use, we took this simple approach a little further and consciously stored more application level information in the DOM than needed for the current display if that information was likely needed in the next step. This way, we’d have easy and quick local access to it without calling – and waiting for – the API again.

    The technical way we did so was a funny hack. Let’s look at the transition from the search result list to the venue detail view to illustrate:

    As for the filters above, the screen class for the detailView has an init() method that populates the DOM structure based on API input as encapsulated on the application level. The trick now is, while rendering the search result list, to register anonymous click handlers for each of its rows, which – JavaScript passing magic – contain a copy of, rather than a reference to, the venue objects used to render the rows themselves:

    renderItems: function (itemArray)
    {
      ...
    
      $.each(itemArray, function(key, itemData)
      {        
        var item = screen.dom.resultRowTemplate.clone().attr('id', 
          itemData.uid).addClass('venueinfo').click(function()
        {
          $('#mapScreen').hide();
          screen.showDetails(itemData);
        });
    
        $('.result-name', item).text(itemData.name);
        $('.result-type-label', item).text(itemData.section);
        $('.result-type', item).text(itemData.subSection);
    
        ...
    
        listContainer.append(item);
      });
    },
    
    ...
    
    showDetails: function (venue)
    {
      require(['screen/detailView'], function (detailView)
      {
        detailView.init(venue);
      });
    },

    In effect, there’s a copy of the data for rendering each venue’s detail view stored in the DOM. But neither in hidden elements nor in custom attributes of the node object, but rather conveniently in each of the anonymous pass-by-value-based click event handlers for the result list rows, with the added benefit that they don’t need to be explicitly read again but actively feed themselves into the venue details screen as soon a row receives a touch event.

    And dummy feeds

    Finishing the app before MWC 2013 was pretty much a race against time, both for us and for Time Out’s API folks, who had an entirely different and equally – if not more so – sportive thing to do. Therefore they had very limited time for adding to the (legacy) API that we were building against. For one data feed, this meant that we had to resort to including static JSON files into the app’s manifest and distribution; then use relative, self-referencing URLs as fake API endpoints. The illustrated list of top venues on the app’s main screen was driven this way.

    Not exactly nice, but much better than throwing static content into the HTML! Also, it kept the display code already fit for switching to the dynamic data source that eventually materialized later, and compatible with our offline data caching strategy.

    As the lack of live data on top venues then extended right to their teaser images, we made the latter physically part of the JSON dummy feed. In Base64 :) But even the low-end reference device did a graceful job of handling this huge load of ASCII garbage.

    State preservation

    We had a whopping 5M of local storage to spam, and different plans already (as well as much higher needs) for storing the map and application data for offline use. So what to do with this liberal and easily accessed storage location? We thought we could at least preserve the current application state here, so you’d find the app exactly as you left it when you returned to it.

    Map

    A city guide is the very showcase of an app that’s not only geo aware but geo centric. Maps fit for quick rendering and interaction in both online and offline use were naturally a paramount requirement.

    After looking around what was available, we decided to go with Leaflet, a free, easy to integrate, mobile friendly JavaScript library. It proved to be really flexible with respect to both behaviour and map sources.

    With its support for pinching, panning and graceful touch handling plus a clean and easy API, Leaflet made us arrive at a well-usable, decent-looking map with moderate effort and little pain:

    For a different project, we later rendered the OSM vector data for most of Europe into terabytes of PNG tiles in cloud storage using on-demand cloud power. Which we’d recommend as an approach if there’s a good reason not to rely on 3rd party hosted apps, as long as you don’t try this at home; Moving the tiles may well be slower and more costly than their generation.

    But as time was tight before the initial release of this app, we just – legally and cautiously(!) – scraped ready-to use OSM tiles off MapQuest.com.

    The packaging of the tiles for offline use was rather easy for Barcelona because about 1000 map tiles are sufficient to cover the whole city area up to street level (zoom level 16). So we could add each tile as a single line into the manifest.appache file. The resulting, fully automatic, browser-based download on first use was only 10M.

    This left us with a lot of lines like

    /mobile/maps/barcelona/15/16575/12234.png
    /mobile/maps/barcelona/15/16575/12235.png
    ...

    in the manifest and wishing for a $GENERATE clause as for DNS zone files.

    As convenient as it may seem to throw all your offline dependencies’ locations into a single file and just expect them to be available as a consequence, there are significant drawbacks to this approach. The article Application Cache is a Douchebag by Jake Archibald summarizes them and some help is given at Html5Rocks by Eric Bidleman.

    We found at the time that the degree of control over the current download state, and the process of resuming the app cache load in case that the initial time users spent in our app didn’t suffice for that to complete was rather tiresome.

    For Barcelona, we resorted to marking the cache state as dirty in Local Storage and clearing that flag only after we received the updateready event of the window.applicationCache object but in the later generalization to more cities, we moved the map away from the app cache altogether.

    Offline storage

    The first step towards offline-readiness was obviously to know if the device was online or offline, so we’d be able to switch the data source between live and local.

    This sounds easier than it was. Even with cross-platform considerations aside, neither the online state property (window.navigator.onLine), the events fired on the <body> element for state changes (“online” and “offline”, again on the <body>), nor the navigator.connection object that was supposed to have the on/offline state plus bandwidth and more, really turned out reliable enough.

    Standardization is still ongoing around all of the above, and some implementations are labeled as experimental for a good reason :)

    We ultimately ended up writing a NetworkStateService class that uses all of the above as hints, but ultimately and very pragmatically convinces itself with regular HEAD requests to a known live URL that no event went missing and the state is correct.

    That settled, we still needed to make the app work in offline mode. In terms of storage opportunities, we were looking at:

    Storage Capacity Updates Access Typical use
    App / app cache, i.e. everything listed in the file that the value of appcache_path in the app‘s webapp.manifest points to, and which is and therefore downloaded onto the device when the app is installed. <= 50M. On other platforms (e.g. iOS/Safari), user interaction required from 10M+. Recommendation from Moziila was to stay <2M. Hard. Requires user interaction / consent, and only wholesale update of entire app possible. By (relative) path HTML, JS, CSS, static assets such as UI icons
    LocalStorage 5M on UTF8-platforms such as FFOS, 2.5M in UTF16, e.g. on Chrome. Details here Anytime from app By name Key-value storage of app status, user input, or entire data of modest apps
    Device Storage (often SD card) Limited only by hardware Anytime from app (unless mounted as UDB drive when cionnected to desktop computer) By path, through Device Storage API Big things
    FileSystem API Bad idea
    Database Unlimited on FFOS. Mileage on other platforms varies Anytime from app Quick and by arbitrary properties Databases :)

    Some aspects of where to store the data for offline operation were decided upon easily, others not so much:

    • the app, i.e. the HTML, JS, CSS, and UI images would go into the app cache
    • state would be maintained in Local Storage
    • map tiles again in the app cache. Which was a rather dumb decision, as we learned later. Barcelona up to zoom level 16 was 10M, but later cities were different. London was >200M and even reduced to max. zoom 15 still worth 61M. So we moved that to Device Storage and added an actively managed download process for later releases.
    • The venue information, i.e. all the names, locations, images, reviews, details, showtimes etc. of the places that Time Out shows in Barcelona. Seeing that we needed lots of space, efficient and arbitrary access plus dynamic updates, this had to to go into the Database. But how?

    The state of affairs across the different mobile HTML5 platforms was confusing at best, with Firefox OS already supporting IndexedDB, but Safari and Chrome (considering earlier versions up to Android 2.x) still relying on a swamp of similar but different sqlite / WebSQL variations.

    So we cried for help and received it, as always when we had reached out to the Mozilla team. This time in the form of a pointer to pouchDB, a JS-based DB layer that at the same time wraps away the different native DB storage engines behind a CouchDB-like interface and adds super easy on-demand synchronization to a remote CouchDB-hosted master DB out there.

    Back last year it still was in pre-alpha state but very usable already. There were some drawbacks, such as the need for adding a shim for WebSql based platforms. Which in turn meant we couldn’t rely on storage being 8 bit clean, so that we had to base64 our binaries, most of all the venue images. Not exactly pouchDB’s fault, but still blowing up the size.

    Harvesting

    The DB platform being chosen, we next had to think how we’d harvest all the venue data from Time Out’s API into the DB. There were a couple of endpoints at our disposal. The most promising for this task was proximity search with no category or other restrictions applied, as we thought it would let us harvest a given city square by square.

    Trouble with distance metrics however being that they produce circles rather than squares. So step 1 of our thinking would miss venues in the corners of our theoretical grid

    while extending the radius to (half the) the grid’s diagonal, would produce redundant hits and necessitate deduplication.

    In the end, we simply searched by proximity to a city center location, paginating through the result indefinitely, so that we could be sure to to encounter every venue, and only once:

    Technically, we built the harvester in PHP as an extension to the CORS-enabled, result-reducing API proxy for live operation that was already in place. It fed the venue information in to the master CouchDB co-hosted there.

    Time left before MWC 2013 getting tight, we didn’t spend much time on a sophisticated data organization and just pushed the venue information into the DB as one table per category, one row per venue, indexed by location.

    This allowed us to support category based and area / proximity based (map and list) browsing. We developed an idea how offline keyword search might be made possible, but it never came to that. So the app simply removes the search icon when it goes offline, and puts it back when it has live connectivity again.

    Overall, the app now

    • supported live operation out of box,
    • checked its synchronization state to the remote master DB on startup,
    • asked, if needed, permission to make the big (initial or update) download,
    • supported all use cases but keyword search when offline.

    The involved components and their interactions are summarized in this diagram:

    Organizing vs. Optimizing the code

    For the development of the app, we maintained the code in a well-structured and extensive source tree, with e.g. each JavaScript class residing in a file of its own. Part of the source tree is shown below:

    This was, however, not ideal for deployment of the app, especially as a hosted Firefox OS app or mobile web site, where download would be the faster, the fewer and smaller files we had.

    Here, Require.js came to our rescue.

    It provides a very elegant way of smart and asynchronous requirement handling (AMD), but more importantly for our purpose, comes with an optimizer that minifies and combines the JS and CSS source into one file each:

    To enable asynchronous dependency management, modules and their requirements must be made known to the AMD API through declarations, essentially of a function that returns the constructor for the class you’re defining.

    Applied to the search result screen of our application, this looks like this:

    define
    (
      // new class being definied
      'screensSearchResultScreen',
    
      // its dependencies
      ['screens/abstractResultScreen', 'app/applicationController'],
    
      // its anonymous constructor
      function (AbstractResultScreen, ApplicationController)
      {
        var SearchResultScreen = $.extend(true, {}, AbstractResultScreen,
        {
          // properties and methods
          dom:
          {
            resultRowTemplate: $('#searchResultRowTemplate'),
            list: $('#search-result-screen-inner-list'),
            ...
          }
          ...
        }
        ...
    
        return SearchResultScreen;
      }
    );

    For executing the optimization step in the build & deployment process, we used Rhino, Mozilla’s Java-based JavaScript engine:

    java -classpath ./lib/js.jar:./lib/compiler.jar   
      org.mozilla.javascript.tools.shell.Main ./lib/r.js -o /tmp/timeout-webapp/
      $1_config.js

    CSS bundling and minification is supported, too, and requires just another call with a different config.

    Outcome

    Four weeks had been a very tight timeline to start with, and we had completely underestimated the intricacies of taking HTML5 to a mobile and offline-enabled context, and wrapping up the result as a Marketplace-ready Firefox OS app.

    Debugging capabilities in Firefox OS, especially on the devices themselves, were still at an early stage (compared to clicking about:app-manager today). So the lights in our Cologne office remained lit until pretty late then.

    Having built the app with a clear separation between functionality and presentation also turned out a wise choice when a week before T0 new mock-ups for most of the front end came in :)

    But it was great and exciting fun, we learned a lot in the process, and ended up with some very useful shiny new tools in our box. Often based on pointers from the super helpful team at Mozilla.

    Truth be told, we had started into the project with mixed expectations as to how close to the native app experience we could get. We came back fully convinced and eager for more.

    In the end, we made the deadline and as a fellow hacker you can probably imagine our relief. The app finally even received its 70 seconds of fame, when Jay Sullivan shortly demoed it at Mozilla’s MWC 2013 press conference as a showcase for HTML5′s and Firefox OS’s offline readiness (Time Out piece at 7:50). We were so proud!

    If you want to play with it, you can find the app in the marketplace or go ahead try it online (no offline mode then).

    Since then, the Time Out Firefox OS app has continued to evolve, and we as a team have used the chance to continue to play with and build apps for FFOS. To some degree, the reusable part of this has become a framework in the meantime, but that’s a story for another day..

    We’d like to thank everyone who helped us along the way, especially Taylor Wescoatt, Sophie Lewis and Dave Cook from Time Out, Desigan Chinniah and Harald Kirschner from Mozilla, who were always there when we needed help, and of course Robert Nyman, who patiently coached us through writing this up.

  5. Building Cordova apps for Firefox OS

    Update: In addition to the Cordova integration described below, Firefox OS is now supported in the 3.4 release of Adobe PhoneGap.

    If you’re already building apps with PhoneGap, you can quickly and easily port your existing apps to Firefox OS. We think this is so cool that we’ve launched a Phones for PhoneGap Apps program, focused specifically on compelling apps built with PhoneGap and/or Cordova. Got a great PhoneGap app? We’d love to send you a device!

    Cordova is a popular Apache Foundation open source project providing a set of device APIs to allow mobile application developers to access native device functions—such as the camera or the accelerometer—from JavaScript. HTML5 applications can be packaged as native apps via the framework and made available for installation from the app stores of supported platforms, including iOS, Android, Blackberry, Windows Phone—and now Firefox OS. Cordova is also the underlying software in the Adobe product PhoneGap.

    Over the past few months, Mozilla has been working with the Cordova team to integrate Firefox OS into the Cordova framework, making it possible to release Cordova apps on the Firefox OS platform. While this is an ongoing project, significant functionality is available now with the 3.4 release of Cordova. In this post we will describe how to use these new capabilities.

    Creating and building a Firefox OS app in Cordova

    The Cordova site explains how to install the software. Note the installation procedure requires Node.js and can be executed from the command line.

    $ sudo npm install -g cordova

    Once Cordova is installed, an application can be built with the Cordova create command. (Parameters for the command are described in the Cordova documentation linked above.)

    $ cordova create hello com.example.hello HelloWorld

    This will create a directory named hello that contains the project and a basic web app in the hello/www directory. In order to produce a Firefox OS app, the proper platform needs to be added next.

    $ cd hello
    $ cordova platform add firefoxos

    With some of the other supported platforms, you would generally run a build command at this stage to produce the output for the platform. Because Firefox OS is an HTML5-based operating system, no compile step is needed to process and produce the app. The only step required is a prepare statement to package the app.

    $ cordova prepare firefoxos

    Those are the basic steps to generate a simple Firefox OS app from a Cordova project. The output for the project will be located in the hello/platforms/firefoxos/www directory.

    Debugging the app

    With most other Cordova platforms you would use the emulate or run commands to test an app. With Firefox OS you currently need to use the App Manager, which is the primary tool for debugging and interrogating a Firefox OS app. The tool offers many capabilities including JavaScript debugging and live editing of the CSS while connected to a device.

    The above link explains how to install and start the App Manager. Once the App Manager is running, you can click the Add Packaged App button and select the hello/platforms/firefoxos/www directory of your app and press the Open button.

    appmanager

    This will add the basic app to the App Manager. You will notice no icons are present. This is because the framework integration does not provide them currently and only a bare-bones manifest.webapp is created. From here you can update the app or debug it. Note that between updates you must run a cordova prepare firefoxos command as this step packages the app and puts it in the platforms/firefoxos/www directory. Below is a screenshot of the Cordova HelloWorld app being debugged.

    debugger

    The Firefox OS Manifest

    Firefox OS apps are essentially HTML5 applications that are described by a manifest file. This manifest file points to artifacts like icons, start page, etc. that will be used by the application. In addition, the manifest controls privilege levels and device-specific APIs that are needed by the app. The manifest documentation is available on MDN.

    With the default integration in Cordova, a very generic manifest is created and placed in the platforms/firefoxos/www directory. In almost all cases this will not suffice, as you will at least want to provide icons for your app. The App Manager will complain if the app does not contain at least a 128×128 pixel sized icon. This does not prevent you from testing your app, but it is required to upload your app to the Firefox Marketplace. The manifest can be created with a simple text editor or you can modify the manifest in the App Manager. An example manifest.webapp is shown below.

    {
      "name": "My App",
      "description": "My elevator pitch goes here",
      "launch_path": "/",
      "icons": {
        "128": "/img/icon-128.png"
      },
      "developer": {
        "name": "Your name or organization",
        "url": "http://your-homepage-here.org"
      },
      "default_locale": "en"
    }

    Make sure the manifest is created or copied into the project/www folder. Subsequent cordova prepare commands will replace the auto-generated manifest with your application-specific manifest.

    Start Up Code

    When creating a Cordova app, the starter code generated includes index.html, css/index.css, img/logo.png and js/index.js files. The code in index.js is initiated in index.hml like this:

    <script type="text/javascript">
      app.initialize();
    </script>

    The initialize function essentially sets up the event trigger for the onDeviceReady event, which signifies that the Cordova framework is loaded and ready. The generated code is sufficient for Firefox OS unless you want to implement a privileged App. Privileged apps are Marketplace-signed apps that require the use of more sensitive APIs–for example, Contacts API. See the Packaged apps documentation for more information. For privileged Apps, code like this violates CSP restrictions because of the inline script tag. To get around this, remove the inline script and initiate the app using a window.onload event in the js/index.js file.

    Sample App

    To test and debug the Cordova/Firefox OS integration we developed a sample app. This application is available on GitHub. It illustrates use of the device-specific plugins. The images and code snippets in the following sections were taken from the sample app. If you want to check out the code and work with it, first create a Cordova project and check the code into the project/www directory. You can then run cordova prepare firefoxos to package the app. Run and debug as described earlier in this post.

    app

    Device APIs

    Cordova uses a plugin architecture to implement device APIs, such as Accelerometer, Geolocation or Contacts. These APIs are very similar to Firefox OS Web APIs and Web Activities and are documented on the Cordova website. Below are the current plugins implemented for Firefox OS and a brief description of how you can include them in your app. You can always see the current status of plugin development for the Firefox OS Platform by checking Jira on the Apache website.

    Notification API

    The notification API is used to alert the user of your app and is implemented in two plugins: org.apache.cordova.dialogs and org.apache.cordova.vibration. Currently we have implemented the alert, confirm, prompt and vibrate functions. To use this functionality, add the plugins to your project with the following commands:

    $ cordova plugin add org.apache.cordova.dialogs
    $ cordova plugin add org.apache.cordova.vibration

    To get proper styling of popup boxes in Firefox OS, you will need to add the notification.css file to your project. After adding the dialogs plugin, change to the project/plugins/org.apache.cordova.dialogs/www/firefoxos directory and copy the notification.css file to your project/www/css folder. Link the CSS file in head element of index.html.

    <link rel="stylesheet" type="text/css" href="css/notification.css" />

    You can now use the notification functions.

    function onPrompt(results) {
        alert("You selected button number " + 
              results.buttonIndex + 
              " and entered " + results.input1);
    }
    navigator.notification.vibrate(500);
    navigator.notification.prompt(
          'Enter Name', // message
           onPrompt, // callback to invoke
           'Prompt Test', // title
            ['Ok', 'Exit'], // buttonLabels
             'Doe, Jane' // defaultText
    );

    prompt

    Compass API

    The compass API is implemented using the org.apache.cordova.device-orientation plugin. This plugin implements the compass getCurrentHeading and watchHeading functions. To use it simply run the plugin add command:

    $ cordova plugin add org.apache.cordova.device-orientation

    Once the plugin is added, you can use the get or watch heading function to get compass information.

    function onSuccess(heading) {
        var element = document.getElementById('heading');
        myHeading = (heading.magneticHeading).toFixed(2);
        console.log("My Heading = " + myHeading);
    }
    function onError(compassError) {
        alert('Compass error: ' + compassError.code);
    }
    var options = {
        frequency: 500
    };
    watchID = navigator.compass.watchHeading(onSuccess, onError, options);

    compass

    Accelerometer API

    The Accelerometer is accessed using the org.apache.cordova.device-motion plugin and gives the developer access to acceleration data in x, y and z directions. This plugin implements the getCurrentAcceleration and watchAcceleration functions.

    To use these functions, add the device-motion plugin to your project by executing the following command.

    $ cordova plugin add org.apache.cordova.device-motion

    You can then monitor the acceleration values using code similar to this:

    var options = {
        frequency: 100
    };
    watchIDAccel = navigator.accelerometer.watchAcceleration(onSuccess, onError, options);
    function onSuccess(acceleration) {
      var acX = acceleration.x.toFixed(1) * -1;
      var acY = acceleration.y.toFixed(1);
      var acZ = acceleration.z.toFixed(1);
      var vals = document.getElementById('accvals');
      var accelstr = "<strong>Accel X: </strong>" + acX + "<br>" + "<strong>Accel Y: </strong>" + acY + "<br>" + "<strong>Accel Z: </strong>" + acZ;
      vals.innerHTML = accelstr;
    }
    function onError() {
      alert('Could not Retrieve Accelerometer Data!');
    }

    You can also monitor the device orienttation event and retrieve alpha, beta, and gamma rotation values like:

    function deviceOrientationEvent(eventData) {
        //skew left and right
        var alpha = Math.round(eventData.alpha);
        //front to back - neg back postive front
        var beta = Math.round(eventData.beta);
        //roll left positive roll right neg
        var gamma = Math.round(eventData.gamma);
        console.log("beta = " + beta + " gamma = " + gamma);
    }
    window.addEventListener('deviceorientation', deviceOrientationEvent);

    Camera API

    The camera API is used to retrieve an image from the gallery or from the device camera. This API is implemented in the org.apache.cordova.camera plugin. To use this feature, add the plugin to your project.

    $ cordova plugin add org.apache.cordova.camera

    In the Firefox OS implementation of this plugin, the getPicture function will trigger a Web Activity that allows the user to select where the image is retrieved.

    Code similar to the following can be used to execute the getPicture function:

    navigator.camera.getPicture(function (src) {
        var img = document.createElement('img');
        img.id = 'slide';
        img.src = src;
      }, function () {}, {
          destinationType: 1
    });

    Contacts API

    The contacts API is used to create or retrieve contacts on the device and is implemented in the org.apache.cordova.contacts plugin. To access this feature, run the following command:

    $ cordova plugin add org.apache.cordova.contacts

    Apps that access contacts must be privileged with the appropriate permission set in the manifest file. See “The Firefox OS Manifest” section earlier in this post to understand how to create a custom manifest for your application. For this API, you will need to add the following permissions to the manifest:

    "permissions": {
      "contacts": {
        "access": "readwrite",
        "description": "creates contacts"
      }
    }

    See the manifest documentation for specific access rights. In addition, you will need to change the type of app to privileged in the manifest.

    "type": "privileged",

    Once the manifest has been changed, you can add contacts with code like the following.

    // create a new contact object
    var contact = navigator.contacts.create();
    var name = new ContactName();
    name.givenName = fname;
    name.familyName = lname;
    contact.name = name;
    contact.save(onSuccess, onError);

    contact

    Geolocation API

    The geolocation API is used to retrieve the location, time and speed values from the devices GPS unit and is implemented in the org.apache.cordova.geolocation device plugin.

    $ cordova plugin add org.apache.cordova.geolocation

    You can retrieve the device latitude, longitude and a timestamp using this API on Firefox OS, but it does require the addition of a permission to the manifest file. See “The Firefox OS Manifest” section earlier in this post to understand how to create a custom manifest for your application.

    "permissions": {
        "geolocation": {
          "description": "Marking out user location"
        }
    }

    Adding this permission causes the app to prompt the user for permission to retrieve the GPS data. You can use either getCurrentPosition to read the GPS once or watchPosition to get an interval based update.

    var onSuccess = function (position) {
        console.log('Latitude: ' + position.coords.latitude + '\n' + 
        'Longitude: ' + position.coords.longitude + '\n'); 
    };
    function onError(error) {
        console.log('Error getting GPS Data');
    }
    navigator.geolocation.getCurrentPosition(onSuccess, onError);

    geolocation

    Join Us

    This post covered some of the basics of the new Firefox OS Cordova integration. We will continue to add more device APIs to the project, so stay tuned. If you are interested in working on the integration or need support for a specific device plugin, please contact us on Stack Overflow under the firefox-os tag or in #cordova on Mozilla IRC.

    In the meantime, if you have a Cordova app that makes use of the APIs discussed above, please try generating it for Firefox OS and submitting it to the Firefox Marketplace!

  6. HTML5, CSS3, and the Bookmarklet that Shook the Web

    On Valentine’s Day last year we released a bookmarklet that went viral riding the popularity of the Harlem Shake meme. On the anniversary of its release we’d like to take a moment look back at the technical nuts and bolts of the bookmarklet as a case study in applying HTML5. In fact, the HTML, JavaScript, and CSS we used wouldn’t have worked on a single browser a few years ago. What follows is a technical discussion on how we took advantage of recent browser developments to shake up the web.

    Background

    Last year the Harlem Shake meme forced itself on to nearly every screen under the sun and, like everyone else, we had joked about doing our office version of the video. After tossing around a few bad video ideas, Ishan half-jokingly suggested a bookmarklet that made a web page do the Harlem Shake. Omar and Hari immediately jumped on the ingenuity of his idea and built a prototype within an hour that had the entire office LOLing. After pulling a classic all nighter we released it on February 14th, declaring “Happy Valentine’s Day, Internet! Behold, the Harlem Shake Bookmarklet”.

    Pretty soon it was picked up by news outlets like TechCrunch and HuffingtonPost, and our traffic skyrocketed. Meanwhile the bookmarklet offered a new avenue of expression in the watch-then-remix cycle that is the lifeblood of a viral meme like the Harlem Shake. Instead of creating a video of people dancing, developers could now remix this symbiotic meme in code. Startups like PivotDesk incorporated the bookmarklet into their websites, and HSMaker used the code to build a Harlem-Shake-As-A-Service website. Eventually, YouTube even built their own version as an easter egg on their site.

    So, how does it work?

    Once you click the Harlem Shake bookmark, a snippet of JS is evaluated on the webpage, just as you’d see by entering javascript:alert(“Hi MozHacks!”); in your address bar. This JavaScript will play the Harlem Shake audio, “shake” DOM nodes (according to timing events attached to the audio), and remove all DOM changes afterward.

    How did we attach the audio to the page and get the timing for the shakes just right?

    HTML5’s extensive audio support made this implementation fairly easy. All that was required was inserting an <audio> tag with the src pointed to the Harlem_Shake.ogg file. Once inserted into the DOM, the file would begin downloading, and playback begins once enough of the file has been buffered.

    HTML5 timed audio events allow us to know exactly when playback begins, updates, and ends. We attach a listener to the audio node which evaluates some JS once the audio reaches certain time. The first node starts shaking once the song is beyond 0.5s. Then, at 15.5s, we flash the screen and begin shaking all of the nodes. At 28.5s, we slow down the animations, and once the audio has ended, we stop all animations and clean up the DOM.

    audioTag.addEventListener("timeupdate", function() {
      var time = audioTag.currentTime,
          nodes = allShakeableNodes,
          len = nodes.length, i;
     
      // song started, start shaking first item
      if(time >= 0.5 && !harlem) {
        harlem = true;
        shakeFirst(firstNode);
      }
     
      // everyone else joins the party
      if(time >= 15.5 && !shake) {
        shake = true;
        stopShakeAll();
        flashScreen();
        for (i = 0; i < len; i++) {
          shakeOther(nodes[i]);
        }
      }
     
      // slow motion at the end
      if(audioTag.currentTime >= 28.4 && !slowmo) {
        slowmo = true;
        shakeSlowAll();
      }
    }, true);
     
    audioTag.addEventListener("ended", function() {
      stopShakeAll();
      removeAddedFiles();
    }, true);

    How did we choose which parts of the page to shake?

    We wrote a few helpers to calculate the rendered size of a given node, determine whether the node is visible on the page, and whether its size is within some (rather arbitrary) bounds:

    var MIN_HEIGHT = 30; // pixels
    var MIN_WIDTH = 30;
    var MAX_HEIGHT = 350;
    var MAX_WIDTH = 350;
     
    function size(node) {
      return {
        height: node.offsetHeight,
        width: node.offsetWidth
      };
    }
    function withinBounds(node) {
      var nodeFrame = size(node);
      return (nodeFrame.height > MIN_HEIGHT &&
              nodeFrame.height < MAX_HEIGHT &&
              nodeFrame.width > MIN_WIDTH &&
              nodeFrame.width < MAX_WIDTH);
    }
    // only calculate the viewport height and scroll position once
    var viewport = viewPortHeight();
    var scrollPosition = scrollY();
    function isVisible(node) {
      var y = posY(node);
      return (y >= scrollPosition && y <= (viewport + scrollPosition));
    }

    We got a lot of questions about how the bookmarklet was uncannily good at iniating the shake on logos and salient parts of the page. It turns out this was the luck of using very simple heuristics. All nodes are collected (via document.getElementsByTagName(“*”)) and we loop over them twice:

    1. On the first iteration, we stop once we find a single node that is within the bounds and visible on the page. We then start playing the audio with just this node shaking. Since elements are searched in the order they appear in the DOM (~ the order on the page), the logo is selected with surprising consistency.
    2. After inserting the audio, we have ~15 seconds to loop through all nodes to identify all shakeable nodes. These nodes get stored in an array, so that once the time comes, we can shake them.
    // get first shakeable node
    var allNodes = document.getElementsByTagName("*"), len = allNodes.length, i, thisNode;
    var firstNode = null;
    for (i = 0; i < len; i++) {
      thisNode = allNodes[i];
      if (withinBounds(thisNode)) {
        if(isVisible(thisNode)) {
          firstNode = thisNode;
          break;
        }
      }
    }
     
    if (thisNode === null) {
      console.warn("Could not find a node of the right size. Please try a different page.");
      return;
    }
     
    addCSS();
     
    playSong();
     
    var allShakeableNodes = [];
     
    // get all shakeable nodes
    for (i = 0; i < len; i++) {
      thisNode = allNodes[i];
      if (withinBounds(thisNode)) {
        allShakeableNodes.push(thisNode);
      }
    }

    How did we make the shake animations not lame?

    We utilized and tweaked Animate.css’s library to speed up the process, its light and easy to use with great results.

    First, all selected nodes gets a base class ‘harlem_shake_me’ that defines animation parameters for duration and how it should apply the styles.

    .mw-harlem_shake_me {
      -webkit-animation-duration: .4s;
         -moz-animation-duration: .4s;
           -o-animation-duration: .4s;
              animation-duration: .4s;
      -webkit-animation-fill-mode: both;
         -moz-animation-fill-mode: both;
           -o-animation-fill-mode: both;
              animation-fill-mode: both;
    }

    The second set of classes that defines the animation’s behavior are randomly picked and assigned to various nodes.

    @-webkit-keyframes swing {
      20%, 40%, 60%, 80%, 100% { -webkit-transform-origin: top center; }
      20% { -webkit-transform: rotate(15deg); } 
      40% { -webkit-transform: rotate(-10deg); }
      60% { -webkit-transform: rotate(5deg); }  
      80% { -webkit-transform: rotate(-5deg); } 
      100% { -webkit-transform: rotate(0deg); }
    }
     
    @-moz-keyframes swing {
      20% { -moz-transform: rotate(15deg); }  
      40% { -moz-transform: rotate(-10deg); }
      60% { -moz-transform: rotate(5deg); } 
      80% { -moz-transform: rotate(-5deg); }  
      100% { -moz-transform: rotate(0deg); }
    }
     
    @-o-keyframes swing {
      20% { -o-transform: rotate(15deg); }  
      40% { -o-transform: rotate(-10deg); }
      60% { -o-transform: rotate(5deg); } 
      80% { -o-transform: rotate(-5deg); }  
      100% { -o-transform: rotate(0deg); }
    }
     
    @keyframes swing {
      20% { transform: rotate(15deg); } 
      40% { transform: rotate(-10deg); }
      60% { transform: rotate(5deg); }  
      80% { transform: rotate(-5deg); } 
      100% { transform: rotate(0deg); }
    }
     
    .swing, .im_drunk {
      -webkit-transform-origin: top center;
      -moz-transform-origin: top center;
      -o-transform-origin: top center;
      transform-origin: top center;
      -webkit-animation-name: swing;
      -moz-animation-name: swing;
      -o-animation-name: swing;
      animation-name: swing;
    }

    Shake it like a polaroid picture

    What started a joke ended up turning into its own mini-phenomenon. The world has moved on from the Harlem Shake meme but the bookmarklet is still inspiring developers to get creative with HTML5.

    If you want to see the full source code or have suggestions, feel free to contribute to the Github repo!

  7. The Pond – building a multi-platform HTML5 game

    The Pond

    Introducing The Pond

    The Pond is a multi-platform HTML5 game (source code) that explores minimalistic design and resolution independent gameplay. The Pond isn’t about reaching a high score, or about buying weapon upgrades. It’s about relaxing and exploring a beautiful world.

    It is available on all these platforms/in all these stores:

    • Firefox Marketplace
    • Google Play
    • Chrome Web Store
    • Amazon App store
    • Clay.io
    • Pokki

    In making The Pond I came across many performance obstacles which I will explore in detail (especially when optimizing the codebase for mobile).

    Tools

    Before I begin, I would like to mention the two tools that made coding The Pond both efficient and highly enjoyable: Light Table and CocoonJS.

    Light Table is an IDE (still in alpha) which provides an integrated development environment for real-time javascript code injection. This means that javascript edited within the editor can be previewed without reloading the page. If we look at the shape of the fish in the game we notice that it is comprised of Bézier curves. Instead of trying to find an editor for creating Bézier curves, I simply estimated a basic shape and modified the variables in real-time until I was satisfied with it’s look and feel.

    CocoonJS on the otherhand provides a canvas optimized compatibility layer for improved performance on mobile devices. Not only does it optimize, it also provides an interface for exporting our application to many devices (Android, iOS, Amazon (android), Pokki, and Chrome Web Store).

    Physics

    The Pond may seem simple on the outside, but on the inside it’s full of performance optimizations and responsive features. As we resize the game, it updates and re-optimizes itself to render less objects and spawn less fish, and if that’s not enough the framerate degrades smoothly to keep physics in check. This is thanks to the use of a fixed interval physics time step. Gameprogrammingpatterns.com provides a good explanation for how to do this and why it matters, but honestly the code makes the most sense:

    var MS_PER_UPDATE = 18; // Time between physics calculations
    var lag = 0.0; // accumulate lag over frames
    var previousTime = 0.0; // used for calculating the time delta
     
    // main game loop
    function draw(time) {
      requestAnimFrame(draw); // immidiately queue another frame
      lag += time - previousTime; // add time delta
      previousTime = time;
     
      var MAX_CYCLES = 18; // prevent infinite looping/hanging on slow machines
     
      // physics calculations
      while(lag >= MS_PER_UPDATE && MAX_CYCLES) {
     
        // user input, movement, and animation calculations
        physics();
        lag -= MS_PER_UPDATE;
        MAX_CYCLES--;
      }
     
      // if we exhausted our cycles, the client must be lagging
      if(MAX_CYCLES === 0) {
     
        // adaptive quality
        lowerQuality();
      }
     
      // if 5 frames behind after update, jump
      // this prevents an infinite input lag from ocurring
      if(lag/MS_PER_UPDATE > 75) {
        lag = 0.0;
      }
     
      // draw to canvas
      paint();
    }

    What’s important to notice here is that physics is not calculated based on the time delta, instead it’s calculated at a fixed 18ms interval. This is important because it means that any client lag will not be reflected in physics calculations, and that slower machines will simply lose framerate.

    Dynamic Quality

    The next optimization we notice is the lowerQuality() function, which adaptively decreases the render quality of the game. The way this works is simply by re-sizing the drawing canvas (it’s still full screen, it simply gets streched out), which in-turn leads to reduced spawns and collisions.

    function resizeWindow() {
     
      // quality is a global variable, updated by lowerQuality()
      $canv.width = window.innerWidth * quality/10
      $canv.height = window.innerHeight * quality/10
      ctx = $canv.getContext('2d')
      ctx.lineJoin = 'round'
     
      // resize HUD elements, and reduce spawning
      if(GAME.state === 'playing') {
        GAME.spawner.resize($canv.width, $canv.height)
        GAME.levelBar.resize($canv.width, $canv.height)
        GAME.levelBalls.resize($canv.width, $canv.height)
      } else {
        if(ASSETS.loaded) drawMenu()
      }
    }

    Spawning

    Now, we’ve been talking about reducing spawning to improve performance so let me explain how that happens. The spawning algorithm works by creating a virtual grid sized based on the window size. As the player travels from one grid zone to another, the adjacent zones are populated with enemies:

    Grid Spawner

    Spawner.prototype.spawn = function(zone) {
      // spawn 1-3  fish per 500sqpx, maybe larger maybe smaller than player
      // 0.5 chance that it will be bigger/smaller
      var mult = this.width*this.height/(500*500)
      for(var i=0, l=(Math.floor(Math.random()*3) + 1) * mult; i < l; i++) {
     
        // spawn coordinates random within a zone
        var x = zone[0]+Math.floor(this.width*Math.random()) - this.width/2
        var y = zone[1]+Math.floor(this.height*Math.random()) - this.height/2
        var size = Math.random() > 0.5 ? this.player.size + Math.floor(Math.random() * 10) : this.player.size - Math.floor(Math.random() * 10)
     
        // spawn a new fish
        var fish = new Fish(true, x, y, size, Math.random()*Math.PI*2-Math.PI, Math.random()*Math.PI)
     
        this.fishes.push(fish)
      }
      return zone
    }

    The last piece of the puzzle is removing enemies as they move far enough away:

    // if far enough away from player, remove
    if(distance(fish, player) > Math.max($canv.width, $canv.height) * 2) {
      fish.dead = true
    }

    Collisions

    The next performance optimization lies with the collision code. Colliding irregularly shaped objects can be extremely difficult and resource intensive. One option is to do color based collision (scan for overlapping colors), but that is much too slow. Another option might be to mathematically calculate Bézier curve collisions, however this is not only CPU intensive, it is also quite difficult to code. I finally opted for an approximation approach using circles. Basically I calculate the position of circles within each fish and detect circle collision among the fish. Boolean circle collision is extremely efficient, as it simply requires measuring the distance between objects. This ends up looking like this (debug mode):

    debug mode

    Fish.prototype.collide = function (fish) {
     
      // the fish has been killed and is being removed or it is far away
      if (this.dying || fish.dying || distance(this, fish) > this.size * 5 + fish.size*5) {
        return false
      }
     
      // there are 6 circles that make up the collision box of each fish
      var c1, c2
      for (var i=-1, l = this.circles.length; ++i < l;) {
        c1 = this.circles[i]
        for (var j=-1, n = fish.circles.length; ++j < n;) {
          c2 = fish.circles[j]
     
          // check if they touch
          if(distance(c1, c2) <= c2.r + c1.r) {
            return true
          }
        }
      }
     
      return false
    }

    We also avoid unnecessary collision calculations by only checking the fish that are visible (or near-visible):

    if(Math.abs(fish2.x - player.x) < $canv.width && Math.abs(fish2.y - player.y) < $canv.height) {
        // check
    }

    Drawing

    After getting the physics+ out of the way, it’s time to optimize drawing operations. Many games use sprite maps for animation (Senshi for example) which can be highly optimized. Unfortunately our fish are dynamically generated so we must find other ways to optimizing drawing. First lets use Chrome’s javascript profiler to identify bottlenecks:

    The Pond CPU profile

    What we see here is that stroke is using a lot of resources. Truth be told, fill used to be there too. This is because both were called heavily when drawing fish. The game looked a bit like this:

    The Pond - Old rendering

    After removing fill I saw a huge performance increase, and the game looked much better. The reason the drawImage function is up there as well is because I take advantage of offscreen canvas rendering. Each fish is drawn on its own offscreen canvas which is then rendered onto the larger visible canvas. This is also what allowed me to easily explode the fish into particles by reading pixel data:

    Fish.prototype.toParticles = function(target) {
      var particles = []
     
      // read canvas pixel data
      var pixels = this.ctx.getImageData(0,0,this.canv.width, this.canv.height).data
      for(var i = 0; i < pixels.length; i += 36 * Math.ceil(this.size/20) * (isMobile ? 6 : 1)) {
        var r = pixels[i]
        var g = pixels[i + 1]
        var b = pixels[i + 2]
     
        // black pixel - no data
        if(!r && !g && !b){
          continue
        }
     
        // Math to calculate position
        var x = i/4 % this.canv.width - (this.canv.width/2 + this.size)
        var y = Math.floor(i/4 / this.canv.width) - (this.canv.height/2)
        var relativePos = rot(x, y, this.dir)
        x=this.x + relativePos[0]
        y=this.y + relativePos[1]
     
        var col = new Color(r, g, b)
        var dir = directionTowards({x: x, y: y}, this)
        particles.push(new Particle(x, y, col, target, Math.PI*Math.random()*2 - Math.PI, this.size/20))
      }
      return particles
    }

    The End

    In the end the performance optimizations paid off and made the game feel more polished and playable even on lower-end mobile devices.

    If you enjoyed this post, I regularly blog about my development projects over at http://zolmeister.com.

    The Pond awaits exploring…

  8. Launching developer Q&A on Stack Overflow

    One thing that is very important for us at Mozilla is the need to directly interact with you developers and help you with challenges and issues while developing using open technologies. We are now happy to announce our presence on Stack Overflow!

    Stack Overflow is one of – if not the – most well-known question and answer sites among developers, and we believe it’s very important to go where developers are and choose to have their discussions while developing.

    Initially our main focus is on Firefox OS app development and Open Web Apps in general. There are three tags that we are monitoring:

    All tags and posts are available at our landing page on Stack Overflow: http://stackoverflow.com/r/mozilla

    Take part!

    We’d love for you to participate with both questions and answers, so we together can collaborate and make the Open Web and open technologies the first choice for developers!

    A little extra motivation might be that if you answer a lot of questions, you’ll show up in the Top Users page for that tag, e.g. http://stackoverflow.com/tags/firefox-os/topusers

    Resources that might be good for you about HTML5, Apps, Developer Tools and more are:

    See you on Stack Overflow!

  9. Getting Started With HTML5 Game Development

    There are plenty of valid ways to create an HTML5 game, and quite a bit of material on the technical aspect of each, so for this article I’ll be giving more of a broad overview of HTML5 game development. How “HTML5″ can be better than native, where to start with the development process, where to go when you’re stuck, and how to monetize and distribute games.

    Benefits of HTML5

    Most of the audience here already sees the value in HTML5, but I want to re-iterate why you should be building an HTML5 game. If you are just targeting iOS for your game, write the game in Objective-C, the cons outweigh the benefits in that scenario… but if you want to build a game that works on a multitude of platforms, HTML5 is the way to go.

    Cross-Platform

    One of the more obvious advantages of HTML5 for games is that the games will work on any modern device. Yes, you will have to put extra thought into how your game will respond to various screen sizes and input types, and yes, you might have to do a bit of ‘personalization’ in the code per platform (the main inhibitor being audio); but it’s far better than the alternative of completely porting the game each time.

    I see too many games that don’t work on mobile and tablets, and in most instances that really is a huge mistake to make when developing your game – keep mobile in mind when developing your HTML5 game!

    Unique Distribution

    Most HTML5 games that have been developed to this point are built in the same manner as Flash and native mobile games. To some extent this makes sense, but what’s overlooked is the actual benefits The Web as a platform adds. It’s like if an iOS developer were to build a game that doesn’t take advantage of how touch is different from a mouse – or if Doodle Jump was built with arrow keys at the bottom of the screen instead of using the device’s accelerator.

    It’s so easy to fall into the mindset of doing what has worked in the past, but that stifles innovation. It’s a trap I’ve fallen into – trying to 100% emulate what has been successful on iOS, Android, and Flash – and it wasn’t until chatting with former Mozillian Rob Hawkes before I fully realized it. While emulating what worked in the past is necessary to an extent, The Open Web is a different vehicle for games, and innovation can only happen when taking a risk and trying something new.

    Distribution for HTML5 games is often thought of as a weakness, but that’s just because we’ve been looking at it in the same sense as native mobile games, where a marketplace is the only way to find games. With HTML5 games you have the incredible powerful hyperlink. Links can so easily be distributed across the web and mobile devices (think of how many links you click in the Facebook and Twitter apps), and it certainly should not just be limited to the main page for the game. The technology is there to be able to link to your game and do more interesting things like jump to a specific point in a game, try to beat a friend’s score, or play real-time against that friend – use it to your advantage!

    Take a good look at was has worked for the virality of websites and apply those same principles to your games.

    Quicker Development Process

    No waiting for compilation, updates and debugging in real-time, and once the game is done, you can push out the update immediately.

    Choosing a Game Engine

    Game engines are just one more level of abstraction that take care of a few of the more tedious tasks of game development. Most take care of asset loading, input, physics, audio, sprite maps and animation, but they vary quite a bit. Some engines are pretty barebones, while some (ImpactJS for example) go as far as including a 2D level editor and debug tools.

    Decide Whether or Not You Need a Game Engine

    This is largely a personal decision. Game Engines will almost always reduce the time it takes for you to create a fully-functional game, but I know some folks just like the process of building everything from the ground up so they can better understand every component of the game.

    For simple games, it really isn’t difficult to build from scratch (assuming you have a JavaScript background and understand how games work). Slime Volley (source) for example was built without having a game engine, and none of the components were rocket science. Of course, Slime Volley is a very basic game, building an RPG from the ground up would likely lead to more hair pulling.

    Choosing Between a “Game Engine” and a “Game Maker”

    Most of the typical audience of Mozilla Hacks are probably going to lean toward using a game engine or building from scratch, but there is also the alternative of using a “Game Maker” like Construct 2. Using a Game Maker means you won’t actually write in JavaScript; instead, you create code-like events in the editor. It’s a trade of ease-of-use and quickness to prototype/develop vs customization and control over the end result. I’ve seen some very impressive games built with either, but as a developer-type, I tend to favor writing from scratch / using an engine.

    Finding the Right Game Engine / Game Maker for you

    There are so many HTML5 game engines out there, which in part is a good thing, but can also be a bad thing since a large percentage have either already stopped being maintained, or will soon stop being maintained. You definitely want to pick an engine that will continually be updated and improved over the years to come.

    HTML5GameEngine.com is a great place to start your search because the hundreds of game engines are narrowed down to about 20 that are established, actively maintained, and have actual games being developed with them.

    For a more complete list of engines (meaning there can be some junk to sift through), this list on GitHub is your best bet.

    Learning Tools

    If you’re going with a game engine, typically their site is the best resource with tutorials and documentation.

    Technical Tutorials

    Game Design Tutorials

    With game development, the technical aspect isn’t everything – what’s more important is that the game actually be fun. Below are a few places to start when learning about game mechanics.

    Helpful Game Tools

    User Retention, Monetization and more

    Full disclosure here: I am a co-founder at Clay.io.

    Making a game function is just part of the equation. You also want players to play longer, come back, tell their friends about it, and maybe even buy something. Common elements in games that focus on these areas are features like user accounts, high scores, achievements, social integration, and in-game payments. On the surface most are typically easy enough to implement, but there are often many cross-platform issues and intricacies that are overlooked. There is also value in having a central service running these across many games – for example, players genuinely care about achievements on Xbox Live because Gamerscore matters to them.

    • Clay.io – user accounts, high scores, achievements, in-game payments, analytics, distribution, and more.
    • Scoreoid – similar to above.

    Development Tools

    • stats.js – A JavaScript performance monitor. Displays framerate, and performance over time.
    • Socket.IO – realtime client-server communication (if you’re going to have a backend for your game).
    • pixi.js – A canvas and WebGL rendering engine.
    • CocoonJS – Improves HTML5 game performance on iOS and Android with an accelerated canvas bound to OpenGL ES.

    Motivation

    Regardless of what you’re building, extra motivation is always helpful. For games, that motivation often comes from surrounding yourself with others who are in the same boat as you – working on games.

    js13kGames

    js13kGames is a competition that is currently taking place at the time of this writing. You have until September 13th, 2013 to develop an HTML5 game that, when compressed, is less than 13kb.

    Mozilla Game On

    Mozilla runs a game competition every year from December through February with some fantastic prizes – last year’s was an all-expense paid, red carpet trip to San Francisco for GDC 2013.

    Clay.io’s “Got Game?”

    Clay.io (full disclosure, I am a founder) runs an annual HTML5 game development competition for students. Last year was the first year and we had over 70 games submitted. The next competition is planned for February / March 2014.

    Ludum Dare

    Ludum Dare isn’t for tangible prizes, nor is is specific to HTML5 games, but there are plenty of HTML5 developers that participate.

    One Game a Month

    One Game a Month isn’t so much a competition as it is an accountability tool. This isn’t restricted to HTML5 games, but many of the participants work with HTML5. The goal is to crank out one game every month. I wouldn’t recommend this long-term since one month is too short of a time to create a great game, but it’s good when learning to force yourself to develop and finish simple games.

    Help From the Community

    HTML5GameDevs.com

    HTML5GameDevs has quickly become the most active community of HTML5 game developers. Most folks are very friendly and willing to help with any issues you run into.

    #BBG

    #BBG is the go-to IRC channel for HTML5 games – you’ll even find quite a few Mozillians hanging around.

    How to Make Money

    In-Game Purchases

    In-game payments, in my opinion, are the way to go for HTML5 game in the long-term. For now, most HTML5 games don’t have enough quality content, nor the game mechanics in place to get player purchasing items.

    This is the revenue model with the highest potential, but it’s also the most difficult to achieve by far – not technically, but having the right game. I’d say the best way to learn how to properly monetize your game in this aspect is to take a look at games that do it really well on Flash and Mobile – games from King.com and Zynga typically have this nailed down pretty well. There’s also some good reading material, like The Top F2P Monetization Tricks on Gamasutra.

    Licensing

    Where we’re at right now with HTML5 games, licensing games is the strongest, most consistent way to make money – if and only if your game works well on mobile devices.

    There are countless “Flash Game Portals” that receive organic mobile traffic, but can’t monetize it with the Flash games they have. Their solution is to go out and find HTML5 games to buy non-exclusive licenses (the right to put the game on their site, often making small adjustments) to offer their mobile visitors.

    Typically non-exclusive HTML5 game licenses (meaning you can sell to more than one site) go for $500-$1,000 depending on the game and publisher. Some publishers will do a revenue share model instead where you’ll get a 40-50% share on any advertising revenue, but no up-front money.

    Licensing is the safest way to make money right now, but the cap is limited – the most you’re going to make with a single game is in the $5,000-$6,000 range, but it is easier to hit that mark than it is with in-game payments or advertising.

    Advertising

    Advertising is the middle-ground between in-game payments and licensing. It’s easier than in-game payments to make money and with a higher potential cap than licensing (but probably less than in-game payments). It’s easy enough to implement ads: just pick your ad network of choice (be wary of Adsense’s strict terms) and implement them either surrounding the game, or at various stopping points.

    The commonly used ad networks are LeadBolt for mobile and CPMStar for desktop. You can also use Clay.io which makes it a bit easier to implement advertising, and tries to maximize the revenue by using different ad networks depending on the device used and other factors.

    Distribution

    The final stage in a game’s development is distribution. The game is done, now you want people playing the game! Fortunately, with HTML5 there are plenty of places to have your game (many of which often go unused).

    More and more marketplaces these days are accepting HTML5 games as-is. Each has their own requirements (Facebook requires SSL, most require a differently formatted manifest file, etc…), but the time it takes to get into each is typically less than 30 minutes. If you want to reduce that even more, Clay.io helps auto-generate the manifest files and promotional image assets you’ll need (as well as takes care of the SSL requirement) – documentation on that here.

    Some marketplaces you’ll need to have a native wrapper for your game – primarily the iOS App Store and Google Play. A wrapper like PhoneGap is one option, but the native webviews have pretty terrible JavaScript engines, so for now you’re better off with tools like CocoonJS and Ejecta.

    Now it’s up to you to go forth and make a great, innovative web game – I’m looking forward to see what’s on the horizon in the coming months and years!

  10. The Making of Face to GIF

    Face to gif is a simple webapp that lets you record yourself and gives you an infinitely looping animated gif. In this post I will walk you through how it came to be and what I’ve learned from building the small app.

    image of the preview window in face to gif

    It started with Chris Heilmann’s post about people losing expressiveness to internet memes. At least, that was what I wanted to understand out of it. I thought it really came down to tooling, like most problems do.

    It is the year 2000 and something and we still haven’t found a solution to simple problems like sending large files, doing taxes automatically and reliably online or recording an animated gif in your browser. Also, because memes are so popular and easily accessible, why would people even bother trying to create original content when they can make do with a cute kitten image. I thought some things should be easier.

    I had already played around with downloading files generated on the client, so I knew text files were trivial and static images were not that hard. But I didn’t find anything about making gif files client side. I thought that I’d figure out the gif part later or even write it myself – how hard could it actually be, right?

    The Humble Beginning

    Since WebRTC is gaining traction, getUserMedia is becoming a somewhat viable API. Getting a stream from a webcam to be displayed on a video element was very easy.

    navigator.getUserMedia({video: true, audio: false}, yes, no);
    …
    video.src = URL.createObjectURL(stream);

    I then needed to capture the images that would later make up the gif’s frames. This was not that hard, either. Luckily, you can paint a video element on a canvas context directly using

    context.drawImage(video, 0,0, width,height);

    This also allows you to scale the captured frames right there, to normalise the different webcam resolutions. Just make sure your canvas element has the correct width and height properties specified, and you should be fine. Also, you should either display: none; it or remove it from the DOM to avoid unnecessary paints.

    <canvas width=320 height=240></canvas>

    To capture frames, just set an interval at your desired frame rate and cache the frames in an array.

    setInterval(function () {
      context.drawImage(video, 0,0, width,height);
      frames.push(context.getImageData(0,0, width,height));
    }, 67);

    Please note that there is no need to use requestAnimationFrame in this case. The video stream continues to play even when the page it’s on is not visible – so I guess capturing it also makes sense. More importantly, you will need a specific interval between frames that will most probably not end up being 60 frames per second.

    After stopping the interval – that is to stop “recording” – you are left with a lot of frames, each frame having a lot of pixel data from the video stream that comes from your webcam. And all that data never leaves the web page that’s being displayed on your computer.

    At one point, I was considering to add a “download raw data” button so people could do other things than just make a gif of themselves. I decided to actually solve the gif part first, then think about bells and whistles.

    The GIF Writer

    After reading too much about the GIF89a and dithering and the LZW algorithm, I cowardly decided to see if I could not find a ready made library. I was lucky to find a demo that combined a series of images into an animated gif – all in JavaScript. I quickly retrofitted the library into my small app and things started moving again.

    gifworker.sendMessage({ images: frames, delay: 67 });
    ...
    gifworker.onmessage = function(event) {
      var img = document.createElement('img');
      img.src = event.data.gifDataString;
      document.body.appendChild(img);
    };

    What needs to be done from there is as follows:

    1. write a binary header that describes a file as a GIF98a file.
    2. write a block describing the width, height and looping control.
    3. write each frame from the image data list.
    4. write a trailer \59 – aka semicolon.

    Using WebWorkers to do the heavy lifting in a separate thread, keeping the UI responsive was a no-brainer. After it’s done processing, the library provides you with a base64 encoded string representing the gif file. That can be used as a data url for an image.

    At one point in my life, I was using data urls so intensively, that I would provide clients with mockups consisting of just one HTML file that had images, css and javascript base64`d in and that wouldn’t require and internet connection to work. And that wouldn’t work in IE.

    But I was about to face a different set of problems this time around…

    Saving the files

    Data urls can be saved if they’re small enough. If you want to save a gif that is too long and displayed via data url, the browser will not even let you try do that. Trying to be clever with the download attribute on links didn’t help either.

    image of the generated gif and its options in face to gif

    While data urls are really cool, there is a limit to how long they can be. I didn’t want to impose what seemed to be a legacy limit on this app.

    I altered the library a little to provide me with the raw bytes instead of a base64 string and I used the raw bytes to create a Blob, then used URL.createObjectURL to make something I could set an image’s source attribute to.

    var blob = new Blob([uint8array], {type: "image/gif"});
    img.src = URL.createObjectURL(blob);

    This method of using user generated resources as source attributes is much more reliable and scalable than the old data url method. This also allowed for easier saving of the image.

    I use a trick for the download link you will find in my app: I place a simple anchor, with an empty href attribute and I attach a simple ‘click’ event handler. When the user clicks on it, my event handler function simply changes the href attribute to be the same with the source attribute of the image. The browser does the rest.

    a.addEventListener('click', function (e) {
        a.href = img.src;
      // the real trick is to let the event bubble up
    }, false);

    We spend so much time as web developers hijacking control from the browser so we may do our own thing. The truth is, we can most of the time just tell the browser where to go and he’ll do a much better job at getting there on its own than if we would be involved.

    Getting back to my app, though, I had gotten it to a place where it was doing what I hoped it would be doing: Recording my face with my webcam and serving me a gif of it.

    The Speed

    The app was rocking, but it was more like a ballad than a heavy metal song. It took 16 seconds of my life for each 1 second of a gif. This was also because I was writing the gif files at 640×480 originally, but also because it turns out that binary operations on pixel data can be quite slow if not optimised.

    I was scrambling for solutions, looking into the library’s TypeScript source code and the generated JavaScript to find ways to improve it, considering asm.js, using TypedArrays more, anything – when I stumbled upon another JavaScript library for writing animated gifs.

    gif.js was leaner, could use several web workers to process the frames and had what I thought was a better looking API. After retrofitting this new library, tweaking the settings and halving the size, I was able to produce gifs, right in the web app at blazing speeds.

    The one downfall was that what I had gained in speed, I had lost in compression. A mere 10 seconds of GIF would produce about 30 MB worth of data. After some more heavy tweaking, I was able to get that down to about 5 MB / 10s. Still a lot, but it is uncompressed and aggressive compression via online tools can bring that down to as little as 600KB.

    The other cool thing about working with Blobs is that you can append them directly to FormData objects, which meant that doing a cross origin ajax call to imgur.com to upload the generated gif was a breeze, and a much welcomed addition to the web app.

    What I’ve learned

    • URL.createObjectURL is a great api for client generated media, solving so many problems you’d have otherwise.
    • Using TypedArrays will boost your data intensive app’s performance a lot.
    • Dividing workload between multiple concurrent WebWorkers actually works and helps.
    • WebRTC is at a pretty stable point where you can use the media devices of about 40% of internet users.
    • It is easy to make an app that lets users generate content without involving your server.
    • People really like playing with their web cams. I think using them in a web app makes perfect sense.
    • It is easy to fill up 2MB, imgur’s file limit, with gif data.

    thumbs up!

    I would also like to thank Johan Nordberg and nobuoka for their hard work coming up with their JavaScript gif writing libraries.

    You can take face to gif for a quick spin, or look at the source code on github and fork it, improve it and have lots of fun; Just like I did.

    I cannot wait until WebRTC becomes really available on mobile devices too!