Mozilla

Featured Articles

Sort by:

View:

  1. Introducing navigator.mozPay() For Web Payments

    What’s wrong with payments on the web? Any website can already host a shopping cart and take credit card payments or something similar. The freedom of today’s web supports many business models. Here’s what’s wrong:

    • Users cannot choose how to pay; they have to select from one of the pre-defined options.
    • In most cases, the user has to type in an actual credit card number on each site. This is like giving someone the keys to your expensive car, letting them drive it around the block in a potentially dangerous neighborhood (the web) and saying please don’t get carjacked!
    • Merchants typically have to manage all this on their own: payment processor setup, costly processing fees, and possibly even PCI compliance.

    There are services to mitigate a lot of these complications such as PayPal, Stripe, and others but they aren’t integrated into web devices very well. Mozilla wants to introduce a common web API to make payments easy and secure on web devices yet still as flexible as the checkout button for merchants.

    As a first step, Mozilla will introduce navigator.mozPay() in Firefox OS so that web apps can accept payments.

    How Does It Work?

    navigator.mozPay() is a JavaScript API inspired by google.payments.inapp.buy() but modified for things like multiple payment providers and carrier billing. When a web app invokes navigator.mozPay() in Firefox OS, the device shows a secure window with a concise UI. After authenticating, the user can easily charge the payment to her mobile carrier bill or credit card. When completed, the app delivers the product. Repeat purchases are quick and easy.

    In an earlier article I talked about purchasing an app and receiving a receipt. navigator.mozPay() is different in that there is no concept of what product is purchased, it’s just an API to facilitate a payment for a digital good or service, whatever that may be.

    The payment starts and finishes in the client but further processing and notifications happen server side. This article briefly explains how it all fits together. For complete, in-depth documentation read the Firefox Marketplace guide to in-app payments.

    Integrating With A Payment Provider

    Multiple providers will facilitate payments behind the scenes of navigator.mozPay(). For example, the Firefox Marketplace will be able to facilitate payments.

    As a developer you will essentially grant permission to each provider that you would like to sell through. In the current design of the API, you do this by asking each provider for an Application Key and an Application Secret so that you can digitally sign payment requests. A signed request prevents unauthorized parties from selling your products and prevents users from tampering with the price, etc.

    Initiating A Payment

    When the user clicks a Buy button in your web app, you create a JSON Web Token (JWT) with your Application Secret. If you have agreements with multiple providers, you would create a JWT for each provider. Starting a payment looks roughly like this:

    document.querySelector('button.buy').onclick = function() {
        navigator.mozPay([mozillaJWT, otherJWT, ...]);
    };

    Defining A Product

    A JWT is a signed JSON object that defines details like the product name and price. Here is an example with some attributes removed for brevity:

    {
      "iss": APPLICATION_KEY,
      "aud": "marketplace.firefox.com",
      ...
      "request": {
        "name": "Magical Unicorn",
        "pricePoint": 1,
        "postbackURL": "https://yourapp.com/postback",
        "chargebackURL": "https://yourapp.com/chargeback"
      }
    }

    You define prices as price points so that the payment provider can handle currency conversions for you in each region. In this example, pricePoint 1 might be €0.89 or $0.99, etc. Micropayments in small amounts will be supported. Consult the navigator.mozPay() spec for details on how to construct a payment request JWT.

    Completing a Payment

    To complete the payment, you need to wait for the provider to POST a result to your server’s postbackURL (on success) or chargebackURL (on failure). A more complete example of requesting a purchase in JavaScript would involve waiting for the postback to arrive, like this:

    var request = navigator.mozPay([mozillaJWT, otherJWT]);
    request.onsuccess = function() {
      // The payment window has closed.
      whenPaymentResultReceived(function() {
        console.log('Success! User has purchased a Magical Unicorn.');
      });
    };

    To implement whenPaymentResultReceived() you might open a web socket to your server, wait for the payment result, and verify the incoming JWT signature. The navigator.mozPay() spec has details on how postback and chargeback notifications work.

    Try It Out

    Payments aren’t fully live yet in the Firefox Marketplace but you can simulate a payment to test out your code. Log into the Firefox Marketplace Developer Hub and generate an Application Key and Application Secret for simulations. With those keys you can add a special parameter to the JWT like this:

    {
      "request": {
        "name": "Magical Unicorn",
        "pricePoint": 1,
        ...
        "simulate": {"result": "postback"}
      }
    }

    This will show a payment window on Firefox OS but it won’t charge you real money. It will let you test your client side JavaScript code and your server postback handlers to make sure everything is integrated smoothly. When you go live, just remove the simulate attribute. If you’re new to Firefox OS development, check out the Firefox OS Simulator.

    If you’re already working on a game or a web app for Firefox OS try thinking about using navigator.mozPay() to offer premium content.

    This Would Be Way Easier With A Library

    We thought so too! We built libraries for Node.JS and Python to make the server side logic for navigator.mozPay() as easy as possible. Libraries for more languages are on the way. We also are experimenting with removing the server prerequisite entirely.

    Current Status

    As you can probably tell by the prefix, navigator.mozPay() is an experimental API and might change drastically or become unprefixed without notice. It will process live payments on the first Firefox OS phones and evolve quickly from real world usage.

    Mozilla plans to work with other vendors through the W3C to reach consensus on a common API that supports web payments in the best way possible. After shipping in Firefox OS, Mozilla plans to add navigator.mozPay() to Firefox for Android and desktop Firefox.

    New Business Models

    Advertising has been the primary business model on the web for a long time but users have made it clear that they don’t want to see ads. Mozilla isn’t trying to directly disrupt the ad business but it is trying to fix serious privacy issues relating to ad networks.

    What if users explicitly paid for content instead? navigator.mozPay() enables this kind of direct payment model: if something is good on the web, you can pay for it. It already seems to be working well for existing mobile apps. Will mobile ads even generate the same revenue for content producers as they do on desktop? I don’t have answers to these questions but one thing is for certain: the web should support businesses of all kinds and payments should be a first class feature of the web.

    Is It Webby?

    Mozilla’s main goal with navigator.mozPay() is to give users and merchants choice, security, and an easy to use payments system. The details about how merchants interact with payment providers is not yet specified in the API and that is clearly a gap. The first Firefox OS phones will ship with a whitelist of allowed payment providers which is also not ideal.

    In a more webby model, all parties involved in the payment would be fully decentralized so that innovation can occur naturally and unknown payment providers could emerge. Who will be the next M-Pesa? An elegant API would support that. Building a secure decentralized payment API is an ambitious challenge; the solution would need to address these core trust issues:

    • How can customers trust that they will receive the goods after paying?
    • How would customers ensure that their payment credentials are handled securely?
    • How do merchants guarantee that they’ll get paid after delivering goods?

    As with anything related to money, there is incentive for fraud every step of the way. BitCoin is a digital currency that solves some of these trust issues with block chains and PaySwarm is a web payment protocol that solves some of these issues with decentralized assets, public keys, etc. Mozilla will be watching PaySwarm as well as other models and hopefully navigator.mozPay() can incorporate some of these concepts eventually.

  2. How to install packaged apps in Firefox OS – options and tools

    I thought this would be a good time to show the options for developers how to install packaged apps, and as an extension to that, installing them on an actual Firefox OS device (more on Open Web apps, if it’s new to you).

    Why a packaged app?

    First and foremost, a packaged app is where all resources in your app is zipped into one file, that gets installed directly on the device. That means that all the files are available directly, without needing an Internet connection.

    The other benefit is that if you need to access some privileged APIs for your app, it needs to be a packaged app to get access to those (for security reasons, since code hosted anywhere on the web naturally can’t have access to sensitive APIs on the device).

    The different APIs are outlined in the Using WebAPIs to make the web layer more capable article and the permissions needed for each API are listed in App permissions.

    Testing in the Firefox OS Simulator

    In the Firefox OS Simulator – latest features listed in previewing version 3.0 – you have the option to install your app as a packaged app.

    In the Simulator Dashboard, you have a button with the label Add Directory. When you do that, and point to the manifest.webapp for your app, it automatically gets installed as a packaged app. Simple as that!

    image

    Not all APIs are supported in the Firefox OS Simulator, but it’s a good start!

    When you push the update button, the app should be refreshed and restarted if already running, and started if not running. If the content or the manifest is not updated after an “update” click, there’s a blocking error, shown in the Simulator Dashboard.

    Push to Device in the Simulator

    As we described here before, one of the new features in version 3.0 of the Firefox OS Simulator is the feature Push to Device.

    If you have Firefox OS device connected when you use the Firefox OS Simulator, you will get a Push button next to each app in the Simulator Dashboard to push it directly to your app.

    image

    You will then need to accept that incoming request on the device to install the app.

    image

    Manual steps

    • Once you’ve pushed the app to the device, you need to manually close and restart it again, to get updated content
    • If you update anything in the manifest, e.g. app name, orientation, you need to reboot the operating system for those changes to have effect

    Things to think about

    • Remote debugging has to be enabled on the device, via
      Settings > Device information > More Information > Developer > Remote debugging
    • For Windows support, make sure to install the necessary drivers and that you use the latest version of the Firefox OS Simulator (currently, that means the 3.0 preview).
    • On Linux (at least Ubuntu), you must create the file /etc/udev/rules.d/51-android.rules as root and then add a manufacturer-specific entry for the device as described by Android’s Setting up a Device for Development. Example for one of our test devices:entry:
      SUBSYSTEM=="usb", ATTR{idVendor}==" 19d2", MODE="0666", GROUP="plugdev"
    • Make sure you have the latest version of Firefox OS on your device (especially due to recent fixes like bug 842725)

    Use make-fxos-install and command line

    If you want to establish a good workflow and you’re a fan of the terminal, we’d like to recommend using make-fxos-install by Harald Kirschner. It’s a command-line tool to easily push apps directly to a device.

    Step 1

    Step 2

    Clone the make-fxos-install repository.

    Step 3

    Go into your local make-fxos-install directory on the command line.

    Choose what app type to install

    Your two options are packaged or hosted. You need to call a make command and point to the folder of your app. For example:

    make FOLDER=../Firefox-OS-Boilerplate-App packaged install

    Alternatively:

    make FOLDER=../Firefox-OS-Boilerplate-App hosted install

    Accept the prompt on the device, and that’s it!

    image

    Please note that the packaged app will have all files in your app, while the hosted app alternative only needs two files to be installed:

    • A manifest.webapp file
    • A metadata.json file

    Then in the metadata.json file you only need to set the manifestURL parameter from where the app should be served.

    Manual steps

    The same steps as described above for Push to Device in the Firefox OS Simulator applies:

    Close and restart app to see changes, and an updated manifest = rebooting the OS to see any of those changes.

    Test it out!

    Please test out the above solutions and see what works best for you (we realize it might be a little while before you get access to an actual device). let us know what works and what doesn’t – we’d love to hear your feedback!

  3. Adding cursor swipe to the Firefox OS keyboard

    In this article we will take a look at how to approach adding features to a core component in the system such as the input keyboard. It turns out it is pretty easy!

    Before we start, take a look at this concept video from Daniel Hooper to get an idea of what we want to implement:

    Cool, huh? Making such a change for other mobile platforms would be pretty hard or just plain impossible, but in Firefox OS it is quite simple and it will take us less than 50 lines of code.

    The plan

    Conceptually, what we want to achieve is that when the user swipes her finger on the keyboard area, the cursor in the input field moves a distance and direction proportional to the swiping, left or right.

    Since a common scenario is that the user might be pressing a wrong key and would like to slide to a close-by key to correct it, we will only start moving the cursor when the swipe distance is longer than the width of a single key.

    Preparing your environment

    In order to start hacking Firefox OS itself, you will need a copy of Gaia (the collection of webapps that make up the frontend of Firefox OS) and B2G desktop (a build of the B2G app runtime used on devices where all apps should run as they would on a device).

    You can take a look at this previous article from Mozilla Hacks in which we guide you through setting up and hacking on Gaia. There is also a complete guide to setting up this environment at https://wiki.mozilla.org/Gaia/Hacking.

    Once you get Gaia to run in B2G, you are ready to hack!

    Ready to hack!

    Firefox OS is all HTML5, and internally it is composed by several ‘apps’. We can find the main system apps in the apps folder in the gaia repository that you cloned before, including the keyboard app that we will be modifying.
    In this post we will be editing only apps/keyboard/js/keyboard.js, which is where
    a big chunk of the keyboard logic lives.

    We start by initializing some extra variables at the top of the file that will help us keep track of the swiping later.

    var swipeStartMovePos = null; // Starting point of the swiping
    var swipeHappening = false; // Are we in the middle of swiping?
    var swipeLastMousex = -1; // Previous mouse position
    var swipeMouseTravel = 0; // Amount traveled by the finger so far
    var swipeStepWidth = 0; // Width of a single keyboard key

    Next we should find where the keyboard processes touch events. At
    the top of keyboard.js we see that the event handlers for touch events are
    declared:

    var eventHandlers = {
      'touchstart': onTouchStart,
      'mousedown': onMouseDown,
      'mouseup': onMouseUp,
      'mousemove': onMouseMove
    };

    Nice! Now we need to store the coordinates of the initial touch event. Both onTouchStart and onMouseDown end up calling the function startPress after they do their respective post-touch tasks, so we will take care of storing the coordinates there.

    startPress does some work for when a key is pressed, like highlighting the key or checking whether the user is pressing backspace. We will write our logic after that. A convenient thing is that one of the arguments in its signature is coords, which refers to the coordinates where the user started touching, in the context of the keyboard element. So storing the coordinates is as easy as that:

    function startPress(target, coords, touchId) {
      swipeStartMovePos = { x: coords.pageX, y: coords.pageY };
      ...

    In that way we will always have available the coordinates of the last touch even starting point.

    The meat of our implementation will happen during the mousemove event, though. We see that the function onMouseMove is just a simple proxy function for the bigger movePress function, where the ‘mouse’ movements are processed. Here is where we will write our cursor-swiping logic.

    We will use the width of a keyboard key as our universal measure. Since the width of keyboard keys changes from device to device, we will first have to retrieve it calling a method in IMERender, which is the object that controls how the keyboard is rendered on the screen:

    swipeStepWidth = swipeStepWidth || IMERender.getKeyWidth();

    Now we can check if swiping is happening, and whether the swiping is longer than swipeStepWidth. Conveniently enough, our movePress function also gets passed the coords object:

    if (swipeHappening || (swipeStartMovePos && Math.abs(swipeStartMovePos.x - coords.pageX) > swipeStepWidth)) {

    Most of our logic will go inside that ‘if’ block. Now that we know that swiping is happening, we have to determine what direction it is going, assigning 1 for right and -1 for left to our previously initialized variable swipeDirection. After that, we add the amount of distance traveled to the variable swipeMouseTravel, and set swipeLastMousex to the current touch coordinates:

    var swipeDirection = coords.pageX > swipeLastMousex ? 1 : -1;
     
    if (swipeLastMousex > -1) {
      swipeMouseTravel += Math.abs(coords.pageX - swipeLastMousex);
    }
    swipeLastMousex = coords.pageX;

    Ok, now we have to decide how the pixels travelled by the user’s finger will translate into cursor movement. Let’s make that half the width of a key. That means that for every swipeStepWidth / 2 pixels travelled, the cursor in the input field will move one character.

    The way we will move the cursor is a bit hacky. What we do is to simulate the pressing of ‘left arrow’ or ‘right arrow’ by the user, even if these keys don’t even exist in the phone’s virtual keyboard. That allows us to move the cursor in the input field. Not ideal, but Mozilla is about to push a new Keyboard IME API that will give the programmer a proper API to manipulate curor positions and selections. For now, we will just workaround it:

    var stepDistance = swipeStepWidth / 2;
    if (swipeMouseTravel > stepDistance) {
      var times = Math.floor(swipeMouseTravel / stepDistance);
      swipeMouseTravel = 0;
      for (var i = 0; i < times; i++)
        navigator.mozKeyboard.sendKey(swipeDirection === -1 ? 37 : 39, undefined);
    }

    After that we just need to confirm that swiping is happening and do some cleanup of timeouts and intervals initialized in other areas of the file, that because of our new swiping functionality ouldn’t get executed otherwise. We also call hideAlternatives to avoid the keyboard to present us with alternative characters while we are swiping.

    swipeHappening = true;
     
    clearTimeout(deleteTimeout);
    clearInterval(deleteInterval);
    clearTimeout(menuTimeout);
    hideAlternatives();
    return;

    The only thing left to do is to reset all the values we’ve set when the user lifts her finger off the screen. The event handler for that is onMouseUp, which calls the function endPress, at the beginning of which we will put our logic:

    // The user is releasing a key so the key has been pressed. The meat is here.
    function endPress(target, coords, touchId) {
        swipeStartMovePos = null;
        ...
        if (swipeHappening === true) {
            swipeHappening = false;
            swipeLastMousex = -1;
            return;
        }

    With this last bit, our implementation is complete. Here is a rough video I’ve made with the working implementation:

    You can see the complete implementation code changes on GitHub.

    Conclusion

    Contributing bugfixes or features to Firefox OS is as easy as getting Gaia, B2G and start hacking in HTML5. If you are comfortable programming in JavaScript and familiar with making web pages, you can already contribute to the mobile operating system from Mozilla.

    Appendix: Finding an area to work on

    If you already know what bug you want to solve or what feature you want to implement in Firefox OS, first check if it has already been filed in Bugzilla, which is the issue repository that Mozilla uses to keep track of bugs. If it hasn’t, feel free to add it. Otherwise, if you are looking for new bugs to fix, a quick search will reveal many new ones that are sill unassigned. Feel free to pick them up!

  4. Making WebRTC Simple with conversat.io

    WebRTC is awesome, but it’s a bit unapproachable. Last week, my colleagues and I at &yet released a couple of tools we hope will help make it more tinkerable and pose a real risk of actually being useful.

    As a demo of these tools, we very quickly built a simple product called conversat.io that lets you create free, multi-user video calls with no account and no plugins, just by going to a url in a modern browser. Anyone who visits that same URL joins the call.

    conversat.io

    The purpose of conversat.io is two fold. First, it’s a useful communication tool. Our team uses And Bang for tasks and group chat, so being able to drop a link to a video conversation “room” into our team chat that people can join is super useful. Second, it’s a demo of the SimpleWebRTC.js library and the little signaling server that runs it, signalmaster.

    (Both SimpleWebRTC and signalmaster are open sourced on Github and MIT licensed. Help us make them better!)

    Quick note on browser support

    WebRTC currently only works in Chrome stable and FireFox Nightlies (with the media.peerconnection.enabled preference enabled in about:config).

    Hopefully we’ll see much broader browser support soon. I’m particularly excited about having WebRTC available on smartphones and tablets.

    Approachability and adoption

    I firmly believe that widespread adoption of new web technologies is directly corellated to how easy they are to play with. When I was a new JS developer, it was jQuery’s approachability that made me feel empowered to build cool stuff.

    My falling in love with javascript all started with doing this with jQuery:

    $('#demo').slideDown();

    And then seeing the element move on my screen. I knew nothing. But as cheesy as it sounds, this simple thing left me feeling empowered to build more interesting things.

    Socket.io did the same thing for people wanting to build apps that pushed data from the server to the client:

    // server:
    client.emit("something", {
        some: "data"
    });
    // client:
    socket = io.connect();
    socket.on("something", function (data) {
        // here's my data!
        console.log(data);
    });

    Rather than having to figure out how to set up long-polling, BOSH, and XMPP in order to get data pushed out to the browser, I could now just send messages to the browser. In fact, if I didn’t want to, I didn’t even have to think about serializing and de-serializing. I could now just pass simple javascript objects seamlessly back and forth between the client and server.

    I’ve heard some “hardcore” devs complain that tools like this lead to too many poorly made tools and too many “wannabe” developers who don’t know what they’re doing. That’s garbage.

    Approachable tools that make developers feel empowered to build cool stuff is the reason the web is as successful and vibrant as it is.

    Tools like this are the gateway drug for getting us hooked on building things on these types of technologies. They introduce the concept and help us think about what could be built. Whether or not we ultimately end up building the final app with the tool whose simplicity introduced it to us is irrelevant.

    The potential of WebRTC

    I’m convinced WebRTC has the potential to have a huge impact on how we communicate. It already has for our team at &yet. Sure, we already used stuff like Skype, Facetime, and Google Hangouts. But the simplicity and convenience of just opening a URL in a browser and instantly being in a conversation is powerful.

    Once this technology is broadly available and on mobile devices, it’s nothing short of a game changer for communications.

    Challenges

    There are definitely quite a few hurdles that get in the way of just playing with WebRTC: complexity and browser differences in instantiating peer connections, generating and processing signaling messages, and attaching media streams to video elements.

    Even at the point you have those things, you still need a way to let two users find each other and have a mechanism for each user to send the proper signaling messages directly to the other user or users that they want to connect to.

    SimpleWebRTC.js is our answer to the clientside complexities. It abstracts away API differences between Firefox and Chrome.

    Using SimpleWebRTC

    At its simplest, you just need to include the SimpleWebRTC.js script, provide a container for your local video, a container for the remote video(s) like this:

    <!DOCTYPE html>
    <html>
        <head>
            <script src="http://simplewebrtc.com/latest.js"></script>
        </head>
        <body>
            <div id="localVideo"></div>
            <div id="remoteVideos"></div>
        </body>
    </html>

    Then in you just init a webrtc object and tell it which containers to use:

    var webrtc = new WebRTC({
        // the id of (or actual element) to hold "our" video
        localVideoEl: 'localVideo',
     
        // the id of or actual element that will hold remote videos
        remoteVideosEl: 'remoteVideos',
     
         // immediately ask for camera access
        autoRequestMedia: true
    });

    At this point, if you run the code above, you’ll see your video turn on and render in the container you gave it.

    The next step is to actually specify who you want to connect to.

    For simplicity and maximum “tinkerability” we do this by asking that both users who want to connect to each other join the same “room”, which basically means: call “join” with the same string.

    So, for demonstration purposes we’ll just tell our webrtc to join a certain room once it’s ready (meaning it’s connected to the signaling server). We do this like so:

    // we have to wait until it's ready
    webrtc.on('readyToCall', function () {
        // you can name it anything
        webrtc.joinRoom('your awesome room name');
    });

    Once a user has done this, he/she is ready and waiting for someone to join.

    If you want to test this locally, you can either open it in Firefox and Chrome or in two tabs within Chrome. (Firefox doesn’t yet let two tabs both access local media).

    At this point, you should automatically be connected and be having a lively (probably very echo-y!) conversation with yourself.

    If you happen to be me, it’d look like this:

    henrik in conversat.io

    The signaling server

    The example above will connect to a sandbox signaling server we keep running to make it easy to mess around with this stuff.

    We aim to keep it available for people to use to play with SimpleWebRTC, but it’s definitely not meant for production use and we may kill it or restart it at any time.

    If you want to actually build an app that depends on it, you can either run one yourself, or if you’d rather not mess with it, we can host, and keep up to date, and help scale one for you. The code for that server is on github.

    You can just pass a URL to a different signaling server as part of your config by passing a “url” option when initiating your webrtc object.

    So, what’s it actually doing under the hood?

    It’s not too bad, really. You can read the full source of the client library here: https://github.com/HenrikJoreteg/SimpleWebRTC/blob/master/simplewebrtc.js and the signaling server here: https://github.com/andyet/signalmaster/blob/master/server.js

    The process of starting a video call in conversat.io looks something like this:

    1. Establish connection to the signaling server. It does this with socket.io and connects to our sandbox signaling server at: http://signaling.simplewebrtc.com:8888

    2. Request access to local video camera by calling browser prefixed getUserMedia.

    3. Create or get local video element and attach the stream that we get from getUserMedia to the video element.

      firefox:

      element.mozSrcObject = stream; element.play();

      webkit:

      element.autoplay = true;
      element.src = webkitURL.createObjectURL(stream);

    4. Call joinRoom which sends a socket.io message to the signaling server telling it the name of the room name it wants to connect to. The signaling server will either create the room if it doesn’t exist or join it if it does. All I mean by “room” is that the particular socket.io session ID is grouped by that room name so we can broadcast messages about people joining/leaving that room to only the clients connected to that room.

    5. Now we play an awesome rocket lander game that @fritzy wrote while we wait for someone to join us:

    6. When someone else joins the same “room” we broadcast that to the other connected users and we create a Conversation object that we’ve defined which wraps the browser’s peerConnection. The peer connection represents, as you’d probably guess, the connection between you and another person.

    7. The signaling server broadcasts the new socket.io session ID to each user in the room and each user’s client creates a Conversation object for every other user in the room.

    8. At this point we have a mechanism of knowing who to connect to and how to send direct messages to each of their sessions.

    9. Now we use the peerConnection to create an “offer” and store our local offer and set it in our peer connection as the local description. This contains information about how another client can reach and talk to our browser.

      peerConnection.createOffer();

      We then send this over our socket.io connection to the other people in the room.

    10. When a client receives and offer we add it to our peer connection:

      var remote = new RTCSessionDescription(message.payload);
      peerConnection.setRemoteDescriptionremoteDescription);

      and generate an answer by calling peerConnection.createAnswer() and send that back to the person we got the offer from.

    11. When the answer is received we set it as the remote description. Then we create and send ICE Candidates much in the same way. This will negotiate our connection and connect us.

    12. If that process is successful we’ll get an onaddstream event from our peer connection and we can then create a video element and attach that stream to it. At this point the video call should be in progress.

    If you wish to dig into it further, send pull requests and file issues on the SimpleWebRTC project on github.

    The road ahead

    This is just a start. Help us make this stuff better!

    There’s a lot more we’d like to see with this:

    1. Making the signaling piece more pluggable (so you can use whatever you want).
    2. Adding support for pausing and resuming video/audio.
    3. It’d be great to be able to figure out who’s talking and emit an event to other connected users when that changes.
    4. Better control over handling/rejecting incoming requests.
    5. Setting max connections, perhaps determined based on HTML5 connection APIs?

    Hit me up on twitter (@henrikjoreteg) if you do something cool with this stuff or run into issues or just want to talk about it. I’d love to hear from you.

    Keep building awesome stuff, you amazing web people! Go go gadget Internet!

  5. Firefox OS Simulator – previewing version 3.0

    Three months ago we were proud to release the 1.0 version of the Firefox OS Simulator. We’ve made a lot of progress since, and version 2.0 came out about a month ago (latest official version). Now, moving forward, we’d like to present and introduce you to a preview of the upcoming 3.0 version!

    image

    We discussed whether we should talk about this new version yet, since it’s a bit rough around the edges, but we decided to give it a go for two reasons:

    • We’re Mozilla. We do things in the open, and we share our progress. Because we want you to know what’s going on and to be able to come along with us in the process
    • It gives you an unique opportunity to test it out, give feedback, contribute and much more before it’s released

    New features in the preview

    We’ve listened to the feedback and have tried to target the most common features being requested and well-needed. New features include:

    • Push to Device
    • Rotation simulation
    • Basic geolocation API simulation
    • Manifest validation
    • Stability fixes for installation and updates to apps
    • Newer versions of the Firefox rendering engine and Gaia (the UI for Firefox OS)

    Push to Device

    This means that if you have an existing device supporting Firefox OS, connected via USB, you will be able to push apps installed in the Firefox OS Simulator directly to that device.

    image

    Please note:

    • Remote debugging has to be enabled on the device, via
      Settings > Device information > More Information > Developer > Remote debugging
    • On Linux (at least Ubuntu), you must create the file /etc/udev/rules.d/51-android.rules as root and then add a manufacturer-specific entry for the device as described by Android’s Setting up a Device for Development. Example for one of our test devices:entry:
      SUBSYSTEM=="usb", ATTR{idVendor}==" 19d2", MODE="0666", GROUP="plugdev"
    • Not complete Windows support yet. Planned to make it into the final release.
    • Make sure you have the latest version of Firefox OS on your device (especially due to recent fixes like bug 842725)

    image

    Rotation simulation

    There’s now a feature to rotate the simulator, get events and more, to adapt your contents to both portrait and landscape. Supports the mozorientationchange event.

    image

    Basic geolocation API simulation

    The simulator now also supports geolocation, so you can test it in your app, and read out longitude and latitude values.

    Coming soon: an enhancement that lets you specify the geolocation to provide!

    Manifest validation

    When you add an app to the Firefox OS Simulator, it also does a quick validation of your manifest file for errors and warnings, including problems that prevent installing the app in the Simulator, usage of APIs that the Simulator doesn’t yet simulate (not all APIs in there yet), and missing properties that are required by the Marketplace or devices.

    image

    Downloading the preview

    We have all the versions of the Firefox OS Simulator on our FTP server, under its working name r2d2b2g. Here are the direct links to the installation files (installs as an extension in Firefox)

    Once installed, it will be available in Firefox in the Tools > Web Developer menu:

    image

    Give us feedback!

    Please let us know in the comments here or by filing a bug. Hopefully you will like the improvements and they will benefit you with developing apps!

    Getting started with Firefox OS & building Open Web Apps

    To get started, we have had a number of articles here on Mozilla Hacks previously:

    Additionally we have some other resources:

  6. Cross-browser camera capture with getUserMedia/WebRTC

    Overview

    With Firefox adding support for getUserMedia, three of the major desktop browsers now have the ability to get data from cameras without the use of plugins. As it’s still early days, however, the implementations differ slightly between browsers. Below is an example of how to work around these differences and a script to do the heavy lifting for you, but first, an overview of how the three browsers stack up.

    Comparison of getUserMedia behaviour in browsers as of February 2013
    Firefox 18 Opera 12 Chrome 24
    Requires vendor prefix Yes (moz) No Yes (webkit)
    Triggered with autoplay attribute No Yes Yes
    Requires enabling by user Yes 1 No No
    Firing of playing event Repeatedly Once Once
    Supports file:// protocol Yes Yes No
    Tab playing notification None Icon Animated icon
    Permission request Each page load First page load only Each page load

    getUserMedia has to be enabled in Firefox by setting the media.peerconnection.enabled option to true in about:config.

    There are a few more differences once we start coding so let’s walk through it. Our recipe for getUserMedia success will be broken down into the following easy steps:

    1. A helping of HTML5
    2. A dollop of feature detection
    3. A spoonful of streaming
    4. Ready for serving
    5. A final tip

    Deep breath – here we go…

    A helping of HTML5

    Our main task in this short tutorial is just to get a moving image displaying in a page. In that respect, it’s no different to regular video so the first step is a simple <video> element in our HTML:

    <video></video>

    That’s it. No controls, no src, no nothing.

    Over to the JavaScript, and obviously we need to get a reference to the <video> element, which we can do like so (or alternatively with an id):

    var video = document.querySelector('video');

    A dollop of feature detection

    Now it gets interesting as we check for getUserMedia support. We’re definitely not going to use unreliable user agent sniffing for this — no, we’ll do it the easy way by checking for the navigator.getUserMedia object. This is prefixed in Firefox and Chrome so first it’s handy to assign it to a common object for all browsers. While we’re at it, let’s do it for the window.URL object as well which we’ll use later on.

    navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia;
    window.URL = window.URL || window.webkitURL || window.mozURL || window.msURL;

    And next, the actual existence checking:

    if (navigator.getUserMedia) {
        // Call the getUserMedia method here
    } else {
        console.log('Native device media streaming (getUserMedia) not supported in this browser.');
        // Display a friendly "sorry" message to the user.
    }

    If getUserMedia is supported, we need to pass it three arguments — an options object, a success callback function and an error callback function. Note that the error callback is required in Firefox but optional in Opera and Chrome. The options argument is a JSON-style object that specifies whether audio, video or both are to be used. The following example code is for video only:

    navigator.getUserMedia({video: true}, successCallback, errorCallback);

    Dialogs requesting camera access

    Dialog in Firefox
    Dialog in Google Chrome
    Dialog in Opera

    A spoonful of streaming

    So far so good, so let’s define what happens next. The success callback function receives an argument containing the video stream from the camera and we want to send that stream to our <video> element. We do this by setting its src attribute but there are a couple of things to bear in mind:

    • Firefox uses the mozSrcObject attribute whereas Opera and Chrome use src.
    • Chrome uses the createObjectURL method whereas Firefox and Opera send the stream directly.

    With Firefox, video.mozSrcObject is initially null rather than undefined so we can rely on this to detect for Firefox’s support (hat tip to Florent). Once the stream knows where to go we can tell the video stream to play.

    function successCallback(stream) {
        if (video.mozSrcObject !== undefined) {
            video.mozSrcObject = stream;
        } else {
            video.src = (window.URL && window.URL.createObjectURL(stream)) || stream;
        };
        video.play();
    }

    Ready for serving

    And there you have it. Add a simple error callback function and we have a working cross-browser script which looks something like this:

    window.addEventListener('DOMContentLoaded', function() {
        'use strict';
        var video = document.querySelector('video');
     
        function successCallback(stream) {
            // Set the source of the video element with the stream from the camera
            if (video.mozSrcObject !== undefined) {
                video.mozSrcObject = stream;
            } else {
                video.src = (window.URL && window.URL.createObjectURL(stream)) || stream;
            }
            video.play();
        }
     
        function errorCallback(error) {
            console.error('An error occurred: [CODE ' + error.code + ']');
            // Display a friendly "sorry" message to the user
        }
     
        navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia;
        window.URL = window.URL || window.webkitURL || window.mozURL || window.msURL;
     
        // Call the getUserMedia method with our callback functions
        if (navigator.getUserMedia) {
            navigator.getUserMedia({video: true}, successCallback, errorCallback);
        } else {
            console.log('Native web camera streaming (getUserMedia) not supported in this browser.');
            // Display a friendly "sorry" message to the user
        }
    }, false);

    Available on GitHub

    To get started with accessing getUserMedia in a cross web browser fashion, we have also put a working example on GitHub: GumWrapper.

    A final tip

    If you want to do anything fancy with the camera’s stream like capture a still image or add fancy effects, you’ll probably want to send its data to a canvas context. You can use drawImage() for this, in which case you’ll need the dimensions of the video. These are available through the video.videoWidth and video.videoHeight properties but beware — they’re only set when the browser has information about the stream. This means you have to listen for certain events before you can get these properties. There are a few relevant events, always fired in the following order:

    1. play
    2. loadedmetadata
    3. loadeddata
    4. playing

    The play event is fired after the video.play() method is called but there may be a slight delay before the video actually starts playing. That’s where the playing event comes in but note that it’s fired repeatedly in Firefox while the stream or video is playing. Before that are a couple of data events, the first of which is just for metadata, however in Firefox this doesn’t include video dimensions. Consequently, the most reliable event to listen for is the loadeddata event — you can then be sure of knowing the width and height of the video stream. You could code it up like this:

    video.addEventListener('loadeddata', function() {
    console.log('Video dimensions: ' + video.videoWidth + ' x ' + video.videoHeight);
    }, false);

    Incidentally, you could also use the stream’s dimensions as a further error check, for example checking whether the width and height are above 0. This would avoid problems such as the user’s webcam being broken or simply not plugged in.

    And there you have it. I’m sure the differences between browsers will disappear as the technology matures but for the time being, the above code should help you on your way.

  7. Using WebAPIs to make the web layer more capable

    Part of making both Firefox OS and the web as a platform a stronger layer and alternative for developers, we are working on a number of WebAPIs. I’d like to introduce you them here!

    Many things covered in this blog post are also available in a talk I’ve given on this topic.

    Standardization

    When you talk about new technology, ideas and solutions, people have a tendency to be wary. They might see it as the Not Invented Here syndrome, but that is not our intent (as we’ve written here before, and as mentioned by our CTO, Brendan Eich).

    Our intention is that everything we do is, or will be, standardized. Developed in the open.

    This blog post focuses on various WebAPIs, their security level and how to use them. In a later post, we will follow up specifically on standardization and progress in that area.

    Types of APIs

    There are basically three types of WebAPIs:

    Regular APIs
    APIs available from any app, hosted or packaged.
    Privileged APIs
    Only available in privileged apps. Privileged apps are apps that are:

    Certified APIs
    Certified APIs are only available to system itself (Mozilla in the case of Firefox OS), meaning they are so sensitive and need to be strictly controlled.

    Regular APIs

    The complete list of regular APIs is:

    • Vibration API
    • Screen Orientation
    • Geolocation API
    • Mouse Lock API
    • Open WebApps
    • Network Information API
    • Battery Status API
    • Alarm API
    • Web Activities
    • Push Notifications API
    • WebFM API
    • WebPayment
    • IndexedDB
    • Ambient light sensor
    • Proximity sensor
    • Notification
    • FMRadio

    Here’s how to use a number of them:

    Battery Status API

    This API is about detecting the current battery level of the computer/device, how long battery life there’s left and whether it’s being charged or not. Works in all three of desktop, Android and Firefox OS.

    var battery = navigator.battery;if (battery) {var batteryLevel = Math.round(battery.level * 100) + "%",

            charging = (battery.charging)? "" : "not ",

            chargingTime = parseInt(battery.chargingTime / 60, 10),

            dischargingTime = parseInt(battery.dischargingTime / 60, 10);
     
            // Set events

            battery.addEventListener("levelchange", showStatus);
            battery.addEventListener("chargingchange", showStatus);
            battery.addEventListener("chargingtimechange", showStatus);
            battery.addEventListener("dischargingtimechange", showStatus);}

    Vibration API

    Making the device vibrate, either once or with a certain pattern.

    // Vibrate for one second

    navigator.vibrate(1000);
     
    // Vibration pattern [vibrationTime, pause,…]

    navigator.vibrate([200, 100, 200, 100]);
     
    // Vibrate for 5 seconds

    navigator.vibrate(5000);
     
    // Turn off vibration

    navigator.vibrate(0);

    Screen Orientation

    Gives you as a developer the chance to lock the orientation, or to specify what you want the primary orientation experience to be.

    /*
        Possible values:
            "landscape",
            "portrait"
            "landscape-primary"
            "landscape-secondary"
            "portrait-primary"
            "portrait-secondary"

    */
    var portraitLock = screen.mozLockOrientation("portrait");
    if (portraitLock) {
        console.log("Orientation locked to portrait");
    }

    Geolocation API

    Finding out where the user is at the moment – approved by the user to share this.

    navigator.geolocation.getCurrentPosition(function (position) {
        /*
            Getting latitude and longitude:
                position.coords.latitude
                position.coords.longitude
    };

    Mouse Lock API

    Locking down the mouse movements and controlling the experience yourself. Especially good when you don’t want the mouse pointer/interaction to end when it hits the end of the web browser window, but rather to continue to scroll – like doing complete 360 degrees spin of an environment or similar.

    var docElm = document.documentElement;// Requesting Pointer Lock

    docElm.requestPointerLock = elem.requestPointerLock ||
                                elem.mozRequestPointerLock ||
                                elem.webkitRequestPointerLock;
    
docElm.requestPointerLock();
     
    document.addEventListener("mousemove", function(e) {
        var movementX = e.movementX       ||
                        e.mozMovementX    ||
                        e.webkitMovementX ||
                        0,
            movementY = e.movementY       ||
                        e.mozMovementY    ||
                        e.webkitMovementY ||
                        0;
     
        // Get the mouse movement delta values
        console.log("movementX=" + movementX, "movementY=" + movementY);}
    );

    Open WebApps

    In general, an APIs needed to install and handle Open Web Apps.

    var installApp = navigator.mozApps.install(manifestURL);
     
    // Successful install
    installApp.onsuccess = function(data) {
        console.log("Success, app installed!");
    };
     
    // Install failed
    installApp.onerror = function() {
        console.log("Install failedn:" + installApp.error.name);
    };

    Network Information API

    Used to get information about network connectivity.

    var connection = window.navigator.mozConnection,
        online = connection.bandwidth,
        metered = connection.metered;
     
    // online can return:
        0 when offline
        Infinity when the bandwidth isn't known
        Estimation of MB/s
     
    // metered implies if the connection is being metered,
       meaning limited in some way from the ISP

    Alarm API

    Making it possible to set alarms.

    request = navigator.mozAlarms.add(
        new Date("May 15, 2012 16:20:00"),
        "honorTimezone",
        {
            mydata: "my event"
        }
    );

    Web Activities

    Through Web Activities you can specify intents on what kind of action you want to do, or that your app can handle.

    var pick = new MozActivity({
         name: "pick",
         data: {
             type: ["image/png", "image/jpg", "image/jpeg"]}
     });
     
    pick.onsuccess = function () {var img = document.createElement("img");
        img.src = window.URL.createObjectURL(this.result.blob);
        document.body.appendChild(img);
    };

    I’ve covered Web Activities in more detail in another blog post here.

    Push Notifications API

    A way for web sites to send messages to suers when aren’t on their web site. API is in a draft, not implemented yet. More can be read in the PushAPI documentation.

    function getPushURL() {
        var push = navigator.push ||
                    navigator.mozPush ||
                    navigator.webkitPush;
     
        // Ask the user to allow notifications
        var request = push.requestURL(watoken, PbK);
     
        request.onsuccess = function () {
            var url = request.result.url;
            console.log('Push URL: ' + url);
        };
    }

    WebFM API

    Implementing a FM Radio in the web layer.

    var fmRadio = navigator.fm || navigator.mozFMRadio || navigator.mozFM;
    fmRadio.frequency = 106.7;

    WebPayment

    Used to enable in-app payments, through JSON Web Token (JWT).

    var request = navigator.mozPay(JWTs);
    request.onsuccess = function () {
        // Money!
    };

    IndexedDB

    Structured client-side storage with high performance search support. Covered in detail in Using IndexedDB and Storing images and files in IndexedDB.

    // Create/open database
    var request = indexedDB.open("elephantFiles", 1),
         createObjectStore = function (dataBase) {
            console.log("Creating objectStore")
            dataBase.createObjectStore("elephants");
    };
     
    request.onsuccess = function (event) {
        console.log("Success creating/accessing IndexedDB database");
        db = request.result;
     
        db.onerror = function (event) {
            console.log("Error creating/accessing IndexedDB database");
        };
    }
     
    // Needed for creating database/upgrading to a new version
    request.onupgradeneeded = function (event) {
        createObjectStore(event.target.result);
    };

    Ambient light sensor

    Detecting the level of ambient light, to be able to serve different versions depending on the environment the user currently is in.

    window.addEventListener("devicelight", function (event) {/* The level of the ambient light in lux

           A lux value for "dim" typically begin below 50,
           and a value for "bright" begin above 10000
        */
        console.log(event.value);});
     
    window.addEventListener("lightlevel", function (event) {
        // Possible values: "normal", "bright", "dim"
        console.log(event.value);});

    Proximity sensor

    Getting an indication of how close the device is to another object.

    window.addEventListener("deviceproximity", function (event) {// Current device proximity, in centimeters

        console.log(event.value);

    

        // The maximum sensing distance the sensor is 

        // able to report, in centimeters

        console.log(event.max);

    

        // The minimum sensing distance the sensor is 

        // able to report, in centimeters

        console.log(event.min);});

    Notification

    Being able to show notifications to the user.

    var notification = navigator.mozNotification;
    
notification.createNotification(
        "See this",
        "This is a notification"
    );
    // You could send an optional third parameter,
    // which would be the URL to an icon

    Privileged APIs

    These APIs can only be used in a privileged app.

    The complete list of privileged APIs is:

    • Device Storage API
    • Browser API
    • TCP Socket API
    • Contacts API
    • systemXHR

    Device Storage API

    Accessing files stored on the device.

    var storage = navigator.getDeviceStorage("videos"),

        cursor = storage.enumerate();

    

    cursor.onerror = function() {
        console.error("Error in DeviceStorage.enumerate()", cursor.error.name);};
     
    

cursor.onsuccess = function() {if (!cursor.result)return;}
     
    
    var file = cursor.result;
     
        // If this isn't a video, skip it

        if (file.type.substring(0, 6) !== "video/") {

            cursor.continue();return;
        }
     
        // If it isn't playable, skip it

        var testplayer = document.createElement("video");if (!testplayer.canPlayType(file.type)) {

            cursor.continue();return;
        }
     
        // Show file
        console.log(file.name);
    };

    Browser API

    Implementing your own web browser, completely with web technologies. Described more in BrowserAPI.

    iframe.addEventListener("mozbrowserlocationchange", function(e) {
        console.log(e.detail);
    });
     
    iframe.addEventListener("mozbrowsersecuritychange", function(e) {
        // "secure", "insecure", or "broken".  "broken" indicates mixed content.
        console.log(e.detail.state);
    });
     
    iframe.addEventListener("mozbrowsercontextmenu", function(e) {
        // Show context menu
    });

    TCP Socket API

    A low-level TCP socket API, that will also include SSL support.

    var TCPSocket = navigator.mozTCPSocket.open(
        host,
        port,
        {
            useSSL: crypto,
            binaryType: "arraybuffer"
        }
    );

    Contacts API

    Accessing contacts on the device – adding, reading or modifying.

    var contact = new mozContact();
    
contact.init({name: "Tom"});
    


    var request = navigator.mozContacts.save(contact);
    
request.onsuccess = function() {
        console.log("Success");};


     
    request.onerror = function() {
        console.log("Error")};

    systemXHR

    Making it possible to allow cross-domain XMLHTTPRequests. Specify in the manifest file in the permissions part that you will want to access it – "systemXHR":{} – and then just do the request.

    var xhr = new XMLHttpRequest();
    xhr.open("GET", anyURL, true);

    Certified APIs

    The certified APIs are only available to the system itself/pre-installed apps. In the fcase of Firefox OS, this means only Mozilla.

    The complete list of certified APIs is:

    • WebTelephony
    • WebSMS
    • Idle API
    • Settings API
    • Power Management API
    • Mobile Connection API
    • WiFi Information API
    • WebBluetooth
    • Permissions API
    • Network Stats API
    • Camera API
    • Time/Clock API
    • Attention screen
    • Voicemail

    I’ll list a few of them here, in case you are interested in testing or contributing to Gaia, the UI of Firefox OS.

    WebTelephony

    For placing, receiving and dealing with calls.

    // Telephony object
    var tel = navigator.mozTelephony;
     
    // Check if the phone is muted (read/write property)
    console.log(tel.muted);
     
    // Check if the speaker is enabled (read/write property)
    console.log(tel.speakerEnabled);
     
    // Place a call
    var call = tel.dial("123456789");
     
    // Events for that call
    call.onstatechange = function (event) {
        /*
            Possible values for state:
            "dialing", "ringing", "busy", "connecting", "connected",
            "disconnecting", "disconnected", "incoming"
        */
        console.log(event.state);
    };
     
    // Above options as direct events
    call.onconnected = function () {
        // Call was connected
    };
     
    call.ondisconnected = function () {
        // Call was disconnected
    };
     
    // Receiving a call
    tel.onincoming = function (event) {
        var incomingCall = event.call;
     
        // Get the number of the incoming call
        console.log(incomingCall.number);
     
        // Answer the call
        incomingCall.answer();
    };
     
    // Disconnect a call
    call.hangUp();

    WebSMS

    For sending and receiving SMS messages.

    // SMS object
    var sms = navigator.mozSMS;
     
    // Send a message
    sms.send("123456789", "Hello world!");
     
    // Receive a message
    sms.onreceived = function (event) {
        // Read message
        console.log(event.message);
    };

    Permissions

    To be able to access certain APIs, in an Open Web App context, you need to specify permissions for the APIs you want to access in the manifest file.

    "permissions": {
        "contacts": {
            "description": "Required for autocompletion in the share screen",
            "access": "readcreate"
        },
        "alarms": {
            "description": "Required to schedule notifications"
        }
    
}

    For all APIs that are considered privileged or certified, this applies.

    For regular APIs, only geolocation and notification are affected (and for geolocation, in a regular web browser context, the user will be presented with a dialog to approve/decline).

    Additionally, the Camera API is restricted as a certified API at this time, but the long-term goal is to make it available to all apps. At the moment, if you want to access the camera, do it through Web Activities.

    Please take a look at the list of all APIs requiring permissions.

    Platform support

    As I’m sure you understand, a number of the WebAPIs are going through a process filled with progress, iterations and improvements. Some of the APIs above will work as described and intended, while others might not at this time.

    To be able to follow the current implementation state, please look at the list of APIs planned for the initial release of Firefox OS.

    The three columns imply:

    • D = Desktop support in Firefox
    • A = Android support in Firefox
    • B = Firefox OS support
    • Green = Implemented and enabled
    • Orange = Implemented, but needs to explicitly be turned on
    • Red = Not implemented
    • Blue = Only available as a Certified API
    • Grey = Not planned for this platform

    APIs planned for the future

    While not being worked on at the moment, I’d like to list APIs that are planned for the future. To show you our intent, and things we want to implement and support, and also to show in which direction we are moving.

    • Resource lock API
    • UDP Datagram Socket API
    • Peer to Peer API
    • WebNFC
    • WebUSB
    • HTTP-cache API
    • Calendar API
    • Spellcheck API
    • LogAPI
    • Keyboard/IME API
    • WebRTC
    • FileHandle API
    • Sync API

    Testing out these new APIs

    Feel free to just copy and paste code from this blog post to test the API(s) that you are interested in. I’ve also implemented support for some of them in the Firefox OS Boilerplate App, available in the webapp.js file (and I plan to add more in the future).

  8. Getting started with Open Web Apps – why and how

    We’ve been talking a lot about Open Web Apps, Firefox OS and more here lately, and I wanted to cover both how to get started, and, maybe more importantly, why.

    Why a web app?

    If we look at the climate for mobile development, it has usually come down to a choice where developers had to pick their platform and skill. Maybe do iOS and Objective-C, or Android and Java.

    The big challenge here, of course, is that if you want to deliver your content on several platforms, you’ve had a few choices:

    • Pick one platform, and ignore the others
    • Learn a number of programming languages
    • Have separate development teams for each platform

    For major organizations, having multiple teams has been something they could do, while many other have struggled with that. And naturally, many mobile apps are in addition to a company/service web site, just adding more and more on top of the things that need to be maintained, supported and developed.

    Therefore, for reasons like saving costs, simplicity of one development language and more, many developers have jumped on the options like PhoneGap, Titanium and more, thus developing with HTML5 and JavaScript, and then packaging it for various mobile operating systems.

    This has been a good and interesting approach, but probably far from optimal in most cases. We at Mozilla want you to be able to use your existing skills as a web developer, but instead of having you jump through hoops to make it work, we want the platforms to evolve and give you more possibilities and power.

    This is accomplished by giving you as a developer access to a large amount of WebAPIs, Web Activities and more, to make the web layer as powerful a platform as it deserves to be.

    The idea with Open Web Apps is not to make you choose a new platform or, worse, excluding others – on the contrary, it’s about giving you means to reuse your existing code and, if desired, make small additions to make it installable as an app.

    Should I build an app?

    While many other platforms have a strong interest in getting you tied into their platform, delivering to their app store etc, I would rather say that the first question you need to ask yourself is:

    Do I really need to make an app out of this?

    In some cases, yes, definitely! But in other cases, you need to be professional and come to the conclusion that it is probably not likely to add any extra value – a few use cases where you don’t need to do an app have been outlined in articles like No, I’m not going to download your bullshit app and Packaged HTML5 Apps: Are we emulating failure?.

    I don’t want to trick you into making an app just for the sake of it, or that you’ll do it just because others do. I would rather see you make a fair assessment, and if your app idea has something additional to offer to the end user and overall user experience, then you should consider doing an app.

    So, what could those cases be? A few of them might be:

    • You want to offer a richer experience than you could offer from a web page, e.g. accessing some WebAPIs specific to the platform/device
    • You need to store a lot of information in localStorage or IndexedDB
    • The user wants to have a real installation
    • The user wants a nice icon on their home screen/desktop for easier accessibility

    Types of Open Web Apps

    There are basically two types of Open Web Apps you can install:

    • Hosted apps
    • Packaged apps

    Hosted apps

    Hosted apps are apps running from a URL, but in an app context. This means that you need to be online to run it, and all resources (e.g. files, images etc) will reside on your server and where you will host it.

    One option to avoid the need for connectivity is make sure your app works offline. This is done by adding an appcache file with listings of assets to be made available offline, and then refer to it from your main page:

    HTML file

    <html manifest="manifest.appcache">

    Appcache file

    CACHE MANIFEST
    # Version 1.0
     
    index.html
    css/base.css
    js/base.js
    js/webapp.js
    js/offline.js
     
    NETWORK:
    *
     
    FALLBACK:
    / fallback.html

    For learning more about offline support and caveats, I strongly recommend looking into these resources:

    Pros

    • You completely control the update process
    • Just run/reuse currently existing code

    Cons

    • Requires connectivity (if offline support isn’t implemented)
    • Doesn’t have as much access to APIs as a packaged app

    Packaged apps

    Packaged apps are where you put all your assets in a ZIP file, and then offer that as a complete package to be installed. This makes these files available at all times, and it also gives you elevated privileges – i.e. you can access more APIs – since all the code can be security cleared before install.

    Pros

    • Available offline by default
    • More API access

    Cons

    • More difficult to maintain
    • Update process for getting new versions out

    At the end of the day, you need to evaluate your needs, workflow, APIs you need to work with and more to make a good decision whether you want to do a hosted or a packaged app.

    Getting started with Open Web Apps

    After all that talk, what do you actually need to build an Open Web App? Not much, as it turns out. We’ve documented it on MDN in Getting started with making apps but I’d like to give you a quick run-down here as well.

    image

    Basically, all you need to do is take an existing web site/service you have and add a manifest file. Voilà, that’s it! Yes, really. And to get it installed, of course.

    The manifest file

    A manifest file describes your app, with things like name, icons, developer and such, but also localization support, path to launch, permission request for certain APIs and more. All manifest fields are available in App manifest on MDN.

    A simple manifest could look like this:

    {
        "version": "1",
        "name": "Firefox OS Boilerplate App",
        "launch_path": "/Firefox-OS-Boilerplate-App/index.html",
        "description": "Boilerplate Firefox OS app with example use cases to get started",
        "icons": {
            "16": "/Firefox-OS-Boilerplate-App/images/logo16.png",
            "32": "/Firefox-OS-Boilerplate-App/images/logo32.png",
            "48": "/Firefox-OS-Boilerplate-App/images/logo48.png",
            "64": "/Firefox-OS-Boilerplate-App/images/logo64.png",
            "128": "/Firefox-OS-Boilerplate-App/images/logo128.png"
        },
        "developer": {
            "name": "Robert Nyman",
            "url": "http://robertnyman.com"
        },
        "installs_allowed_from": ["*"],
        "default_locale": "en"
    }

    Save this file with a .webapp extension, for instance, manifest.webapp. One very important thing to note here is that this file needs to be served with the Content-type: application/x-web-app-manifest+json.

    This is something you need to set up on your server, e.g. through an .htaccess file in Apache:

    AddType application/x-web-app-manifest+json .webapp

    Once you have your manifest, make sure to validate your app to see that it has the correct format.

    image

    Installing the app

    Now that your manifest is in order and served with the right Content-type, let’s offer a way to install it. In your web page, you can add an install button that calls this code:

    var installApp = navigator.mozApps.install(manifestURL);
     
    // Successful install
    installApp.onsuccess = function(data) {
        console.log("Success, app installed!");
    };
     
    // Install failed
    installApp.onerror = function() {
        console.log("Install failednn:" + installApp.error.name);
    };

    Make sure that the URL to the manifest is absolute – a simple way to do this is to extract the URL from the current page with the install button, and to have the manifest file in the same location:

    var manifestURL = location.href.substring(0, location.href.lastIndexOf("/")) + "/manifest.webapp";

    Optionally, you can also provide a second parameter to the install method, receipts, which is a JSON object. More on that in the documentation for the install method.

    Installing a packaged app

    The above solution, with a manifest file and an install call, works well with hosted apps. With Packaged apps, you need to go through some extra steps:

    ZIP all app content

    Make sure to ZIP all the files (not the containing folder), including the regular manifest file. The manifest file has to be named manifest.webapp

    Create a mini manifest

    Create another manifest file, for instance named package.webapp, and make sure the package_path is absolute to where the ZIP file is located.

    Also, developer name and info has to match between mini manifest and the regular one in the ZIP file.

    {
        "name": "Firefox OS Boilerplate App",
        "package_path" : "http://localhost/Firefox-OS-Boilerplate-App/Firefox-OS-Boilerplate-App.zip",
        "version": "1",
        "developer": {
            "name": "Robert Nyman",
            "url": "http://robertnyman.com"
        }
    }

    Installing a package

    Instead of using the regular install method, you now call the installPackage method, which points to the mini manifest, which in turn points to the ZIP file/package:

    var manifestURL = location.href.substring(0, location.href.lastIndexOf("/")) + "/package.webapp";
    var installApp = navigator.mozApps.installPackage(manifestURL);

    Turning on Developer Mode

    For this to work in the Firefox OS Simulator, you need to turn on the Developer Mode:

    Settings > Device Information > More Information >
    Developer > Developer mode
    

    Note: this is a work in progress, and the availability of this option might vary, and not be available in your version of the Simulator or an actual Firefox OS device.

    All the releases and pre-releases of the Firefox OS Simulator are available on Mozilla’s FTP.

    Specifying permissions

    If you plan on using some APIs that only Packaged apps have access to, you need to add a couple of things to your regular (manifest.webapp) file:

    • Add type property (e.g. “type” : “privileged”)
    • Specify permission access
    "permissions": {
        "contacts": {
            "description": "Required for autocompletion in the share screen",
            "access": "readcreate"
        },
        "alarms": {
            "description": "Required to schedule notifications"
        }
    }

    There is also an interesting option, in the form of packaged-app-server, which offers zipping the files as a package when they get requested at run time.

    $ cd ~/myapp
    $ python ~/serve_packaged_apps.py

    An example app

    As an example app – if you want to check out something easy to dissect, tweak and get started with – feel free to test the Firefox OS Boilerplate App. It supports:

    • An install button, offering you to install it as a hosted app
    • Web Activities – lots of examples and use cases
    • WebAPIs in action
    • Offline support (disabled by default)
    • Packaged apps – install your app as a ZIP file

    image

    Which platforms are supported?

    Let’s look at where we are right now with Open Web Apps. They are supported in:

    Firefox OS

    You can install an Open Web App in Firefox OS (the Simulator or on a device) and most WebAPIs and Web Activities will work.

    image

    Firefox on Android

    In Firefox on Android you can install an app, and have it installed on your home screen with the correct icon. However, it doesn’t support the WebAPIs and Web Activities.

    image

    Nightly/Aurora versions on Desktop

    You can install and run a stand-alone app in Firefox Nightly/Firefox Aurora, but it doesn’t have access to many WebAPIs and no Web Activities.

    image

    The initial strong focus at the moment is support on mobile, but the hope and goal is that Open Web Apps will work on all platforms and devices, by adding support to needed APIs and more.

    Marketplace

    When it comes to Open Web Apps, you can use or install them from anywhere. Completely up to you.However, if you are interested in being listed, hosted and much more, I recommend taking a look at the Firefox Marketplace.

    image

    You can also visit the Developer Hub for a lot more information about app development.

    image

    Conclusion

    Open Web Apps aren’t here to change your developing ways – they are here to give you the option to install your existing web solutions as apps, access device-specific APIs and more.

    Don’t reinvent the wheel – just make it a bit more powerful!

  9. Hello Chrome, it's Firefox calling!

    Mozilla is excited to announce that we’ve achieved a major milestone in WebRTC development: WebRTC RTCPeerConnection interoperability between Firefox and Chrome. This effort was made possible because of the close collaboration between the open Web community and engineers from both Mozilla and Google.

    RTCPeerConnection (also known simply as PeerConnection or PC) interoperability means that developers can now create Firefox WebRTC applications that make direct audio/video calls to Chrome WebRTC applications without having to install a third-party plugin. Because the functionality is now baked into the browser, users can avoid problems with first-time installs and buggy plugins, and developers can deploy their apps much more easily and universally.

    To help celebrate this momentous milestone, we thought it would be fun to call up our friends at Google to discuss it with them. Check out this Firefox-Chrome demonstration call between Mozilla’s Chief Innovation Officer, Todd Simpson, and Google’s Director of Product Management, Hugh Finnan, and read what Google had to say about this momentous occasion in their blog post.

    This milestone builds on an earlier demo we showed late last year of WebRTC integrated with Social API. There we demonstrated an industry first with our implementation of DataChannels, a powerful component of WebRTC that can combined with an audio/video chat to allow users to share almost anything on their computer or device. Send vacation photos, memorable videos, links news stories etc., simply by dragging the item into your video chat window. Look out for more on this to come.

    The purpose of WebRTC, an open standard being defined jointly at the W3C and IETF standards organizations, is to provide a common platform for all user devices to communicate and share audio, video and data in real-time. This is a first step toward that vision of interoperability and true, open, real-time communication on the web.

    Posted by:
    Serge Lachapelle, Chrome Product Manager and Maire Reavy, Firefox Media Product Lead

    Start Developing Using RTCPeerConnection in Firefox

    For JavaScript developers who haven’t tried RTCPeerConnection in Firefox yet (since it is a brand new feature for us), you can try this out using the most recent Firefox Nightly by setting the media.peerconnection.enabled pref to “true” (browse to about:config and search for the media.peerconnection.enabled pref in the list of prefs). Here is a snippet of code from a sample app that shows off how to initiate, accept, and end a WebRTC call in Firefox using RTCPeerConnection:

    function initiateCall(user) {
      document.getElementById("main").style.display = "none";
      document.getElementById("call").style.display = "block";
     
      // Here's where you ask user permission to access the camera and microphone streams
      navigator.mozGetUserMedia({video:true, audio:true}, function(stream) {
        document.getElementById("localvideo").mozSrcObject = stream;
        document.getElementById("localvideo").play();
        document.getElementById("localvideo").muted = true;
     
        // Here's where you set up a Firefox PeerConnection
        var pc = new mozRTCPeerConnection();
        pc.addStream(stream);
     
        pc.onaddstream = function(obj) {
          log("Got onaddstream of type " + obj.type);
          document.getElementById("remotevideo").mozSrcObject = obj.stream;
          document.getElementById("remotevideo").play();
          document.getElementById("dialing").style.display = "none";
          document.getElementById("hangup").style.display = "block";
        };
     
        pc.createOffer(function(offer) {
          log("Created offer" + JSON.stringify(offer));
          pc.setLocalDescription(offer, function() {
            // Send offer to remote end.
            log("setLocalDescription, sending to remote");
            peerc = pc;
            jQuery.post(
              "offer", {
                to: user,
                from: document.getElementById("user").innerHTML,
                offer: JSON.stringify(offer)
              },
              function() { console.log("Offer sent!"); }
            ).error(error);
          }, error);
        }, error);
      }, error);
    }
     
    function acceptCall(offer) {
      log("Incoming call with offer " + offer);
      document.getElementById("main").style.display = "none";
      document.getElementById("call").style.display = "block";
     
      // Here's where you ask user permission to access the camera and microphone streams
      navigator.mozGetUserMedia({video:true, audio:true}, function(stream) {
        document.getElementById("localvideo").mozSrcObject = stream;
        document.getElementById("localvideo").play();
        document.getElementById("localvideo").muted = true;
     
        // Here's where you set up a Firefox PeerConnection
        var pc = new mozRTCPeerConnection();
        pc.addStream(stream);
     
        pc.onaddstream = function(obj) {
          document.getElementById("remotevideo").mozSrcObject = obj.stream;
          document.getElementById("remotevideo").play();
          document.getElementById("dialing").style.display = "none";
          document.getElementById("hangup").style.display = "block";
        };
     
        pc.setRemoteDescription(JSON.parse(offer.offer), function() {
          log("setRemoteDescription, creating answer");
          pc.createAnswer(function(answer) {
            pc.setLocalDescription(answer, function() {
              // Send answer to remote end.
              log("created Answer and setLocalDescription " + JSON.stringify(answer));
              peerc = pc;
              jQuery.post(
                "answer", {
                  to: offer.from,
                  from: offer.to,
                  answer: JSON.stringify(answer)
                },
                function() { console.log("Answer sent!"); }
              ).error(error);
            }, error);
          }, error);
        }, error);
      }, error);
    }
     
    function endCall() {
      log("Ending call");
      document.getElementById("call").style.display = "none";
      document.getElementById("main").style.display = "block";
     
      document.getElementById("localvideo").mozSrcObject.stop();
      document.getElementById("localvideo").mozSrcObject = null;
      document.getElementById("remotevideo").mozSrcObject = null;
     
      peerc.close();
      peerc = null;
    }

    You’ll notice that Firefox still prefixes the RTCPeerConnection API call as mozRTCPeerConnection because the standards committee is not yet done defining it. Chrome prefixes it as webkitRTCPeerConnection. Once the standards committee finishes its work, we will remove the prefixes and use the same API, but in the meantime, you’ll want to support both prefixes so that your app works in both browsers.

    Trying Interop Yourself

    For those eager to give interop a try, here are instructions and information about “trying this at home”.

    This is Firefox’s and Chrome’s first version of PeerConnection interoperability. As with most early releases, there are still bugs to fix, and interop isn’t supported yet in every network environment. But this is a major step forward for this new web feature and for the Web itself. We thank the standards groups and every contributor to the WebRTC community. While there’s more work to do, we hope you’ll agree that the Web is about to get a lot more awesome.

  10. Introducing the Firefox OS Boilerplate App

    When coming to a new platform or context, it’s always good to get a peek at some code and examples how to make things work. With Firefox OS and app development, it’s just the web with a few additions.

    Before here at Mozilla Hacks, we’ve covered a few ways to get started with building apps for Firefox OS:

    My experiences

    Lately I’ve been fortunate enough to give and take part in a number of workshops around Firefox OS, to see developers trying to build things for it, port their existing web apps and much more.

    This has been a fantastic learning lesson for me, and it’s been crucial to see where people might need pointers, help or examples!

    The Firefox OS Boilerplate App

    This led to me creating the Firefox OS Boilerplate App. As the name implies, it’s there to provide you with the most basic features to get started with building an app from scratch, or tools to port your existing web app.

    The idea is also to avoid any dependency on external libraries or resources, but rather be self-contained.

    It contains:

    • An install button, offering you to install it as a hosted app
    • Web Activities – lots of examples and use cases
    • WebAPIs in action
    • Offline support (disabled by default)
    • Packaged apps – install your app as a ZIP file

    It’s available on GitHub:

    How to use it

    The easiest way to get started, installing it and testing the various features, is to navigate to the Firefox OS Boilerplate App in the web browser on a Firefox OS device or in the Firefox OS Simulator.

    Alternatively, install it in the Firefox OS Simulator Dashboard by providing either of these URLs:

    Running it locally

    Once you’re ready to get started developing, download the code and run it on a web server, or point out your local version of the Firefox OS Boilerplate App in the Firefox OS Simulator.

    Note: make sure that the paths in the manifest file are valid on your localhost – bear in mind that these paths are relative to the root of the web site they are being served at.

    Also make sure to configure your server to send the manifest file with the right Content-type: application/x-web-app-manifest+json.

    This is, for instance, easy to set up in an .htaccess file in Apache:

    AddType application/x-web-app-manifest+json .webapp

    Offline support

    I’ve provided an .appcache file for enabling offline support (it’s disabled by default).

    To enable offline capabilities, just add this to the index.html file:

    <html manifest="manifest.appcache">

    Please make sure to do your homework before enabling offline support, to avoid possible initial issues:

    Remember that the .appcache file has to be served as a text/cache-manifest file:

    AddType text/cache-manifest .appcache

    Packaged apps

    When you develop web apps, by default they are being delivered from a server, thus needing online connectivity or offline support to be enabled, to work as expected.

    You do have another option, though, which is packaged apps. Basically, what this is, is putting all the files of your app into a ZIP file, making them available directly on the device itself.

    Packaged apps can also request an elevated access to certain WebAPIs in Firefox OS that aren’t available to hosted apps (we’ll go more into the differences in a later post here on Mozilla Hacks).

    There are a couple of files included in the Firefox Boilerplate OS App to help you get started, if you are interested in this.

    To create and install a packaged app, you need to go through a few steps:

    • ZIP all app content (not containing folder), including regular manifest
    • Create a mini manifest (the package.manifest file) and make sure the “package_path” is absolute to where the ZIP is located
    • Developer name and info has to match between mini manifest and the regular one in the ZIP file
    • Have an installPackage call in JavaScript pointing to the mini manifest (instead of the regular install one) – this is shown in comments in the base.js file
    • Turn on Developer Mode in the Firefox OS Simulator (Settings > Device Information > More Information > Developer > Developer mode)
    • Add type property (e.g. "type" : "privileged") in the manifest if you want access to certain APIs

    Work in progress

    The Firefox OS Boilerplate App is a work in progress, meaning that it’s likely to change over time. I believe, however, that it gives you a good head start and look into what’s possible with web apps in Firefox OS.

    Hope you like it, and please let me know what you think!