Mozilla

Featured Articles

Sort by:

View:

  1. Don't miss out on the real-time fun! Use Firefox OS Push Notifications

    Firefox OS v1.1 introduces Push Notifications to Open Web Apps, allowing web developers to take advantage of real-time updates without implementing difficult polling logic themselves. Native Push Notifications support means
    that only one connection has to be maintained by Firefox OS devices, and applications can be shutdown, improving battery life, and device responsiveness, while still offering immediate updates to users.

    Firefox OS Push Notifications are designed for one thing – waking up apps. They do not deal with data, desktop notifications and other features, since there are other Web APIs that provide them. In this Hacks article, we’ll take a look at building a small chat application that uses Push Notifications to update the conversation.

    Scope and workflow

    Before we start on the application, it is a good idea to take a look at the scope and work flow of Push Notifications.

    No polling

    The scope of Push Notifications is to notify web applications on clients that something on the network has changed, so that the data the client has should be updated to the latest data on a remote server. Rather than the application having to constantly poll using XHR or use WebSockets, Push takes over responsibility and notifies the application of the change. This data is usually stored on the application server (in the cloud) and shared on multiple devices distributed amongst one or more user.

    For example, a user can have a personal calendar, which is shared between a laptop, a mobile and a tablet. In addition the user may have a work calendar that is shared with co-workers. A good calendar application on these devices will keep the devices in sync with the ‘master’ calendar on the server. With Push Notifications, Firefox OS devices can now do this sync immediately and be very responsive.

    No data

    Firefox OS Push Notifications do not support sending messages over push. From the very beginning Push Notifications was designed to explicitly not carry a payload. This might seem absurd, but it falls in line with Mozilla’s mission of ensuring its user’s privacy. Multiple organizations will be running Push Servers, and if data was to be allowed on the Push protocol, this data ends up in the hands of these third parties. It is very easy to accidentally leak confidential user data. For example if the chat application sent the actual messages over Push, and a user decided to send his credit card number to another user, the credit card number is now in the hands of the Push Server operator.

    To prevent this, the protocol simply does not allow data. The only application specific data allowed is a version number (optional). This version number is useful for collaborative applications. If a version number is sent, and the client device has already received a message with the same or older version number, it will not be notified again. The version number can be used to merge changes made by the user on this device with changes received from the server.

    Workflow

    Continuing with the calendar example, the flow of push notifications works like this. The mobile application registers for a push endpoint. This push endpoint is a unique URL that represents one channel for receiving updates. These URLs point to a third party Push Server, which is set on the device and may be operated by Mozilla or other organizations or individuals. Each application can have multiple push endpoints (imagine one for each calendar the user has). The mobile application then sends this endpoint to the application server. A calendar server would associate this endpoint with the user and the specific calendar. Every calendar would have a set of endpoints, one for every device of every user that is subscribed to the calendar.

    When any user changes the calendar on his device, the application saves those changes to the server. This is the point at which Push Notifications kick in. The application server will make a HTTP request to every one of the endpoints in the set for the calendar that was modified. Each Push Server now takes responsibility for delivering messages to the devices. When each device receives the notification, it starts the calendar application on the device and notifies it. The calendar app can then go and fetch all the changes from the calendar server.

    NOTE: This URL should be treated as opaque and your application and application server should not make any assumptions about them, nor edit them.

    What happens if my application is not running?

    Push notifications aren’t just a convenience, they are essential to mobile applications that need to keep in sync but may not always be running. If your application is not running when a push notification is received, the application will be started, navigated to the handling page and the message delivered to it. If you open the application switcher after receiving a push, you’ll be able to see the application running in the background.

    Setting up the environment

    There are many resources on writing applications for Firefox OS with proper tooling and using all the facilities provided by the system. The two part hacks series Building a Todo app for Firefox OS is a great example. The chat application will instead be very bare bones. We’ll not concern ourselves with extra tooling like Volo. Nor will the app be very good looking by using standard UX elements. We’ll stick to simple HTML and JavaScript, with only jQuery and Bootstrap to for a few utilities.

    The server will be written in Python, using the Flask framework, which is small and should be easy to understand even if you haven’t used Python before. In addition the server uses Redis for a persistent data store.

    All the code is available on Github.

    Since Push Notifications is available on Firefox OS v1.1 onwards, you’ll currently need preview builds of the Firefox OS Simulator (please note that these are experimental, and not all might work as expected). Paste the URL for your platform into Firefox, install the Simulator add-on and you should be good to go with using Push.

    The stable builds of the simulator on addons.mozilla.org DO NOT have push.

    The app manifest

    Using Push Notifications requires three important entries in the applications manifest.

    The push permission allows the application to use Push Notifications. We also request desktop-notification permission so we can show a notification when a user is mentioned in the chat.

    The messages entries specify which page of the application should be notified when a push notification is received. In this case, since our application is a single page application, we set it to /. Firefox OS v1.1 only allows the page that registers for Push Notifications to recieve them, so you cannot have / call navigator.push.register() and have /push.html be the receiver that uses navigator.mozSetMessageHandler().

    The push message is the message delivered when an actual notification is recieved. The push-register message is a error recovery mechanism. In rare cases the Push Server may suffer data loss or it may decide to free up some old endpoints. In either case, if your application’s endpoints are no longer valid, it will receive a push-register message, in which case your application should register for new endpoints and update the app server.

    The main page

    Our chat application will be a single page application with the latest 50 messages, followed by a field to enter a new message.

    The unordered list chat will hold the chat messages, which will be loaded by JavaScript. theform is used to submit messages. The div nickPrompt is used as a modal dialog to ask the user’s nickname the first time the application is run. Finally we include the various scripts that drive the application.

    Offline web apps

    As web applications are used on mobile devices, it is important to design them such that they can be used offline as much as possible. A common pattern in such a case is to have persistent storage on the client side using localStorage or IndexedDB. This storage is updated when a network connection is available, and the user content is displayed solely from this offline copy. This way the user can always interact with content that is already on the device. Our application follows a similar model. The latest chat messages are stored on the device using IndexedDB. A push notification will first update this local data and then update the UI.

    Chat Database

    The storage is implemented in model.js which abstracts an IndexedDB database to support the operations a simple chat implementation requires. Every message is stored as a object:

    {
      "id": 5, // Integer message ID. Unique for every message.
      "nick": "EdwardSnowden", // The sender of the message.
      "message": "The NSA is spying!" // The actual text.
    }

    Only the latest 50 messages are stored on both the server and the client in a bid to compete against Snapchat.

    The id field is a perfect candidate to use as the version field of the push notifications. In the case of a chat, every message will be new, so the version field is not so important, but we’ll use it for the purposes of demonstration.

    Installing the application

    In Firefox OS v1.1, Push Notifications are only available to applications and not to web pages loaded in the browser. So when the user visits our application in the browser, it will prompt to install the app.

    Registering for Push Notifications

    When the user starts the application, go() is called. This prompts the user for a nickname on the first run, and populates the chat list using any existing messages already available offline.

    The important bit for Push Notifications is this:

    navigator.push is the PushManager object that provides the Push Notifications operations. You can check for

    if ('push' in navigator) {
    }

    to see if the page can use Push Notifications.

    registrations() is used to check if the application has existing endpoints registered. Like several other Firefox OS APIs it returns a DOMRequest object since it is a async operation. On success, the result is a list of PushRegistration objects. A PushRegistration has the following fields:

    interface PushRegistration {
      string pushEndpoint; // A URL specifying the endpoint.
      unsigned long int version; // The latest known version.
    }

    If we have no registrations, we ask for a new registration using navigator.push.register(). On a successful registration, the result field of the DOMRequest will be a URL, which is the endpoint for this registration. Now that we have a endpoint, we use XHR to send it to our server.

    On the server, lets just store all the endpoints in one list called endpoints:

    In addition we send a quick ping to the new endpoint that was added. We use the great Requests library for this, and you can see how easy it is to use Push Notifications on the server. Just do a HTTP PUT request to the endpoint with the body of the request being:

    version=<number>

    The call to navigator.push.register() can fail due to several reasons. The most common is that the device does not have network connectivity. The DOMRequest’s onerror handler will be fired in such a case and the request’s
    error will have a short descriptive name of the error.

    Dispatching notifications

    Every time a new message is posted to the chat, we want to notify all clients via push notifications. When a form is submitted:

    the server is sent the message text and the sender’s nickname.

    We request an id for the message from Redis using the INCR atomic command. We also want to enforce two things on the server:

    1. Keep the messages sorted by this message id, so we can quickly serve messages from a certain ID onwards. For this we use Redis’ sorted sets with the score as the message id. This keeps it sorted, and allows us to quickly fetch messages from a certain ID onwards, so that clients do not have to download all messages but we do not have to maintain individual queues for clients.
    2. Restrict the messages to the latest 50 (MAX_MESSAGES). We do this by truncating the set using ZREMRANGEBYRANK.

    We use redis-py‘s pipeline system to do both of these atomically so that if adding fails, so does removing older messages. Your language/framework will have similar operations to achieve this consistency.

    The message itself is stored as a string of the JSON representation so that sending messages to clients is just a matter of extracting elements and concatenating them into a JSON list.

    After that we put the new message id in the notification_queue. This brings us to a best practice when using push notifications. The application server will often be notifying multiple endpoints about changes. In such a case, blocking the main server is a bad idea, Instead you should have a dedicated thread or process that can ping endpoints. This can even be moved off to a dedicated machine. The version number system allows you to parallelize the pinging.

    In this simple server, we’re using Python’s builtin Queue which lets us implement a producer-consumer system. The thread that notifies the push server is started in app.py and runs the notify() function:

    The notify() function blocks on the notification_queue. When a new message id is available, it loops through all the endpoints and notifies them of the new version.

    A successful PUT will result in a 200 OK response from the endpoint. This means the notification has been queued for delivery. When the device is online and connected to the Push Server, delivery happens within a few seconds.

    The server’s job is done! Now the application has to handle the push notification.

    Receiving push notifications

    Receiving push notifications is done by setting a function to be invoked when the ‘push’ message is received. This is done using mozSetMessageHandler. NOTE: The page that sets the receiver and the page listed as the message
    receiver in the manifest file must match!

    After the model is successfully initialized we set the handler. message will be a:

    interface PushMessage {
      string pushEndpoint; // The push endpoint that changed.
      unsigned long int version; // The new version.
    }

    The pushEndpoint allows the app to identify which endpoint (among the many it
    may have registered) has changed.

    In this case, we just want to use this version to download new chat messages if required.

    model.latest() returns the latest message ID in our local storage, and we download new messages only if the version of the push notification is newer.

    The server has a /message/<id> REST interface, which given a message ID will return all the messages after it:

    It simply uses the message ID as the sorted set score and returns a JSON response.

    On the client, we update the local storage. To handle mentions, we also loop through the new messages received and check if any of the messages have the user’s nickname. If one is found, a desktop notification is created containing the message, so that the user notices a presumably important message.

    Finally the UI is updated:

    Since old messages have to be removed from view, we remove all the list items and add new ones. Modern Web Runtimes will do only one repaint even if multiple new elements are added to the DOM, as long as they are all added at once. So there is no flickering. Here too, we give messages mentioning the user a different colour.

    A little thing to handle is the push-register message. If you remember, I mentioned that this is a error recovery mechanism that may be invoked.

    Our demo application simply shows an error to the user, but ideally you’d silently register for new endpoints in the background.

    Unregistering

    A very common design pattern with push notifications is to associate one endpoint per user, where the user has a unique ID in your application and application server (such as an email ID). In that case your application may be designed to support switching users. In this case you no longer want to receive notifications when something of interest to the old user happens. There may be other situations in which you are no longer interested in a push endpoint. The way to express this lack of interest is to use navigator.push.unregister().

    // Assuming pushEndpoint1 is the endpoint associated with an old user
    var request = navigator.push.unregister(pushEndpoint1);

    unregister() returns a DOMRequest whose onsuccess will fire as soon as the registration is deleted by Firefox OS. It is extremely unlikely that a call to unregister() will fail. The most likely cause of failure is that the endpoint is not a valid endpoint that was issued to the application by Firefox OS.

    Good design: Decouple Push from the user flow

    Although the most common use of Push Notifications for web apps will be to show a desktop notification to the user, the process of push registration, receiving pushes, and unregistering should be kept out of the user’s UI flow as
    much as possible. This is most applicable during registration.

    Similar to progressive enhancement of the Web Application user experience, your app should be ready to deal with lack of Push notifications. For example, you should not block the user from signing into your application simply because navigator.push.register() failed due to lack of connectivity. Instead, let the user continue to use the offline part of your application and queue the registration request to try again after some time. The Alarm API is useful to wake your application up later to finish such ‘background’ tasks.

    Similarly, data heavy applications should not wait around for the user to respond to a desktop notification before they fetch data. A good calendar application will immediately go and update its local storage on receiving a push. Only then will it pop up any reminders so that as soon as the user launches the application, he immediately sees new events and changes. At the same time it should unregister() endpoints if the user selects a manual sync option.

    Important security considerations

    The push endpoint obtained from register() should be kept secret! The best way to do this is:

    1) Always send endpoints to the application server over a secure (https) connection. Otherwise an attacker could perform a man-in-the-middle attack and capture endpoints going over the wire.

    2) Keep your database secure. The endpoints should be just as well protected as users personal data.

    If an attacker were to gain access to endpoints, he or she could affect devices that have those endpoints. For example they could cause your application to run frequently, affecting the user experience. Or they could do a push to the endpoint with a very high version number so that subsequent valid pushes that your application server does will not be passed on to the app.

    Connections between the Push Server and the devices are always over a secure connection and thus there can be a reasonable expectation of safety on that part of the exchange.

    With that we’ve taken a short tour of the Firefox OS Push Notifications API. It will be exciting to see how you developers will leverage this to create fantastic web applications.

    The Push Notifications team may be reached on the #push IRC channel on irc.mozilla.org.

    Extras

    Push Notification delivery guarantees

    Push Notifications is a best effort system. The Push Server will try to inform a device of the latest version it has received for an endpoint. This means that updates are collapsed. If a endpoint is updated to version 254 while the device is offline, and then updated to 255, and then the device comes online, it will receive only the update for 255. When device and server end up in a inconsistent state that cannot be fixed, push-register is usually fired to allow applications to reset itself. Despite these precautions, there are rare situations in which:

    Notifications may fail to be delivered

    If the device has been offline for more than a few days (a week or more), the Push Server may drop pending notifications to conserve its resources. In such a case, the device will not receive updates after it comes back online until the application server sends new updates. This is possible if the user is travelling on roaming and switches off mobile data.

    If the device battery dies or the device crashes just as the notification is being delivered to the application, the application will miss that notification. This is extremely rare.

    push-register may not be delivered

    push-register may not be delivered even after the Push Server has reached an inconsistent state with the device.

    If the device was offline when the Push Server lost state, the Push Server may not be able to inform the device that all state was lost when it comes back online.

    For most applications the above should not be a problem since they will occur very rarely. If your application requires more reliability, you can mitigate the above possibilities by using a few other methods to sync. One way is to acquire new endpoints periodically rather than waiting for a push-register. The other way is to use the Alarm API to do an unconditional synchronization with the application server after an extended period of time. For example, the application would use push to receive regular updates, but set an alarm for midnight every day when it would go and fetch updates regardless of the push status.

    Statuatory warning about experimental nature

    The Push Notifications API for Firefox OS is experimental. One of the major changes coming in newer versions of Firefox OS will be a switch away from DOMRequest to Promises. Until Bug 800431 is fixed, the page calling register() will have to be the same one that handles the notifications (‘push’ system message). In the future we would like to add support for background services or workers to handle notifications instead. This will ensure that no UI is launched unless the application requests it. There may be additional features added and in an extremely rare case, existing APIs may change.

    While Mozilla will do its best to not break existing applications, developers should be aware of and plan for changes that may be required to their applications.

  2. WebRTC and the Ocean of Acronyms

    My experience getting started with WebRTC can be summarised in a three letter acronym so I decided to write this article dedicated to answering my many questions. I’ve always said, if you don’t know an acronym, it’s probably a networking protocol.

    What is ICE?

    Interactive Connectivity Establishment (ICE) is a framework to allow your web browser to connect with peers. There are many reasons why a straight up connection from Peer A to Peer B simply won’t work. It needs to bypass firewalls that would prevent opening connections, give you a unique address if like most situations your device doesn’t have a public IP address, and relay data through a server if your router doesn’t allow you to directly connect with peers. ICE uses some of the following techniques described below to achieve this:

    What is STUN?

    Session Traversal Utilities for NAT (STUN) (acronym within an acronym) is a protocol to discover your public address and determine any restrictions in your router that would prevent a direct connection with a peer.

    The client will send a request to a STUN server on the internet who will reply with the client’s public address and whether or not the client is accessible behind the router’s NAT.

    What is NAT?

    Network Address Translation (NAT) is used to give your device a public IP address. A router will have a public IP address and every device connected to the router will have a private IP address. Requests will be translated from the device’s private IP to the router’s public IP with a unique port. That way you don’t need a unique public IP for each device but can still be discovered on the internet.

    Some routers will have restrictions on who can connect to devices on the network. This can mean that even though we have the public IP address found by the STUN server, not anyone can create a connection. In this situation we need to turn to TURN.

    What is TURN?

    Some routers using NAT employ a restriction called ‘Symmetric NAT’. This means the router will only accept connections from peers you’ve previously connected to.

    Traversal Using Relays around NAT (TURN) is meant to bypass the Symmetric NAT restriction by opening a connection with a TURN server and relaying all information through that server. You would create a connection with a TURN server and tell all peers to send packets to the server which will then be forwarded to you. This obviously comes with some overhead so is only used if there are no other alternatives.

    What is SDP?

    Session Description Protocol (SDP) is a standard for describing the multimedia content of the connection such as resolution, formats, codecs, encryption, etc so that both peers can understand each other once the data is transferring. This is not the media itself but more the metadata.

    What is an Offer/Answer and Signal Channel?

    Unfortunately WebRTC can’t create connections without some sort of server in the middle. We call this the Signal Channel. It’s any sort of channel of communication to exchange information before setting up a connection, whether by email, post card or a carrier pigeon… it’s up to you.

    The information we need to exchange is the Offer and Answer which just contains the SDP mentioned above.

    Peer A who will be the initiator of the connection, will create an Offer. They will then send this offer to Peer B using the chosen signal channel. Peer B will receive the Offer from the signal channel and create an Answer. They will then send this back to Peer A along the signal channel.

    What is an ICE candidate?

    As well as exchanging information about the media (discussed above in Offer/Answer and SDP), peers must exchange information about the network connection. This is know as an ICE candidate and details the available methods the peer is able to communicate (directly or through a TURN server).

    The entire exchange in a complicated diagram

    image

  3. Firefox OS Simulator 4.0 released

    It’s a good day for Firefox OS developers as we are releasing version 4.0 of the Firefox OS Simulator to the masses. In particular, this release is a boon for those developers that want to make money using their app in the Marketplace.

    What’s New in 4.0

    4.0 Updated Simulator Dashboard
    An overview shot of the revised Dashboard

    New Connect Button

    There is a new ‘Connect’ button for each app that opens a developer toolbox connected to that specific app. This means that you won’t have to search through messages in the Console or filter through scripts in the Debugger in order to find information specific to your app.

    Testing Receipts for Paid Apps

    There is now a dropdown menu in each app’s dashboard where you can select a receipt type. The simulator add-on will then download a test receipt from a Marketplace receipt service and reinstall the app using it. This way you can test receipt verification with the various types of receipts that you may require – valid, invalid, and refunded.

    4.0 Updated Simulator Dashboard Buttons
    The new Connect button, Refresh Button, and Receipts drop-down

    Remote CSS Styling

    If you connect to an app while using a Nightly or Aurora build of Firefox, there is a Style Editor tool you can use to edit the style sheets for your app. Changes are applied instantaneously.

    4.0 Updated Simulator Live Style Editing
    Live editing the Firefox OS Boilerplate app to have a less than charming red background

    Simulated Touch Events

    Gaia’s touch events simulation has been integrated such that interacting with the Simulator using a mouse now generates real touch events. This fixes a myriad of issues in core Gaia apps that assume touch interactions. It also means you can test third party apps that rely on touch events without needing to fall back to mouse events.

    Hidden Feature: Shift-Ctrl/Cmd-R

    When using the keyboard shortcut Ctrl-R (Cmd-R on Mac) to refresh an app, if you also hold down the Shift key, then the Simulator will clear persistent data such as AppCache, localStorage, sessionStorage, and IndexedDB while refreshing the app.

    Check Out The Simulator Walkthrough

    Still want to get into grittier details? Check out the simulator walkthrough to get a deep dive into the details of the Simulator, and the MDN documentation here.

    Download and Install the Simulator

    You can install or update the Simulator from the add-ons website.

    Bugs? Feedback?

    Leave general feedback in the comments below, we’re listening! If you encounter a bug we would be grateful if you could file it here.

  4. Firefox OS devices officially released!

    Almost two years ago, we announced Boot to Gecko (B2G) here on Mozilla Hacks. We discussed the aims of the project and the work we were planning to do. Today, all that work has paid off and we now have official Firefox OS devices in store!

    Last week the first Firefox OS phones went out in stores in Madrid, Spain, for sale by Telefónica. These phones are a great example of a low-cost telephone built on Open Web Technologies.

    Firefox OS launch pics, Madrid, Spain

    It means that if you haven’t gotten started at looking how to build your apps or your content with HTML5 and Responsive Design, now is the time, to have the chance to be among the early options in a brand new market.

    Coming to more countries

    Telefonica’s launch in Spain was the first for Firefox OS, but Poland, Colombia and Venezuela also have upcoming launches soon. It doesn’t stop there, though, look out for more countries coming soon.

    Open Web Apps

    Firefox OS runs the web and has a web browser, but if you want to offer an app-like experience, get access to more hardware and APIs and more, building an Open Web App could be a perfect option for you.

    Firefox OS launch pics, Madrid, Spain

    Reuse your existing HTML5 skills and content and just package it up. We’ve outlined Open Web Apps and new possibilities in detail in previous articles:

    The Firefox OS Simulator

    Tool-wise, we believe the Firefox OS Simulator is a great option to quickly test your apps.

    Version 4 is just about to be released – more details in the announcement this Thursday, July 11th – but the list of highlights include:

    • New Connect Button, to have a developer toolbox connected to your specific app(s)
    • Remote Style Editor, enable remote style editing
    • Hidden Feature: Shift + Ctrl/Cmd-R, refreshing the app and clearing persistent data (AppCache, localStorage, sessionStorage, and IndexedDB)
    • Updated Dashboard Design, reduced clutter and improved usability
    • Touch Events, integrating Gaia’s touch events simulation, so interacting with the Simulator using a mouse now generates real touch events

    Introducing payments

    As part of the Firefox OS offering, we have the Firefox Marketplace, where you can upload and share your apps. We are also happy to say that if you are interested in accepting payments for your app, this is the place!

    They are described in the Payments page as part of the Developer Hub aimed at app developers, and we’ve also written about them here before in Building A Paid App For Firefox OS.

    We have two kinds of payment options in the Firefox Marketplace:

    • Purchase an app
    • In-app payments

    Firefox OS is here

    All in all, Firefox OS is here available on released devices, and we hope the capabilities for you as an Open Web Developer to offer good content and experiences sound good. We are continuously working on making the options and possibilities as good as we can for you!

  5. Network Monitor, now in Firefox Beta

    The Firefox Developer Tools team is particularly proud announce that Firefox 23 (in Firefox Beta, to be released today) ships with an initial but very functional Network Monitor tool that not only provides similar functionality to other tool sets, but in many improves on them. This important step is the result of lots of hard work from the entire team, but in particular Victor Porof and Mihai Sucan.

    Let’s get started

    To give you the best idea of how the network monitor works, here’s a series of steps to follow:

    1. Download, install and run Firefox 23 beta if you haven’t already (or Firefox Aurora or Firefox Nightly – more on Firefox and the release channels).
    2. Open the Network Monitor tool:
      • from the menu: Tools => Web Developer => Network
      • from the keyboard: Ctrl + Alt + Q (Windows/Linux) or Cmd + Alt + Q (Mac OS X)
    3. Load an interesting site.

    As Firefox loads the page you’ll see each individual request get added as a row, much as you would expect from other tools. In particular it is now very easy to visualize not just how quickly parts of the page load and in what order, but also where problems are: missing assets, slow web servers, buggy apis.

    As you get used to using the Network Monitor, you’ll also notice that you don’t need to necessarily hit refresh on a page that is already loaded, once you open the network tool any subsequent requests that happen will be logged. For example, if you’re interested in xhr requests made by an app, just open the monitor and start clicking buttons – you should see any api calls logged as normal without needing to reload.

    Status at a glance

    One thing you should notice right away about the way the Network tool displays a page load is that we’ve taken care to provide both information and design cues to show you what is happening at a glance as the page loads:

    We’ve tried to create a clean UI that delivers key aspects of how an entire page loads without creating too much visual clutter:

    • An error such as a 404 are colored dark red to make them stand out visually from the green successful or yellow cached requests.
    • The url is parsed out and only the file name and domain are shown.
    • By default the load is sorted in chronological order, but you can also sort by any of the top columns, for example by domain or filename.
    • You can easily see the type of request, whether it be html content, an image or an API request.
    • You can click on the column headers to sort requests by method, file name, domain, type or size.

    It’s all in the details

    Once all the page’s assets have been loaded, clicking on a single request opens up the sidebar view which provides detailed information about that request, broken down into panels that display different aspects of a given request: Headers, Cookies, Parameters, Response and Timings. Similar to how the Inspector tool works, this panel is responsive and will neatly pop down below the request list if you’ve docked the tools to the right side:

    A quick tour

    The Headers panel is particularly useful for debugging asset caching problems, and this panel provides parsed lists for both the request and response headers that also supports filtering based on either name or value:

    The Timings panel gives you a detailed, visual breakdown of where time was spent for the selected request. Is it DNS? Is it the server itself?

    If you’re debugging API requests or form posts, you’ll really like how we’ve parsed out the url parameters in the Params panel:

    Are you using a lot of API calls to improve the responsiveness of your app? JSON responses are parsed and displayed as a navigable object in the Response panel to make it easy to check on the data coming back from the server. You can filter the data based on JSON values as well.

    Next Steps

    It’s great to deliver the Network Monitor to a much wider set of developers in Firefox 23 Beta but we haven’t stopped adding new features in the meantime. Firefox 24 ( now in the Aurora channel ) features several notable improvements that have landed in the last 6 weeks or so:

    • You can now filter requests based on the request ‘type’, including XHR, images, html.
    • POST request parameters are supported in the Params panel.
    • You can browse the list of requests via the Up/Down arrow keys.
    • Many more bug fixes and refinements!
  6. WebRTC comes to Firefox

    As we mentioned in the Hacks blog back in April , WebRTC will be on by default in Firefox 22. getUserMedia (gUM) has been on by default since Firefox 20. PeerConnection and DataChannel, which enable video/audio calling and peer-to-peer data sharing, are what’s new in Firefox 22 (due to be released today).

    WebRTC brings real-time communication to the web for the first time ever, and we’re excited to get this new technology into the hands of developers. We believe the industry has only scratched the surface of what’s possible with WebRTC, and only by getting it into the hands of developers and early adopters will we see this technology’s true potential.

    Known issues/limitations

    There are a few known issues/limitations in the early releases:

    • We are initially focused on getting 1:1 calling working well. We’ve done nothing to prevent conference or mesh calling, but depending on the capabilities of your device, video calls with multiple participants may be sluggish. We will be improving multi-person calling in future releases. Our roadmap includes full support for multi-person/conference/mesh calling and we expect to improve the experience in future releases.
    • You may hear echo on calls when you or the party you’re talking to is playing sound over your computer speakers. We’re working on improving echo cancellation but for the time being, try wearing headphones if you experience this problem.
    • On some systems, you may experience audio delay relative to the video. We’ve isolated the problem and are working on a fix for a near-term Firefox release.
    • If you are behind a particularly restrictive NAT or firewall, you may have trouble connecting. We are adding support for media relaying (TURN) in Firefox 23, so you should find this improving soon.

    Trying WebRTC support today

    If you’d like to try out Firefox’s WebRTC support today, here are some sites that support WebRTC calling:

    NOTE: most of these sites support 3 or more callers. We expect basic 1:1 (2-person) calling to perform well enough for developer and early adopter use. As mentioned above, you may find that your mileage may vary with 3-or-more person calling using the current release.

    If you’re a developer interested in embedding WebRTC video chat into your website, please check out article on that.

    Testing DataChannels

    You can also try out DataChannels in Firefox, which is the first browser to launch a spec-compliant implementation of DataChannels to the market. Some sites and projects that use DataChannels:

    Using Firefox Nightly to test the latest

    I still encourage developers to use Firefox Nightly because it has the latest and greatest code and improvements, and we will be continuing to improve existing features and add new ones as we get feedback from developers and users and as the WebRTC standard itself evolves.

    Rapid progress!

    We expect new WebRTC sites, supporting PeerConnection and DataChannels, to come online rapidly over the next several months. We’ll keep you updated on our progress and on WebRTC’s progress here on Mozilla Hacks.

  7. Optimizing your JavaScript game for Firefox OS

    When developing on a quad core processor with 16 gigabytes of RAM you can easily forget to consider how it will perform on a mobile device. This article will detail some best practices and things to consider for moving a game to Firefox OS or any similar hardware target.

    Making the best of 256 Mb RAM/800 Mhz CPU

    There are many areas of focus to keep in mind while developing a game. When your goal is to draw 60 times a second, garbage collection and inefficient drawing calls start to get in your way. Let’s start with the basics…

    Don’t over-optimize

    This might sound counter-intuitive in an article about game optimization but optimization is the last step; performed on complete, working code. While it’s never a bad idea to keep these tips and tricks in mind, you don’t know whether you’ll need them until you’ve finished the game and played it on a device.

    Optimize Drawing

    Drawing on HTML5 2D canvas is the biggest bottleneck in most JavaScript games, as all other updates are usually just algebra without touching the DOM. Canvas operations are hardware accelerated, which can give you some extra room to breath.

    Use whole-pixel rendering

    Sub-pixel rendering occurs when you render objects on a canvas without whole values.

    ctx.drawImage(myImage, 0.3, 0.5)

    This causes the browser to do extra calculations to create the anti-aliasing effect. To avoid this, make sure to round all co-ordinates used in calls to drawImage using Math.floor or as you’ll reader further in the article, bitwse operators.

    jsPerf – drawImage whole pixels.

    Cache drawing in an offscreen canvas

    If you find yourself with complex drawing operations on each frame, consider creating an offscreen canvas, draw to it once (or whenever it changes) on the offscreen canvas, then on each frame draw the offscreen canvas.

    myEntity.offscreenCanvas = document.createElement(“canvas”);
    myEntity.offscreenCanvas.width = myEntity.width;
    myEntity.offscreenCanvas.height = myEntity.height;
    myEntity.offscreenContext = myEntity.offscreenCanvas.getContext(“2d”);
     
    myEntity.render(myEntity.offscreenContext);

    Use moz-opaque on the canvas tag (Firefox Only)

    If your game uses canvas and doesn’t need to be transparent, set the moz-opaque attribute on the canvas tag. This information can be used internally to optimize rendering.

    <canvas id="mycanvas" moz-opaque></canvas>

    Described more in Bug 430906 – Add moz-opaque attribute on canvas.

    Scaling canvas using CSS3 transform

    CSS3 transforms are faster by using the GPU. Best case is to not scale the canvas or have a smaller canvas and scale up rather than a bigger canvas and scale down. For Firefox OS, target 480 x 320 px.

    var scaleX = canvas.width / window.innerWidth;
    var scaleY = canvas.height / window.innerHeight;
     
    var scaleToFit = Math.min(scaleX, scaleY);
    var scaleToCover = Math.max(scaleX, scaleY);
     
    stage.style.transformOrigin = "0 0"; //scale from top left
    stage.style.transform = "scale(" + scaleToFit + ")";

    See it working in this jsFiddle.

    Nearest-neighbour rendering for scaling pixel-art

    Leading on from the last point, if your game is themed with pixel-art, you should use one of the following techniques when scaling the canvas. The default resizing algorithm creates a blurry effect and ruins the beautiful pixels.

    canvas {
      image-rendering: crisp-edges;
      image-rendering: -moz-crisp-edges;
      image-rendering: -webkit-optimize-contrast;
      -ms-interpolation-mode: nearest-neighbor;
    }

    or

    var context = canvas.getContext(‘2d’);
    context.webkitImageSmoothingEnabled = false;
    context.mozImageSmoothingEnabled = false;
    context.imageSmoothingEnabled = false;

    More documentation is available on MDN for image-rendering.

    CSS for large background images

    If like most games you have a static background image, use a plain DIV element with a CSS background property and position it under the canvas. This will avoid drawing a large image to the canvas on every tick.

    Multiple canvases for layers

    Similar to the last point, you may find you have some elements that are frequently changing and moving around whereas other things (like UI) never change. An optimization in this situation is to create layers using multiple canvas elements.

    For example you could create a UI layer that sits on top of everything and is only drawn during user input. You could create game layer where the frequently updating entities exist and a background layer for entities that rarely update.


    Don’t scale images in drawImage

    Cache various sizes of your images on an offscreen canvas when loading as opposed to constantly scaling them in drawImage.

    jsPerf – Canvas drawImage Scaling Performance.

    Be careful with heavy physics libraries

    If possible, roll your own physics as libraries like Box2D don’t perform well on low-end Firefox OS devices.

    When asm.js support lands in Firefox OS, Emscripten-compiled libraries can take advantage of near-native performance. More reading in Box2d Revisited.

    Use WebGL instead of Context 2D

    Easier said than done but giving all the heavy graphics lifting to the GPU will free up the CPU for greater good. Even though WebGL is 3D, you can use it to draw 2D surfaces. There are some libraries out there that aim to abstract the drawing contexts.

    Minimize Garbage Collection

    JavaScript can spoil us when it comes to memory management. We generally don’t need to worry about memory leaks or conservatively allocating memory. But if we’ve allocated too much and garbage collection occurs in the middle of a frame, that can take up valuable time and result in a visible drop in FPS.

    Pool common objects and classes

    To minimise the amount of objects being cleaned during garbage collection, use a pre-initialised pool of objects and reuse them rather than creating new objects all the time.

    Code example of generic object pool:


    Avoid internal methods creating garbage

    There are various JavaScript methods that create new objects rather than modifying the existing one. This includes: Array.slice, Array.splice, Function.bind.

    Read more about JavaScript garbage collection

    Avoid frequent calls to localStorage

    LocalStorage uses file IO and blocks the main thread to retrieve and save data. Use an in-memory object to cache the values of localStorage and even save writes for when the player is not mid-game.

    Code example of an abstract storage object:


    Async localStorage API with IndexedDB

    IndexedDB is a non-blocking API for storing data on the client but may be overkill for small and simple data. Gaia’s library to make localStorage API asynchronous, using IndexedDB is available on Github: async_storage.js.

    Miscellaneous micro-optimization

    Sometimes when you’ve exhausted all your options and it just won’t go any faster, you can try some micro-optimizations below. However do note that these only start to make a difference in heavy usage when every millisecond counts. Look for them in your hot game loops.

    Use x | 0 instead of Math.floor
    Clear arrays with .length = 0 to avoid creating a new Array
    Sacrifice some CPU time to avoid creating garbage.
    Use if .. else over switch
    jsPerf – switch vs if-else
    Date.now() over (+ new Date)
    jsPerf – Date.now vs new Date().getTime() vs +new Date
    Or performance.now() for a sub-millisecond solution
    Use TypedArrays for floats or integers (e.g. vectors and matrices)
    gl-matrix – Javascript Matrix and Vector library for High Performance WebGL apps

    Conclusion

    Building for mobile devices and not-so-strong hardware is a good and creative exercise, and we hope you will want to make sure your games work well on all platforms!

  8. Firefox Developer Tool Features for Firefox 23

    Another uplift has left the building and it’s time to take a look at what’s in Firefox Developer Tools in Firefox 23 currently Aurora, our pre-beta channel. You can download it from the Aurora Download page today. Firefox 23 is currently scheduled to hit the release channel on Tuesday August 6th, 2013.

    Episode XXIII is a barn-storming, hair-raising spectacle of incredible epicness that is sure to delight and amuse. If you want a sneak peak at features under active development, give Nightly a try.

    Network Monitor

    There’s a new tool in the toolbox: The Network Monitor. It’s a classic waterfall timeline view of network activity on a site. This data’s been available since Firefox 4 via the Web Console, albeit in a less visually pleasing way.

    Please file bugs under the Developer Tools: Netmonitor component in Bugzilla.

    Remote Style Editor

    In Firefox 23, you can now edit styles via a Remote Connection on a suitably-enabled device. This should be great help for App Developers interested in testing and debugging styles on a mobile device over the remote protocol in real time.

    As of the time of this writing, the Remote Style Editor should be compatible with Firefox for Android version 23, also scheduled for uplift to Aurora. We are working on incorporating the Style Editor Actors for the remote protocol into the Firefox OS Simulator and investigating what it will take to backport them to Firefox OS release.

    Options Panel

    We’ve added a Gear menu to the Toolbar containing an Options panel for turning tools on or off. As we add more stuff, this is going to be a popular place to manage preferences related to the Developer Tools.

    Currently, there are options to turn on the Light or Dark theme for the tools and enable Chrome Debugging.

    Initial SourceMap Support for Debugger Protocol

    The first pieces of SourceMap support for the Debugger have landed and we are now able to serve up SourceMapped JS files for your debugging pleasure. Soon to follow will be column support for breakpoints allowing you to debug minified JS with a SourceMap.

    Watch for the upcoming blog post by Nick Fitzgerald on Hacks explaining the magic.

    Variables View Everywhere

    Our Variables View is an improved Object Inspector and an integral part of our Debugger. Naturally, we wanted to put it in Everything. So now the Web Console and Scratchpad have a Variables View. Use the ESC key to close it.

    Browser Console

    If you have Chrome Debugging turned on, check out the Browser Console. It’s a replacement for the ancient Error Console and gives you a Chrome-context command line for executing JavaScript against the browser. It’s nice and should be enabled by default in Firefox 24.

    GCLI Appcache Command

    We finally have a little something for developers trying to use App Cache to store offline data. A new appcache command for the Graphical Command Line. You can read about it in Mike Ratcliffe’s The Application Cache is no longer a Douchebag.

    Web Console in Debugger Frame

    The Web Console is now fully-remoted (and has been since version 18). It now makes use of the Debugger’s current Frame if paused on a breakpoint.

    Multiple Paused Debuggers

    You can now debug multiple tabs at the same time using the Script Debugger. Previously, when attempting to use the debugger on two separate tabs, you’d be given a notification to resume the debugger in the other tab. Now you can debug as many tabs as you like.

    There is one caveat to this awesome power, however. Due to the nested event loops each Debugger creates, you have to resume each tab in the order in which they were paused. Debug carefully and always carry a big stack.

    You can see a comprehensive list of bugfixes in table form in Firefox 23 Developer Tools Fixes.

  9. Embedding WebRTC Video Chat Right Into Your Website

    Most of you remember the Hello Chrome, it’s Firefox calling! blog post right here in Mozilla Hacks demonstrating WebRTC video chat between Firefox and Chrome. It raised a lot of attention. Since then we here at Fresh Tilled Soil have seen a tremendous amount of startups and companies which have sprung up building products based WebRTC video chat technology. Tsashi Levent-Levi who is a WebRTC evangelist has been interviewing most of these companies on his blog, the list is quite impressive!

    WebRTC chat demo

    Much like most of early adopters we have been playing around with WebRTC for quite awhile now. We have of course created our own WebRTC video chat demo and have also very recently released WebRTC video chat widgets.

    The widgets work very simply, anybody can take the following HTML embed code:

    <!-- Begin Fresh Tilled Soil Video Chat Embed Code -->
    <div id="freshtilledsoil_embed_widget" class="video-chat-widget"></div>
    <script id="fts" src="http://freshtilledsoil.com/embed/webrtc-v5.js?r=FTS0316-CZ6NqG97"></script>
    <!-- End Fresh Tilled Soil Video Chat Embed Code -->

    and add this code to any website or blog post. You’ll see the following widget on their website:

    From here it’s dead simple to start a WebRTC video chat, just make up a name for a room, type it in and click start chat. Tell the other person to do the same and you’re all set.

    As always make sure you’re giving this a try in Firefox Nightly or the latest stable build of Google Chrome. If you are on a tablet make sure you are on Google Chrome beta if you are using the Google Chrome browser.

    Something else to note is that for this first version our video chat is limited to just two participants per a room. If a room name is occupied by two people the third person who tries to connect to this room simply won’t be able to connect.

    How It Works

    Without getting too deep into the code behind how WebRTC video chat actually works, let’s briefly go over what is actually happening behind the scenes when you click the start chat button and how WebRTC video chat actually works. Here is a step by step timeline of what actually happens to give you a better idea:

    A quick note about this step: “Once remote media starts streaming stop adding ICE candidates” – this is a temporary solution which might result in suboptimal media routing for many network topologies. It should only be used until Chrome’s ICE support is fixed.

    A quick and very important tip to remember when you are trying to get this to work. We used a ‘polyfill’ like technique as shown in this article by Remy Sharp. As Remy describes we wrote a piece of code to adapt for the Firefox syntax to get cross-browser functionality.

    Issues We Ran Into and How We Solved Them

    As you might expect we ran into a number of problems and issues trying to build this. WebRTC is evolving quickly so we are working through a number of issues every day. Below are just some of the problems we ran into and how we solved them.

    PeerConnection capability in Google Chrome

    While working with the new PeerConnection capability in Chrome we discovered a strict order of operation for it to work; more specifically:

    • Peers must be present with local streaming video before sending SIP (offer/answer SDP)
    • For ‘Answerer'; Do not add ICE candidate until the peer generates the ‘Answer SDP’
    • Once remote media starts streaming stop adding ICE candidates
    • Never create peer connect for answerer until you get the ‘Offer SDP’

    We fixed it by handling those issues and handling the connection in the order described above. This was crucial to making the connection work flawlessly. Before we did that it would work only every once in a while.

    Added latency due to lag

    When streaming to a mobile device there is added latency due to lag and limitations of surfing the net via mobile phone.

    We solved this by making the resolution of streamed video reduced via a hash tag at the end of the URL. URL can optionally contain '#res=low' for low resolution stream video & '#res=hd' for HiDefinition streaming video as an optional URL parameter. A quick note here that other configurable properties are now available such as frames per second which you can use for this same purpose.

    Recording the WebRTC demo

    We’ve been dabbling with recording video WebRTC demo. When recording video we used the new JavaScript type arrays to save the streaming data. We quickly discovered that it is only possible to record the video and audio separately.

    We solved this by creating two instances of recording, one for the audio and one for the video, that utilized the new javascript data types and recorded both streams simultaneously.

    Conclusion

    It’s exciting to dabble in this stuff, we love WebRTC so much that we created an entire page dedicated to our experiments with this technology and others which we believe will transform the web in 2013. If you have any question please give us a shout.

  10. Firefox OS Simulator 3.0 released

    About 6 weeks ago, we wrote about the preview of Firefox OS Simulator 3.0 and now we’re happy to release it!

    New features

    In our preview coverage, we listed the new features for this version:

    • Push to Device
    • Rotation simulation
    • Basic geolocation API simulation
    • Manifest validation
    • Stability fixes for installation and updates to apps
    • Newer versions of the Firefox rendering engine and Gaia (the UI for Firefox OS)

    Since then, apart from verifying and making the above functionality stable, other notable improvements are:

    • Keyboard shortcut to repackage/reinstall/restart the current app – drastically improving workflow speed, especially for packaged apps
    • Significantly reduced download/installation size of the Firefox OS Simulator
    • Faster start time (partly due to above with smaller size)
    • Much newer versions of Firefox OS and Gaia
    • General bug fixes, making it more functional and reliable
    • OS-standard ways in the menu to close the Simulator, with corresponding keyboard shortcuts (Alt + F4 on Windows, Cmd + Q on Mac, Ctrl + Q on Linux)

    Firefox OS Simulator documentation

    As part of this release, we’d also like to highlight our updated Firefox OS Simulator documentation on MDN and also our Firefox OS Simulator walkthrough example.

    Download/install the Firefox OS Simulator

    You can download version 3.0 of the Firefox OS Simulator from our add-ons web site.

    Please let us know what you think in the comments! If you encounter any bugs or problems, please file a bug.