Mozilla

Featured Articles

Sort by:

View:

  1. WebRTC comes to Firefox

    As we mentioned in the Hacks blog back in April , WebRTC will be on by default in Firefox 22. getUserMedia (gUM) has been on by default since Firefox 20. PeerConnection and DataChannel, which enable video/audio calling and peer-to-peer data sharing, are what’s new in Firefox 22 (due to be released today).

    WebRTC brings real-time communication to the web for the first time ever, and we’re excited to get this new technology into the hands of developers. We believe the industry has only scratched the surface of what’s possible with WebRTC, and only by getting it into the hands of developers and early adopters will we see this technology’s true potential.

    Known issues/limitations

    There are a few known issues/limitations in the early releases:

    • We are initially focused on getting 1:1 calling working well. We’ve done nothing to prevent conference or mesh calling, but depending on the capabilities of your device, video calls with multiple participants may be sluggish. We will be improving multi-person calling in future releases. Our roadmap includes full support for multi-person/conference/mesh calling and we expect to improve the experience in future releases.
    • You may hear echo on calls when you or the party you’re talking to is playing sound over your computer speakers. We’re working on improving echo cancellation but for the time being, try wearing headphones if you experience this problem.
    • On some systems, you may experience audio delay relative to the video. We’ve isolated the problem and are working on a fix for a near-term Firefox release.
    • If you are behind a particularly restrictive NAT or firewall, you may have trouble connecting. We are adding support for media relaying (TURN) in Firefox 23, so you should find this improving soon.

    Trying WebRTC support today

    If you’d like to try out Firefox’s WebRTC support today, here are some sites that support WebRTC calling:

    NOTE: most of these sites support 3 or more callers. We expect basic 1:1 (2-person) calling to perform well enough for developer and early adopter use. As mentioned above, you may find that your mileage may vary with 3-or-more person calling using the current release.

    If you’re a developer interested in embedding WebRTC video chat into your website, please check out article on that.

    Testing DataChannels

    You can also try out DataChannels in Firefox, which is the first browser to launch a spec-compliant implementation of DataChannels to the market. Some sites and projects that use DataChannels:

    Using Firefox Nightly to test the latest

    I still encourage developers to use Firefox Nightly because it has the latest and greatest code and improvements, and we will be continuing to improve existing features and add new ones as we get feedback from developers and users and as the WebRTC standard itself evolves.

    Rapid progress!

    We expect new WebRTC sites, supporting PeerConnection and DataChannels, to come online rapidly over the next several months. We’ll keep you updated on our progress and on WebRTC’s progress here on Mozilla Hacks.

  2. Optimizing your JavaScript game for Firefox OS

    When developing on a quad core processor with 16 gigabytes of RAM you can easily forget to consider how it will perform on a mobile device. This article will detail some best practices and things to consider for moving a game to Firefox OS or any similar hardware target.

    Making the best of 256 Mb RAM/800 Mhz CPU

    There are many areas of focus to keep in mind while developing a game. When your goal is to draw 60 times a second, garbage collection and inefficient drawing calls start to get in your way. Let’s start with the basics…

    Don’t over-optimize

    This might sound counter-intuitive in an article about game optimization but optimization is the last step; performed on complete, working code. While it’s never a bad idea to keep these tips and tricks in mind, you don’t know whether you’ll need them until you’ve finished the game and played it on a device.

    Optimize Drawing

    Drawing on HTML5 2D canvas is the biggest bottleneck in most JavaScript games, as all other updates are usually just algebra without touching the DOM. Canvas operations are hardware accelerated, which can give you some extra room to breath.

    Use whole-pixel rendering

    Sub-pixel rendering occurs when you render objects on a canvas without whole values.

    ctx.drawImage(myImage, 0.3, 0.5)

    This causes the browser to do extra calculations to create the anti-aliasing effect. To avoid this, make sure to round all co-ordinates used in calls to drawImage using Math.floor or as you’ll reader further in the article, bitwse operators.

    jsPerf – drawImage whole pixels.

    Cache drawing in an offscreen canvas

    If you find yourself with complex drawing operations on each frame, consider creating an offscreen canvas, draw to it once (or whenever it changes) on the offscreen canvas, then on each frame draw the offscreen canvas.

    myEntity.offscreenCanvas = document.createElement(“canvas”);
    myEntity.offscreenCanvas.width = myEntity.width;
    myEntity.offscreenCanvas.height = myEntity.height;
    myEntity.offscreenContext = myEntity.offscreenCanvas.getContext(“2d”);
     
    myEntity.render(myEntity.offscreenContext);

    Use moz-opaque on the canvas tag (Firefox Only)

    If your game uses canvas and doesn’t need to be transparent, set the moz-opaque attribute on the canvas tag. This information can be used internally to optimize rendering.

    <canvas id="mycanvas" moz-opaque></canvas>

    Described more in Bug 430906 – Add moz-opaque attribute on canvas.

    Scaling canvas using CSS3 transform

    CSS3 transforms are faster by using the GPU. Best case is to not scale the canvas or have a smaller canvas and scale up rather than a bigger canvas and scale down. For Firefox OS, target 480 x 320 px.

    var scaleX = canvas.width / window.innerWidth;
    var scaleY = canvas.height / window.innerHeight;
     
    var scaleToFit = Math.min(scaleX, scaleY);
    var scaleToCover = Math.max(scaleX, scaleY);
     
    stage.style.transformOrigin = "0 0"; //scale from top left
    stage.style.transform = "scale(" + scaleToFit + ")";

    See it working in this jsFiddle.

    Nearest-neighbour rendering for scaling pixel-art

    Leading on from the last point, if your game is themed with pixel-art, you should use one of the following techniques when scaling the canvas. The default resizing algorithm creates a blurry effect and ruins the beautiful pixels.

    canvas {
      image-rendering: crisp-edges;
      image-rendering: -moz-crisp-edges;
      image-rendering: -webkit-optimize-contrast;
      -ms-interpolation-mode: nearest-neighbor;
    }

    or

    var context = canvas.getContext(‘2d’);
    context.webkitImageSmoothingEnabled = false;
    context.mozImageSmoothingEnabled = false;
    context.imageSmoothingEnabled = false;

    More documentation is available on MDN for image-rendering.

    CSS for large background images

    If like most games you have a static background image, use a plain DIV element with a CSS background property and position it under the canvas. This will avoid drawing a large image to the canvas on every tick.

    Multiple canvases for layers

    Similar to the last point, you may find you have some elements that are frequently changing and moving around whereas other things (like UI) never change. An optimization in this situation is to create layers using multiple canvas elements.

    For example you could create a UI layer that sits on top of everything and is only drawn during user input. You could create game layer where the frequently updating entities exist and a background layer for entities that rarely update.

    Don’t scale images in drawImage

    Cache various sizes of your images on an offscreen canvas when loading as opposed to constantly scaling them in drawImage.

    jsPerf – Canvas drawImage Scaling Performance.

    Be careful with heavy physics libraries

    If possible, roll your own physics as libraries like Box2D don’t perform well on low-end Firefox OS devices.

    When asm.js support lands in Firefox OS, Emscripten-compiled libraries can take advantage of near-native performance. More reading in Box2d Revisited.

    Use WebGL instead of Context 2D

    Easier said than done but giving all the heavy graphics lifting to the GPU will free up the CPU for greater good. Even though WebGL is 3D, you can use it to draw 2D surfaces. There are some libraries out there that aim to abstract the drawing contexts.

    Minimize Garbage Collection

    JavaScript can spoil us when it comes to memory management. We generally don’t need to worry about memory leaks or conservatively allocating memory. But if we’ve allocated too much and garbage collection occurs in the middle of a frame, that can take up valuable time and result in a visible drop in FPS.

    Pool common objects and classes

    To minimise the amount of objects being cleaned during garbage collection, use a pre-initialised pool of objects and reuse them rather than creating new objects all the time.

    Code example of generic object pool:

    Avoid internal methods creating garbage

    There are various JavaScript methods that create new objects rather than modifying the existing one. This includes: Array.slice, Array.splice, Function.bind.

    Read more about JavaScript garbage collection

    Avoid frequent calls to localStorage

    LocalStorage uses file IO and blocks the main thread to retrieve and save data. Use an in-memory object to cache the values of localStorage and even save writes for when the player is not mid-game.

    Code example of an abstract storage object:

    Async localStorage API with IndexedDB

    IndexedDB is a non-blocking API for storing data on the client but may be overkill for small and simple data. Gaia’s library to make localStorage API asynchronous, using IndexedDB is available on Github: async_storage.js.

    Miscellaneous micro-optimization

    Sometimes when you’ve exhausted all your options and it just won’t go any faster, you can try some micro-optimizations below. However do note that these only start to make a difference in heavy usage when every millisecond counts. Look for them in your hot game loops.

    Use x | 0 instead of Math.floor
    Clear arrays with .length = 0 to avoid creating a new Array
    Sacrifice some CPU time to avoid creating garbage.
    Use if .. else over switch
    jsPerf – switch vs if-else
    Date.now() over (+ new Date)
    jsPerf – Date.now vs new Date().getTime() vs +new Date
    Or performance.now() for a sub-millisecond solution
    Use TypedArrays for floats or integers (e.g. vectors and matrices)
    gl-matrix – Javascript Matrix and Vector library for High Performance WebGL apps

    Conclusion

    Building for mobile devices and not-so-strong hardware is a good and creative exercise, and we hope you will want to make sure your games work well on all platforms!

  3. Firefox Developer Tool Features for Firefox 23

    Another uplift has left the building and it’s time to take a look at what’s in Firefox Developer Tools in Firefox 23 currently Aurora, our pre-beta channel. You can download it from the Aurora Download page today. Firefox 23 is currently scheduled to hit the release channel on Tuesday August 6th, 2013.

    Episode XXIII is a barn-storming, hair-raising spectacle of incredible epicness that is sure to delight and amuse. If you want a sneak peak at features under active development, give Nightly a try.

    Network Monitor

    There’s a new tool in the toolbox: The Network Monitor. It’s a classic waterfall timeline view of network activity on a site. This data’s been available since Firefox 4 via the Web Console, albeit in a less visually pleasing way.

    Please file bugs under the Developer Tools: Netmonitor component in Bugzilla.

    Remote Style Editor

    In Firefox 23, you can now edit styles via a Remote Connection on a suitably-enabled device. This should be great help for App Developers interested in testing and debugging styles on a mobile device over the remote protocol in real time.

    As of the time of this writing, the Remote Style Editor should be compatible with Firefox for Android version 23, also scheduled for uplift to Aurora. We are working on incorporating the Style Editor Actors for the remote protocol into the Firefox OS Simulator and investigating what it will take to backport them to Firefox OS release.

    Options Panel

    We’ve added a Gear menu to the Toolbar containing an Options panel for turning tools on or off. As we add more stuff, this is going to be a popular place to manage preferences related to the Developer Tools.

    Currently, there are options to turn on the Light or Dark theme for the tools and enable Chrome Debugging.

    Initial SourceMap Support for Debugger Protocol

    The first pieces of SourceMap support for the Debugger have landed and we are now able to serve up SourceMapped JS files for your debugging pleasure. Soon to follow will be column support for breakpoints allowing you to debug minified JS with a SourceMap.

    Watch for the upcoming blog post by Nick Fitzgerald on Hacks explaining the magic.

    Variables View Everywhere

    Our Variables View is an improved Object Inspector and an integral part of our Debugger. Naturally, we wanted to put it in Everything. So now the Web Console and Scratchpad have a Variables View. Use the ESC key to close it.

    Browser Console

    If you have Chrome Debugging turned on, check out the Browser Console. It’s a replacement for the ancient Error Console and gives you a Chrome-context command line for executing JavaScript against the browser. It’s nice and should be enabled by default in Firefox 24.

    GCLI Appcache Command

    We finally have a little something for developers trying to use App Cache to store offline data. A new appcache command for the Graphical Command Line. You can read about it in Mike Ratcliffe’s The Application Cache is no longer a Douchebag.

    Web Console in Debugger Frame

    The Web Console is now fully-remoted (and has been since version 18). It now makes use of the Debugger’s current Frame if paused on a breakpoint.

    Multiple Paused Debuggers

    You can now debug multiple tabs at the same time using the Script Debugger. Previously, when attempting to use the debugger on two separate tabs, you’d be given a notification to resume the debugger in the other tab. Now you can debug as many tabs as you like.

    There is one caveat to this awesome power, however. Due to the nested event loops each Debugger creates, you have to resume each tab in the order in which they were paused. Debug carefully and always carry a big stack.

    You can see a comprehensive list of bugfixes in table form in Firefox 23 Developer Tools Fixes.

  4. Embedding WebRTC Video Chat Right Into Your Website

    Most of you remember the Hello Chrome, it’s Firefox calling! blog post right here in Mozilla Hacks demonstrating WebRTC video chat between Firefox and Chrome. It raised a lot of attention. Since then we here at Fresh Tilled Soil have seen a tremendous amount of startups and companies which have sprung up building products based WebRTC video chat technology. Tsashi Levent-Levi who is a WebRTC evangelist has been interviewing most of these companies on his blog, the list is quite impressive!

    WebRTC chat demo

    Much like most of early adopters we have been playing around with WebRTC for quite awhile now. We have of course created our own WebRTC video chat demo and have also very recently released WebRTC video chat widgets.

    The widgets work very simply, anybody can take the following HTML embed code:

    <!-- Begin Fresh Tilled Soil Video Chat Embed Code -->
    <div id="freshtilledsoil_embed_widget" class="video-chat-widget"></div>
    <script id="fts" src="http://freshtilledsoil.com/embed/webrtc-v5.js?r=FTS0316-CZ6NqG97"></script>
    <!-- End Fresh Tilled Soil Video Chat Embed Code -->

    and add this code to any website or blog post. You’ll see the following widget on their website:

    From here it’s dead simple to start a WebRTC video chat, just make up a name for a room, type it in and click start chat. Tell the other person to do the same and you’re all set.

    As always make sure you’re giving this a try in Firefox Nightly or the latest stable build of Google Chrome. If you are on a tablet make sure you are on Google Chrome beta if you are using the Google Chrome browser.

    Something else to note is that for this first version our video chat is limited to just two participants per a room. If a room name is occupied by two people the third person who tries to connect to this room simply won’t be able to connect.

    How It Works

    Without getting too deep into the code behind how WebRTC video chat actually works, let’s briefly go over what is actually happening behind the scenes when you click the start chat button and how WebRTC video chat actually works. Here is a step by step timeline of what actually happens to give you a better idea:

    A quick note about this step: “Once remote media starts streaming stop adding ICE candidates” – this is a temporary solution which might result in suboptimal media routing for many network topologies. It should only be used until Chrome’s ICE support is fixed.

    A quick and very important tip to remember when you are trying to get this to work. We used a ‘polyfill’ like technique as shown in this article by Remy Sharp. As Remy describes we wrote a piece of code to adapt for the Firefox syntax to get cross-browser functionality.

    Issues We Ran Into and How We Solved Them

    As you might expect we ran into a number of problems and issues trying to build this. WebRTC is evolving quickly so we are working through a number of issues every day. Below are just some of the problems we ran into and how we solved them.

    PeerConnection capability in Google Chrome

    While working with the new PeerConnection capability in Chrome we discovered a strict order of operation for it to work; more specifically:

    • Peers must be present with local streaming video before sending SIP (offer/answer SDP)
    • For ‘Answerer’; Do not add ICE candidate until the peer generates the ‘Answer SDP’
    • Once remote media starts streaming stop adding ICE candidates
    • Never create peer connect for answerer until you get the ‘Offer SDP’

    We fixed it by handling those issues and handling the connection in the order described above. This was crucial to making the connection work flawlessly. Before we did that it would work only every once in a while.

    Added latency due to lag

    When streaming to a mobile device there is added latency due to lag and limitations of surfing the net via mobile phone.

    We solved this by making the resolution of streamed video reduced via a hash tag at the end of the URL. URL can optionally contain '#res=low' for low resolution stream video & '#res=hd' for HiDefinition streaming video as an optional URL parameter. A quick note here that other configurable properties are now available such as frames per second which you can use for this same purpose.

    Recording the WebRTC demo

    We’ve been dabbling with recording video WebRTC demo. When recording video we used the new JavaScript type arrays to save the streaming data. We quickly discovered that it is only possible to record the video and audio separately.

    We solved this by creating two instances of recording, one for the audio and one for the video, that utilized the new javascript data types and recorded both streams simultaneously.

    Conclusion

    It’s exciting to dabble in this stuff, we love WebRTC so much that we created an entire page dedicated to our experiments with this technology and others which we believe will transform the web in 2013. If you have any question please give us a shout.

  5. Firefox OS Simulator 3.0 released

    About 6 weeks ago, we wrote about the preview of Firefox OS Simulator 3.0 and now we’re happy to release it!

    New features

    In our preview coverage, we listed the new features for this version:

    • Push to Device
    • Rotation simulation
    • Basic geolocation API simulation
    • Manifest validation
    • Stability fixes for installation and updates to apps
    • Newer versions of the Firefox rendering engine and Gaia (the UI for Firefox OS)

    Since then, apart from verifying and making the above functionality stable, other notable improvements are:

    • Keyboard shortcut to repackage/reinstall/restart the current app – drastically improving workflow speed, especially for packaged apps
    • Significantly reduced download/installation size of the Firefox OS Simulator
    • Faster start time (partly due to above with smaller size)
    • Much newer versions of Firefox OS and Gaia
    • General bug fixes, making it more functional and reliable
    • OS-standard ways in the menu to close the Simulator, with corresponding keyboard shortcuts (Alt + F4 on Windows, Cmd + Q on Mac, Ctrl + Q on Linux)

    Firefox OS Simulator documentation

    As part of this release, we’d also like to highlight our updated Firefox OS Simulator documentation on MDN and also our Firefox OS Simulator walkthrough example.

    Download/install the Firefox OS Simulator

    You can download version 3.0 of the Firefox OS Simulator from our add-ons web site.

    Please let us know what you think in the comments! If you encounter any bugs or problems, please file a bug.

  6. WebRTC Update: Our first implementation will be in release soon. Welcome to the Party! But Please Watch Your Head.

    I want to share some useful and exciting updates on Firefox’s WebRTC implementation and provide a sneak peak at some of our plans for WebRTC moving forward. I’ll then ask Adam Roach, who has worked in the VoIP/SIP space on IETF standards for over a decade and who joined the Mozilla WebRTC in November, to provide some historical background on the WebRTC feature itself and how things are progressing in general.

    Getting ready to release our first implementation of WebRTC on Desktop

    Firefox has made significant progress with our initial implementation of WebRTC on Desktop in terms of security and stability. For the last several weeks, PeerConnection and DataChannels have been pref’d on by default in Nightly and Aurora, and we expect to keep them pref’d on for all release phases of Firefox 22 (which means we expect WebRTC will go to Aurora, Beta and General Release).

    We also got a chance to update the DataChannels implementation in Firefox 22 to match the recent spec changes (agreed to at IETF last month). Note: the changes we needed to make to comply with the spec changes are not backwards compatible with previous DataChannels implementations (in Firefox 21 or earlier). So please use Firefox 22 and later for testing your DataChannels apps.

    TURN support has landed

    And there’s more good news. We just added TURN support to Firefox Nightly and are in the process of testing it. This is a big deal since TURN will increase the likelihood that a call will successfully connect, regardless of the types of NATs that the end points are behind.

    TURN (Traversal Using Relays behind NATs) is a standard for managing (allocating, using, and destroying) a relay session on a remote external server. This relay session enables WebRTC to connect when the NATs at both ends would otherwise cause the call to fail. Because TURN can introduce delay, especially if the TURN server is remote to both endpoints, and TURN servers can be expensive (because it has to handle all the media flows during a call), ICE typically uses TURN only when other methods (like STUN) to get media flowing during a call fail to work.

    WebRTC on Firefox for Android is Ready for Experimentation and Feedback

    Back in February at Mobile World Congress we showed exciting, new demos of WebRTC calls on Firefox for Android. We have just finished landing this code in Nightly. The code for both getUserMedia (gUM) and PeerConnection is behind a pref (as Desktop was initially), but you can enable it by setting both the media.navigator.enabled pref and the media.peerconnection.enabled pref to “true” (browse to about:config and search for media.navigator.enabled and media.peerconnection.enabled in the list of prefs).

    In the same list of prefs, you can also set media.navigator.permission.disabled to “true” to automatically give permission to access the camera/microphone and bypass the permission/selection dialog when testing gUM and WebRTC.

    This code is still in the early phases of testing, but we encourage you to try it, report back problems, and ask questions and provide feedback. Please be sure to mention “Android” if you are reporting a problem or providing feedback since we are tracking both Android and Desktop issues now. Like with Desktop, we will be working to improve, stabilize and expand this code over the next year.

    WebRTC on Firefox OS Coming Next

    Mozilla is also working hard to get this code running on Firefox OS. WebRTC on Firefox OS is not yet as far along as WebRTC on Firefox for Android, but the Firefox OS team is working to close the gap. You can follow Bug 750011 to track this work.

    Future WebRTC Features

    Since all our products share the gecko engine, improvements and new features made to core “platform” or gecko code typically benefit all our products. Over the next year we plan to make the following improvements:

    • Complete error and state reporting (spec compliant)
    • Recording API support
    • Persona integration
    • Multiple audio and video flows per Peer Connection (beyond 1 video flow and 1 audio flow)
    • Persistent permissions support (in the UX/UI)
    • AEC improvements
    • Improved call quality (especially audio latency)

    We hope to do even more, but these are on the top of our list. We also have a WebRTC documentation and help page that we’re working to fill out over the next few weeks and then keep up-to-date. That will have links to what we’ve implemented and what we’re working on, as well as ways for contributors to get involved.

    Moving Forward

    Although we can’t wait to release our first implementation of WebRTC on Desktop (which is a huge milestone for Mozilla and the WebRTC feature itself), I am still encouraging all developers experimenting with WebRTC to continue to use Nightly for the foreseeable future because that is where the latest and greatest features from the spec land and bugs get fixed first — and that is where your feedback will be most helpful.

    And even more importantly, the WebRTC spec itself is still being standardized. For more details on that, as well as some behind-the-scenes history on WebRTC, I hand the rest of this article off to Adam.

    -Maire Reavy, Product Manager, Mozilla’s Media Platform Team

    WebRTC is the Real Future of Communications

    This is Adam.

    About three years ago, my dear friend and VoIP visionary Henry Sinnreich spent some time over lunch trying to convince me that the real future of communications lay in the ability to make voice and video calls directly from the ubiquitous web browser. I can still envision him enthusiastically waving his smartphone around, emphasizing how pervasive web browsers had become. My response was that his proposal would require unprecedented cooperation between the IETF and W3C to make happen, and that it would demand a huge effort and commitment from the major browser vendors. In short: it’s a beautiful vision, but Herculean in scope.

    Then, something amazing happened.

    WebRTC sees the light of day

    Over the course of 2011, the groundwork for exactly such IETF/W3C collaboration was put in place, and a broad technical framework was designed. During 2012, Google and Mozilla began work in earnest to implement towards the developing standard.

    Last November, San Francisco hosted the first WebRTC expo. The opening keynote was packed to capacity, standing room only, with people spilling out into the hallway. During the following two days, we saw countless demos of nascent WebRTC services, and saw dozens of companies committed to working with the WebRTC ecosystem. David Jodoin shared with us the staggering fact that half of the ten largest US banks are already planning their WebRTC strategy.

    And in February, Mozilla and Google drove the golden spike into the WebRTC railroad by demonstrating a real time video call between Firefox and Chrome.

    So Where Are We?

    With that milestone, it’s tempting to view WebRTC as “almost done,” and easy to imagine that we’re just sanding down the rough edges right now. As much as I’d love that to be the case, there’s still a lot of work to be done.

    Last February in Boston, we had a joint interim meeting for the various standards working groups who are contributing to the WebRTC effort. Topics included issues ranging from the calling conventions of the WebRTC javascript APIs to the structure of how to signal multiple video streams – things that will be important for wide adoption of the standard. I’m not saying that the WebRTC standards effort is struggling. Having spent the past 16 years working on standards, I’m can assure you that this pace of development is perfectly normal and expected for a technology this ambitious. What I am saying is that the specification of something this big, something this important, and something with this many stakeholders takes a long time.

    Current state of standards

    Even if the standards work were complete today, the magnitude of what WebRTC is doing will take a long time to get implemented, to get debugged, to get right. Our golden spike interop moment took substantial work on both sides, and revealed a lot of shortcomings in both implementations. Last February also marked the advent of SIPit 30, which included the first actual WebRTC interop testing event. This testing predictably turned up several new bugs (both in our implementation as well as others’), on top of those limitations that we already knew about.

    When you add in all the features that I know neither Mozilla nor Google has begun work on, all the features that aren’t even specified yet, there’s easily a year of work left before we can start putting the polish on WebRTC.

    We’re furiously building the future of communications on the Internet, and it’s difficult not to be excited by the opportunities afforded by this technology. I couldn’t be more pleased by the warm reception that WebRTC has received. But we all need to keep in mind that this is still very much a work in progress.

    Come and play! But please watch your head.

    So, please, come in, look around, and play around with what we’re doing. But don’t expect everything to be sleek and finished yet. While we are doing our best to limit how the changing standards impact application developers and users, there will be inevitable changes as the specifications evolve and as we learn more about what works best. We’ll keep you up to date with those changes here on the Hacks blog and try to minimize their impact, but I fully expect application developers to have to make tweaks and adjustments as the platform evolves. Expect us to take us a few versions to get voice and video quality to a point that we’re all actually happy about. Most importantly, understand that no one’s implementation is going to completely match the rapidly evolving W3C specifications for quite a while.

    I’m sure we all want 2013 to be “The Year of WebRTC,” as some have already crowned it. And for early adopters, this is absolutely the time to be playing around with what’s possible, figuring out what doesn’t quite work the way you expect, and — above all — providing feedback to us so we can improve our implementation and improve the developing standards.

    As long as you’re in a position to deal with minor disruptions and changes; if you can handle things not quite working as described; if you are ready to roll up your sleeves and influence the direction WebRTC is going, then we’re ready for you. Bring your hard hat, and keep the lines of communication open.

    For those of you looking to deploy paid services, reliable channels to manage your customer relationships, mission critical applications: we want your feedback too, but temper your launch plans. I expect that we’ll have a stable platform that’s well and truly open for business some time next year.

    Credits: Original hardhat image from openclipart.org; Anthony Wing Kosner first applied the “golden spike” analogy to WebRTC interop.

  7. Detecting touch: it’s the ‘why’, not the ‘how’

    One common aspect of making a website or application “mobile friendly” is the inclusion of tweaks, additional functionality or interface elements that are particularly aimed at touchscreens. A very common question from developers is now “How can I detect a touch-capable device?”

    Feature detection for touch

    Although there used to be a few incompatibilities and proprietary solutions in the past (such as Mozilla’s experimental, vendor-prefixed event model), almost all browsers now implement the same Touch Events model (based on a solution first introduced by Apple for iOS Safari, which subsequently was adopted by other browsers and retrospectively turned into a W3C draft specification).

    As a result, being able to programmatically detect whether or not a particular browser supports touch interactions involves a very simple feature detection:

    if ('ontouchstart' in window) {
      /* browser with Touch Events
         running on touch-capable device */
    }

    This snippet works reliably in modern browser, but older versions notoriously had a few quirks and inconsistencies which required jumping through various different detection strategy hoops. If your application is targetting these older browsers, I’d recommend having a look at Modernizr – and in particular its various touch test approaches – which smooths over most of these issues.

    I noted above that “almost all browsers” support this touch event model. The big exception here is Internet Explorer. While up to IE9 there was no support for any low-level touch interaction, IE10 introduced support for Microsoft’s own Pointer Events. This event model – which has since been submitted for W3C standardisation – unifies “pointer” devices (mouse, stylus, touch, etc) under a single new class of events. As this model does not, by design, include any separate ‘touch’, the feature detection for ontouchstart will naturally not work. The suggested method of detecting if a browser using Pointer Events is running on a touch-enabled device instead involves checking for the existence and return value of navigator.maxTouchPoints (note that Microsoft’s Pointer Events are currently still vendor-prefixed, so in practice we’ll be looking for navigator.msMaxTouchPoints). If the property exists and returns a value greater than 0, we have touch support.

    if (navigator.msMaxTouchPoints > 0) {
      /* IE with pointer events running
         on touch-capable device */
    }

    Adding this to our previous feature detect – and also including the non-vendor-prefixed version of the Pointer Events one for future compatibility – we get a still reasonably compact code snippet:

    if (('ontouchstart' in window) ||
         (navigator.maxTouchPoints > 0) ||
         (navigator.msMaxTouchPoints > 0)) {
          /* browser with either Touch Events of Pointer Events
             running on touch-capable device */
    }

    How touch detection is used

    Now, there are already quite a few commonly-used techniques for “touch optimisation” which take advantage of these sorts of feature detects. The most common use cases for detecting touch is to increase the responsiveness of an interface for touch users.

    When using a touchscreen interface, browsers introduce an artificial delay (in the range of about 300ms) between a touch action – such as tapping a link or a button – and the time the actual click event is being fired.

    More specifically, in browsers that support Touch Events the delay happens between touchend and the simulated mouse events that these browser also fire for compatibility with mouse-centric scripts:

    touchstart > [touchmove]+ > touchend > delay > mousemove > mousedown > mouseup > click

    See the event listener test page to see the order in which events are being fired, code available on GitHub.

    This delay has been introduced to allow users to double-tap (for instance, to zoom in/out of a page) without accidentally activating any page elements.

    It’s interesting to note that Firefox and Chrome on Android have removed this delay for pages with a fixed, non-zoomable viewport.

    <meta name="viewport" value="... user-scalable = no ...">

    See the event listener with user-scalable=no test page, code available on GitHub.

    There is some discussion of tweaking Chrome’s behavior further for other situations – see issue 169642 in the Chromium bug tracker.

    Although this affordance is clearly necessary, it can make a web app feel slightly laggy and unresponsive. One common trick has been to check for touch support and, if present, react directly to a touch event (either touchstart – as soon as the user touches the screen – or touchend – after the user has lifted their finger) instead of the traditional click:

    /* if touch supported, listen to 'touchend', otherwise 'click' */
    var clickEvent = ('ontouchstart' in window ? 'touchend' : 'click');
    blah.addEventListener(clickEvent, function() { ... });

    Although this type of optimisation is now widely used, it is based on a logical fallacy which is now starting to become more apparent.

    The artificial delay is also present in browsers that use Pointer Events.

    pointerover > mouseover > pointerdown > mousedown > pointermove > mousemove > pointerup > mouseup > pointerout > mouseout > delay > click

    Although it’s possible to extend the above optimisation approach to check navigator.maxTouchPoints and to then hook up our listener to pointerup rather than click, there is a much simpler way: setting the touch-action CSS property of our element to none eliminates the delay.

    /* suppress default touch action like double-tap zoom */
    a, button {
      -ms-touch-action: none;
          touch-action: none;
    }

    See the event listener with touch-action:none test page, code available on GitHub.

    False assumptions

    It’s important to note that these types of optimisations based on the availability of touch have a fundamental flaw: they make assumptions about user behavior based on device capabilities. More explicitly, the example above assumes that because a device is capable of touch input, a user will in fact use touch as the only way to interact with it.

    This assumption probably held some truth a few years back, when the only devices that featured touch input were the classic “mobile” and “tablet”. Here, touchscreens were the only input method available. In recent months, though, we’ve seen a whole new class of devices which feature both a traditional laptop/desktop form factor (including a mouse, trackpad, keyboard) and a touchscreen, such as the various Windows 8 machines or Google’s Chromebook Pixel.

    As an aside, even in the case of mobile phones or tablets, it was already possible – on some platforms – for users to add further input devices. While iOS only caters for pairing an additional bluetooth keyboard to an iPhone/iPad purely for text input, Android and Blackberry OS also let users add a mouse.

    On Android, this mouse will act exactly like a “touch”, even firing the same sequence of touch events and simulated mouse events, including the dreaded delay in between – so optimisations like our example above will still work fine. Blackberry OS, however, purely fires mouse events, leading to the same sort of problem outlined below.

    The implications of this change are slowly beginning to dawn on developers: that touch support does not necessarily mean “mobile” anymore, and more importantly that even if touch is available, it may not be the primary or exclusive input method that a user chooses. In fact, a user may even transition between any of their available input methods in the course of their interaction.

    The innocent code snippets above can have quite annoying consequences on this new class of devices. In browsers that use Touch Events:

    var clickEvent = ('ontouchstart' in window ? 'touchend' : 'click');

    is basically saying “if the device support touch, only listen to touchend and not click” – which, on a multi-input device, immediately shuts out any interaction via mouse, trackpad or keyboard.

    Touch or mouse?

    So what’s the solution to this new conundrum of touch-capable devices that may also have other input methods? While some developers have started to look at complementing a touch feature detection with additional user agent sniffing, I believe that the answer – as in so many other cases in web development – is to accept that we can’t fully detect or control how our users will interact with our web sites and applications, and to be input-agnostic. Instead of making assumptions, our code should cater for all eventualities. Specifically, instead of making the decision about whether to react to click or touchend/touchstart mutually exclusive, these should all be taken into consideration as complementary.

    Certainly, this may involve a bit more code, but the end result will be that our application will work for the largest number of users. One approach, already familiar to developers who’ve strived to make their mouse-specific interfaces also work for keyboard users, would be to simply “double up” your event listeners (while taking care to prevent the functionality from firing twice by stopping the simulated mouse events that are fired following the touch events):

    blah.addEventListener('touchend', function(e) {
      /* prevent delay and simulated mouse events */
      e.preventDefault();
      someFunction()
    });
    blah.addEventListener('click', someFunction);

    If this isn’t DRY enough for you, there are of course fancier approaches, such as only defining your functions for click and then bypassing the dreaded delay by explicitly firing that handler:

    blah.addEventListener('touchend', function(e) {
      /* prevent delay and simulated mouse events */
      e.preventDefault();
      /* trigger the actual behavior we bound to the 'click' event */
      e.target.click();
    })
    blah.addEventListener('click', function() {
      /* actual functionality */
    });

    That last snippet does not cover all possible scenarios though. For a more robust implementation of the same principle, see the FastClick script from FT labs.

    Being input-agnostic

    Of course, battling with delay on touch devices is not the only reason why developers want to check for touch capabilities. Current discussions – such as this issue in Modernizr about detecting a mouse user – now revolve around offering completely different interfaces to touch users, compared to mouse or keyboard, and whether or not a particular browser/device supports things like hovering. And even beyond JavaScript, similar concepts (pointer and hover media features) are being proposed for Media Queries Level 4. But the principle is still the same: as there are now common multi-input devices, it’s not straightforward (and in many cases, impossible) anymore to determine if a user is on a device that exclusively supports touch.

    The more generic approach taken in Microsoft’s Pointer Events specification – which is already being scheduled for implementation in other browser such as Chrome – is a step in the right direction (though it still requires extra handling for keyboard users). In the meantime, developers should be careful not to draw the wrong conclusions from touch support detection and avoid unwittingly locking out a growing number of potential multi-input users.

    Further links

  8. Introducing navigator.mozPay() For Web Payments

    What’s wrong with payments on the web? Any website can already host a shopping cart and take credit card payments or something similar. The freedom of today’s web supports many business models. Here’s what’s wrong:

    • Users cannot choose how to pay; they have to select from one of the pre-defined options.
    • In most cases, the user has to type in an actual credit card number on each site. This is like giving someone the keys to your expensive car, letting them drive it around the block in a potentially dangerous neighborhood (the web) and saying please don’t get carjacked!
    • Merchants typically have to manage all this on their own: payment processor setup, costly processing fees, and possibly even PCI compliance.

    There are services to mitigate a lot of these complications such as PayPal, Stripe, and others but they aren’t integrated into web devices very well. Mozilla wants to introduce a common web API to make payments easy and secure on web devices yet still as flexible as the checkout button for merchants.

    As a first step, Mozilla will introduce navigator.mozPay() in Firefox OS so that web apps can accept payments.

    How Does It Work?

    navigator.mozPay() is a JavaScript API inspired by google.payments.inapp.buy() but modified for things like multiple payment providers and carrier billing. When a web app invokes navigator.mozPay() in Firefox OS, the device shows a secure window with a concise UI. After authenticating, the user can easily charge the payment to her mobile carrier bill or credit card. When completed, the app delivers the product. Repeat purchases are quick and easy.

    In an earlier article I talked about purchasing an app and receiving a receipt. navigator.mozPay() is different in that there is no concept of what product is purchased, it’s just an API to facilitate a payment for a digital good or service, whatever that may be.

    The payment starts and finishes in the client but further processing and notifications happen server side. This article briefly explains how it all fits together. For complete, in-depth documentation read the Firefox Marketplace guide to in-app payments.

    Integrating With A Payment Provider

    Multiple providers will facilitate payments behind the scenes of navigator.mozPay(). For example, the Firefox Marketplace will be able to facilitate payments.

    As a developer you will essentially grant permission to each provider that you would like to sell through. In the current design of the API, you do this by asking each provider for an Application Key and an Application Secret so that you can digitally sign payment requests. A signed request prevents unauthorized parties from selling your products and prevents users from tampering with the price, etc.

    Initiating A Payment

    When the user clicks a Buy button in your web app, you create a JSON Web Token (JWT) with your Application Secret. If you have agreements with multiple providers, you would create a JWT for each provider. Starting a payment looks roughly like this:

    document.querySelector('button.buy').onclick = function() {
        navigator.mozPay([mozillaJWT, otherJWT, ...]);
    };

    Defining A Product

    A JWT is a signed JSON object that defines details like the product name and price. Here is an example with some attributes removed for brevity:

    {
      "iss": APPLICATION_KEY,
      "aud": "marketplace.firefox.com",
      ...
      "request": {
        "name": "Magical Unicorn",
        "pricePoint": 1,
        "postbackURL": "https://yourapp.com/postback",
        "chargebackURL": "https://yourapp.com/chargeback"
      }
    }

    You define prices as price points so that the payment provider can handle currency conversions for you in each region. In this example, pricePoint 1 might be €0.89 or $0.99, etc. Micropayments in small amounts will be supported. Consult the navigator.mozPay() spec for details on how to construct a payment request JWT.

    Completing a Payment

    To complete the payment, you need to wait for the provider to POST a result to your server’s postbackURL (on success) or chargebackURL (on failure). A more complete example of requesting a purchase in JavaScript would involve waiting for the postback to arrive, like this:

    var request = navigator.mozPay([mozillaJWT, otherJWT]);
    request.onsuccess = function() {
      // The payment window has closed.
      whenPaymentResultReceived(function() {
        console.log('Success! User has purchased a Magical Unicorn.');
      });
    };

    To implement whenPaymentResultReceived() you might open a web socket to your server, wait for the payment result, and verify the incoming JWT signature. The navigator.mozPay() spec has details on how postback and chargeback notifications work.

    Try It Out

    Payments aren’t fully live yet in the Firefox Marketplace but you can simulate a payment to test out your code. Log into the Firefox Marketplace Developer Hub and generate an Application Key and Application Secret for simulations. With those keys you can add a special parameter to the JWT like this:

    {
      "request": {
        "name": "Magical Unicorn",
        "pricePoint": 1,
        ...
        "simulate": {"result": "postback"}
      }
    }

    This will show a payment window on Firefox OS but it won’t charge you real money. It will let you test your client side JavaScript code and your server postback handlers to make sure everything is integrated smoothly. When you go live, just remove the simulate attribute. If you’re new to Firefox OS development, check out the Firefox OS Simulator.

    If you’re already working on a game or a web app for Firefox OS try thinking about using navigator.mozPay() to offer premium content.

    This Would Be Way Easier With A Library

    We thought so too! We built libraries for Node.JS and Python to make the server side logic for navigator.mozPay() as easy as possible. Libraries for more languages are on the way. We also are experimenting with removing the server prerequisite entirely.

    Current Status

    As you can probably tell by the prefix, navigator.mozPay() is an experimental API and might change drastically or become unprefixed without notice. It will process live payments on the first Firefox OS phones and evolve quickly from real world usage.

    Mozilla plans to work with other vendors through the W3C to reach consensus on a common API that supports web payments in the best way possible. After shipping in Firefox OS, Mozilla plans to add navigator.mozPay() to Firefox for Android and desktop Firefox.

    New Business Models

    Advertising has been the primary business model on the web for a long time but users have made it clear that they don’t want to see ads. Mozilla isn’t trying to directly disrupt the ad business but it is trying to fix serious privacy issues relating to ad networks.

    What if users explicitly paid for content instead? navigator.mozPay() enables this kind of direct payment model: if something is good on the web, you can pay for it. It already seems to be working well for existing mobile apps. Will mobile ads even generate the same revenue for content producers as they do on desktop? I don’t have answers to these questions but one thing is for certain: the web should support businesses of all kinds and payments should be a first class feature of the web.

    Is It Webby?

    Mozilla’s main goal with navigator.mozPay() is to give users and merchants choice, security, and an easy to use payments system. The details about how merchants interact with payment providers is not yet specified in the API and that is clearly a gap. The first Firefox OS phones will ship with a whitelist of allowed payment providers which is also not ideal.

    In a more webby model, all parties involved in the payment would be fully decentralized so that innovation can occur naturally and unknown payment providers could emerge. Who will be the next M-Pesa? An elegant API would support that. Building a secure decentralized payment API is an ambitious challenge; the solution would need to address these core trust issues:

    • How can customers trust that they will receive the goods after paying?
    • How would customers ensure that their payment credentials are handled securely?
    • How do merchants guarantee that they’ll get paid after delivering goods?

    As with anything related to money, there is incentive for fraud every step of the way. BitCoin is a digital currency that solves some of these trust issues with block chains and PaySwarm is a web payment protocol that solves some of these issues with decentralized assets, public keys, etc. Mozilla will be watching PaySwarm as well as other models and hopefully navigator.mozPay() can incorporate some of these concepts eventually.

  9. How to install packaged apps in Firefox OS – options and tools

    I thought this would be a good time to show the options for developers how to install packaged apps, and as an extension to that, installing them on an actual Firefox OS device (more on Open Web apps, if it’s new to you).

    Why a packaged app?

    First and foremost, a packaged app is where all resources in your app is zipped into one file, that gets installed directly on the device. That means that all the files are available directly, without needing an Internet connection.

    The other benefit is that if you need to access some privileged APIs for your app, it needs to be a packaged app to get access to those (for security reasons, since code hosted anywhere on the web naturally can’t have access to sensitive APIs on the device).

    The different APIs are outlined in the Using WebAPIs to make the web layer more capable article and the permissions needed for each API are listed in App permissions.

    Testing in the Firefox OS Simulator

    In the Firefox OS Simulator – latest features listed in previewing version 3.0 – you have the option to install your app as a packaged app.

    In the Simulator Dashboard, you have a button with the label Add Directory. When you do that, and point to the manifest.webapp for your app, it automatically gets installed as a packaged app. Simple as that!

    image

    Not all APIs are supported in the Firefox OS Simulator, but it’s a good start!

    When you push the update button, the app should be refreshed and restarted if already running, and started if not running. If the content or the manifest is not updated after an “update” click, there’s a blocking error, shown in the Simulator Dashboard.

    Push to Device in the Simulator

    As we described here before, one of the new features in version 3.0 of the Firefox OS Simulator is the feature Push to Device.

    If you have Firefox OS device connected when you use the Firefox OS Simulator, you will get a Push button next to each app in the Simulator Dashboard to push it directly to your app.

    image

    You will then need to accept that incoming request on the device to install the app.

    image

    Manual steps

    • Once you’ve pushed the app to the device, you need to manually close and restart it again, to get updated content
    • If you update anything in the manifest, e.g. app name, orientation, you need to reboot the operating system for those changes to have effect

    Things to think about

    • Remote debugging has to be enabled on the device, via
      Settings > Device information > More Information > Developer > Remote debugging
    • For Windows support, make sure to install the necessary drivers and that you use the latest version of the Firefox OS Simulator (currently, that means the 3.0 preview).
    • On Linux (at least Ubuntu), you must create the file /etc/udev/rules.d/51-android.rules as root and then add a manufacturer-specific entry for the device as described by Android’s Setting up a Device for Development. Example for one of our test devices:entry:
      SUBSYSTEM=="usb", ATTR{idVendor}==" 19d2", MODE="0666", GROUP="plugdev"
    • Make sure you have the latest version of Firefox OS on your device (especially due to recent fixes like bug 842725)

    Use make-fxos-install and command line

    If you want to establish a good workflow and you’re a fan of the terminal, we’d like to recommend using make-fxos-install by Harald Kirschner. It’s a command-line tool to easily push apps directly to a device.

    Step 1

    Step 2

    Clone the make-fxos-install repository.

    Step 3

    Go into your local make-fxos-install directory on the command line.

    Choose what app type to install

    Your two options are packaged or hosted. You need to call a make command and point to the folder of your app. For example:

    make FOLDER=../Firefox-OS-Boilerplate-App packaged install

    Alternatively:

    make FOLDER=../Firefox-OS-Boilerplate-App hosted install

    Accept the prompt on the device, and that’s it!

    image

    Please note that the packaged app will have all files in your app, while the hosted app alternative only needs two files to be installed:

    • A manifest.webapp file
    • A metadata.json file

    Then in the metadata.json file you only need to set the manifestURL parameter from where the app should be served.

    Manual steps

    The same steps as described above for Push to Device in the Firefox OS Simulator applies:

    Close and restart app to see changes, and an updated manifest = rebooting the OS to see any of those changes.

    Test it out!

    Please test out the above solutions and see what works best for you (we realize it might be a little while before you get access to an actual device). let us know what works and what doesn’t – we’d love to hear your feedback!

  10. Adding cursor swipe to the Firefox OS keyboard

    In this article we will take a look at how to approach adding features to a core component in the system such as the input keyboard. It turns out it is pretty easy!

    Before we start, take a look at this concept video from Daniel Hooper to get an idea of what we want to implement:

    Cool, huh? Making such a change for other mobile platforms would be pretty hard or just plain impossible, but in Firefox OS it is quite simple and it will take us less than 50 lines of code.

    The plan

    Conceptually, what we want to achieve is that when the user swipes her finger on the keyboard area, the cursor in the input field moves a distance and direction proportional to the swiping, left or right.

    Since a common scenario is that the user might be pressing a wrong key and would like to slide to a close-by key to correct it, we will only start moving the cursor when the swipe distance is longer than the width of a single key.

    Preparing your environment

    In order to start hacking Firefox OS itself, you will need a copy of Gaia (the collection of webapps that make up the frontend of Firefox OS) and B2G desktop (a build of the B2G app runtime used on devices where all apps should run as they would on a device).

    You can take a look at this previous article from Mozilla Hacks in which we guide you through setting up and hacking on Gaia. There is also a complete guide to setting up this environment at https://wiki.mozilla.org/Gaia/Hacking.

    Once you get Gaia to run in B2G, you are ready to hack!

    Ready to hack!

    Firefox OS is all HTML5, and internally it is composed by several ‘apps’. We can find the main system apps in the apps folder in the gaia repository that you cloned before, including the keyboard app that we will be modifying.
    In this post we will be editing only apps/keyboard/js/keyboard.js, which is where
    a big chunk of the keyboard logic lives.

    We start by initializing some extra variables at the top of the file that will help us keep track of the swiping later.

    var swipeStartMovePos = null; // Starting point of the swiping
    var swipeHappening = false; // Are we in the middle of swiping?
    var swipeLastMousex = -1; // Previous mouse position
    var swipeMouseTravel = 0; // Amount traveled by the finger so far
    var swipeStepWidth = 0; // Width of a single keyboard key

    Next we should find where the keyboard processes touch events. At
    the top of keyboard.js we see that the event handlers for touch events are
    declared:

    var eventHandlers = {
      'touchstart': onTouchStart,
      'mousedown': onMouseDown,
      'mouseup': onMouseUp,
      'mousemove': onMouseMove
    };

    Nice! Now we need to store the coordinates of the initial touch event. Both onTouchStart and onMouseDown end up calling the function startPress after they do their respective post-touch tasks, so we will take care of storing the coordinates there.

    startPress does some work for when a key is pressed, like highlighting the key or checking whether the user is pressing backspace. We will write our logic after that. A convenient thing is that one of the arguments in its signature is coords, which refers to the coordinates where the user started touching, in the context of the keyboard element. So storing the coordinates is as easy as that:

    function startPress(target, coords, touchId) {
      swipeStartMovePos = { x: coords.pageX, y: coords.pageY };
      ...

    In that way we will always have available the coordinates of the last touch even starting point.

    The meat of our implementation will happen during the mousemove event, though. We see that the function onMouseMove is just a simple proxy function for the bigger movePress function, where the ‘mouse’ movements are processed. Here is where we will write our cursor-swiping logic.

    We will use the width of a keyboard key as our universal measure. Since the width of keyboard keys changes from device to device, we will first have to retrieve it calling a method in IMERender, which is the object that controls how the keyboard is rendered on the screen:

    swipeStepWidth = swipeStepWidth || IMERender.getKeyWidth();

    Now we can check if swiping is happening, and whether the swiping is longer than swipeStepWidth. Conveniently enough, our movePress function also gets passed the coords object:

    if (swipeHappening || (swipeStartMovePos && Math.abs(swipeStartMovePos.x - coords.pageX) > swipeStepWidth)) {

    Most of our logic will go inside that ‘if’ block. Now that we know that swiping is happening, we have to determine what direction it is going, assigning 1 for right and -1 for left to our previously initialized variable swipeDirection. After that, we add the amount of distance traveled to the variable swipeMouseTravel, and set swipeLastMousex to the current touch coordinates:

    var swipeDirection = coords.pageX > swipeLastMousex ? 1 : -1;
     
    if (swipeLastMousex > -1) {
      swipeMouseTravel += Math.abs(coords.pageX - swipeLastMousex);
    }
    swipeLastMousex = coords.pageX;

    Ok, now we have to decide how the pixels travelled by the user’s finger will translate into cursor movement. Let’s make that half the width of a key. That means that for every swipeStepWidth / 2 pixels travelled, the cursor in the input field will move one character.

    The way we will move the cursor is a bit hacky. What we do is to simulate the pressing of ‘left arrow’ or ‘right arrow’ by the user, even if these keys don’t even exist in the phone’s virtual keyboard. That allows us to move the cursor in the input field. Not ideal, but Mozilla is about to push a new Keyboard IME API that will give the programmer a proper API to manipulate curor positions and selections. For now, we will just workaround it:

    var stepDistance = swipeStepWidth / 2;
    if (swipeMouseTravel > stepDistance) {
      var times = Math.floor(swipeMouseTravel / stepDistance);
      swipeMouseTravel = 0;
      for (var i = 0; i < times; i++)
        navigator.mozKeyboard.sendKey(swipeDirection === -1 ? 37 : 39, undefined);
    }

    After that we just need to confirm that swiping is happening and do some cleanup of timeouts and intervals initialized in other areas of the file, that because of our new swiping functionality ouldn’t get executed otherwise. We also call hideAlternatives to avoid the keyboard to present us with alternative characters while we are swiping.

    swipeHappening = true;
     
    clearTimeout(deleteTimeout);
    clearInterval(deleteInterval);
    clearTimeout(menuTimeout);
    hideAlternatives();
    return;

    The only thing left to do is to reset all the values we’ve set when the user lifts her finger off the screen. The event handler for that is onMouseUp, which calls the function endPress, at the beginning of which we will put our logic:

    // The user is releasing a key so the key has been pressed. The meat is here.
    function endPress(target, coords, touchId) {
        swipeStartMovePos = null;
        ...
        if (swipeHappening === true) {
            swipeHappening = false;
            swipeLastMousex = -1;
            return;
        }

    With this last bit, our implementation is complete. Here is a rough video I’ve made with the working implementation:

    You can see the complete implementation code changes on GitHub.

    Conclusion

    Contributing bugfixes or features to Firefox OS is as easy as getting Gaia, B2G and start hacking in HTML5. If you are comfortable programming in JavaScript and familiar with making web pages, you can already contribute to the mobile operating system from Mozilla.

    Appendix: Finding an area to work on

    If you already know what bug you want to solve or what feature you want to implement in Firefox OS, first check if it has already been filed in Bugzilla, which is the issue repository that Mozilla uses to keep track of bugs. If it hasn’t, feel free to add it. Otherwise, if you are looking for new bugs to fix, a quick search will reveal many new ones that are sill unassigned. Feel free to pick them up!