Mozilla

Articles by Robert Nyman [Editor emeritus]

Sort by:

View:

  1. Easy audio capture with the MediaRecorder API

    The MediaRecorder API is a simple construct, used inside Navigator.getUserMedia(), which provides an easy way of recording media streams from the user’s input devices and instantly using them in web apps. This article provides a basic guide on how to use MediaRecorder, which is supported in Firefox Desktop/Mobile 25, and Firefox OS 2.0.

    What other options are available?

    Capturing media isn’t quite as simple as you’d think on Firefox OS. Using getUserMedia() alone yields raw PCM data, which is fine for a stream, but then if you want to capture some of the audio or video you start having to perform manual encoding operations on the PCM data, which can get complex very quickly.

    Then you’ve got the Camera API on Firefox OS, which until recently was a certified API, but has been downgraded to privileged recently.

    Web activities are also available to allow you to grab media via other applications (such as Camera).

    the only trouble with these last two options is that they would capture only video with an audio track, and you would still have separate the audio if you just wanted an audio track. MediaRecorder provides an easy way to capture just audio (with video coming later — it is _just_ audio for now.)

    A sample application: Web Dictaphone

    An image of the Web dictaphone sample app - a sine wave sound visualization, then record and stop buttons, then an audio jukebox of recorded tracks that can be played back.

    To demonstrate basic usage of the MediaRecorder API, we have built a web-based dictaphone. It allows you to record snippets of audio and then play them back. It even gives you a visualization of your device’s sound input, using the Web Audio API. We’ll concentrate on the recording and playback functionality for this article.

    You can see this demo running live, or grab the source code on Github (direct zip file download.)

    CSS goodies

    The HTML is pretty simple in this app, so we won’t go through it here; there are a couple of slightly more interesting bits of CSS worth mentioning, however, so we’ll discuss them below. If you are not interested in CSS and want to get straight to the JavaScript, skip to the “Basic app setup” section.

    Keeping the interface constrained to the viewport, regardless of device height, with calc()

    The calc function is one of those useful little utility features that’s cropped up in CSS that doesn’t look like much initially, but soon starts to make you think “Wow, why didn’t we have this before? Why was CSS2 layout so awkward?” It allows you do a calculation to determine the computed value of a CSS unit, mixing different units in the process.

    For example, in Web Dictaphone we have theee main UI areas, stacked vertically. We wanted to give the first two (the header and the controls) fixed heights:

    header {
      height: 70px;
    }
     
    .main-controls {
      padding-bottom: 0.7rem;
      height: 170px;
    }

    However, we wanted to make the third area (which contains the recorded samples you can play back) take up whatever space is left, regardless of the device height. Flexbox could be the answer here, but it’s a bit overkill for such a simple layout. Instead, the problem was solved by making the third container’s height equal to 100% of the parent height, minus the heights and padding of the other two:

    .sound-clips {
      box-shadow: inset 0 3px 4px rgba(0,0,0,0.7);
      background-color: rgba(0,0,0,0.1);
      height: calc(100% - 240px - 0.7rem);
      overflow: scroll;
    }

    Note: calc() has good support across modern browsers too, even going back to Internet Explorer 9.

    Checkbox hack for showing/hiding

    This is fairly well documented already, but we thought we’d give a mention to the checkbox hack, which abuses the fact that you can click on the <label> of a checkbox to toggle it checked/unchecked. In Web Dictaphone this powers the Information screen, which is shown/hidden by clicking the question mark icon in the top right hand corner. First of all, we style the <label> how we want it, making sure that it has enough z-index to always sit above the other elements and therefore be focusable/clickable:

    label {
        font-family: 'NotoColorEmoji';
        font-size: 3rem;
        position: absolute;
        top: 2px;
        right: 3px;
        z-index: 5;
        cursor: pointer;
    }

    Then we hide the actual checkbox, because we don’t want it cluttering up our UI:

    input[type=checkbox] {
       position: absolute;
       top: -100px;
    }

    Next, we style the Information screen (wrapped in an <aside> element) how we want it, give it fixed position so that it doesn’t appear in the layout flow and affect the main UI, transform it to the position we want it to sit in by default, and give it a transition for smooth showing/hiding:

    aside {
       position: fixed;
       top: 0;
       left: 0;
       text-shadow: 1px 1px 1px black;
       width: 100%;
       height: 100%;
       transform: translateX(100%);
       transition: 0.6s all;
       background-color: #999;
        background-image: linear-gradient(to top right, rgba(0,0,0,0), rgba(0,0,0,0.5));
    }

    Last, we write a rule to say that when the checkbox is checked (when we click/focus the label), the adjacent <aside> element will have it’s horizontal translation value changed and transition smoothly into view:

    input[type=checkbox]:checked ~ aside {
      transform: translateX(0);
    }

    Basic app setup

    To grab the media stream we want to capture, we use getUserMedia() (gUM for short). We then use the MediaRecorder API to record the stream, and output each recorded snippet into the source of a generated <audio> element so it can be played back.

    First, we’ll add in a forking mechanism to make gUM work, regardless of browser prefixes, and so that getting the app working on other browsers once they start supporting MediaRecorder will be easier in the future.

    navigator.getUserMedia = ( navigator.getUserMedia ||
                           navigator.webkitGetUserMedia ||
                           navigator.mozGetUserMedia ||
                           navigator.msGetUserMedia);

    Then we’ll declare some variables for the record and stop buttons, and the <article> that will contain the generated audio players:

    var record = document.querySelector('.record');
    var stop = document.querySelector('.stop');
    var soundClips = document.querySelector('.sound-clips');

    Finally for this section, we set up the basic gUM structure:

    if (navigator.getUserMedia) {
       console.log('getUserMedia supported.');
       navigator.getUserMedia (
          // constraints - only audio needed for this app
          {
             audio: true
          },
     
          // Success callback
          function(stream) {
     
     
          },
     
          // Error callback
          function(err) {
             console.log('The following gUM error occured: ' + err);
          }
       );
    } else {
       console.log('getUserMedia not supported on your browser!');
    }

    The whole thing is wrapped in a test that checks whether gUM is supported before running anything else. Next, we call getUserMedia() and inside it define:

    • The constraints: Only audio is to be captured; MediaRecorder only supports audio currently anyway.
    • The success callback: This code is run once the gUM call has been completed successfully.
    • The error/failure callback: The code is run if the gUM call fails for whatever reason.

    Note: All of the code below is placed inside the gUM success callback.

    Capturing the media stream

    Once gUM has grabbed a media stream successfully, you create a new Media Recorder instance with the MediaRecorder() constructor and pass it the stream directly. This is your entry point into using the MediaRecorder API — the stream is now ready to be captured straight into a Blob, in the default encoding format of your browser.

    var mediaRecorder = new MediaRecorder(stream);

    There are a series of methods available in the MediaRecorder interface that allow you to control recording of the media stream; in Web Dictaphone we just make use of two. First of all, MediaRecorder.start() is used to start recording the stream into a Blob once the record button is pressed:

    record.onclick = function() {
      mediaRecorder.start();
      console.log(mediaRecorder.state);
      console.log("recorder started");
      record.style.background = "red";
      record.style.color = "black";
    }

    When the MediaRecorder is recording, the MediaRecorder.state property will return a value of “recording”.

    Second, we use the MediaRecorder.stop() method to stop the recording when the stop button is pressed, and finalize the Blob ready for use somewhere else in our application.

    stop.onclick = function() {
      mediaRecorder.stop();
      console.log(mediaRecorder.state);
      console.log("recorder stopped");
      record.style.background = "";
      record.style.color = "";
    }

    When recording has been stopped, the state property returns a value of “inactive”.

    Note that there are other ways that a Blob can be finalized and ready for use:

    • If the media stream runs out (e.g. if you were grabbing a song track and the track ended), the Blob is finalized.
    • If the MediaRecorder.requestData() method is invoked, the Blob is finalized, but recording then continues in a new Blob.
    • If you include a timeslice property when invoking the start() method — for example start(10000) — then a new Blob will be finalized (and a new recording started) each time that number of milliseconds has passed.

    Grabbing and using the blob

    When the blob is finalized and ready for use as described above, a dataavailable event is fired, which can be handled using a mediaRecorder.ondataavailable handler:

    mediaRecorder.ondataavailable = function(e) {
      console.log("data available");
     
      var clipName = prompt('Enter a name for your sound clip');
     
      var clipContainer = document.createElement('article');
      var clipLabel = document.createElement('p');
      var audio = document.createElement('audio');
      var deleteButton = document.createElement('button');
     
      clipContainer.classList.add('clip');
      audio.setAttribute('controls', '');
      deleteButton.innerHTML = "Delete";
      clipLabel.innerHTML = clipName;
     
      clipContainer.appendChild(audio);
      clipContainer.appendChild(clipLabel);
      clipContainer.appendChild(deleteButton);
      soundClips.appendChild(clipContainer);
     
      var audioURL = window.URL.createObjectURL(e.data);
      audio.src = audioURL;
     
      deleteButton.onclick = function(e) {
        evtTgt = e.target;
        evtTgt.parentNode.parentNode.removeChild(evtTgt.parentNode);
      }
    }

    Let’s go through the above code and look at what’s happening.

    First, we display a prompt asking the user to name their clip.

    Next, we create an HTML structure like the following, inserting it into our clip container, which is a <section> element.

    <article class="clip">
      <audio controls></audio>
      <p><em>your clip name</em></p>
      <button>Delete</button>
    </article>

    After that, we create an object URL pointing to the event’s data attribute, using window.URL.createObjectURL(e.data): this attribute contains the Blob of the recorded audio. We then set the value of the <audio> element’s src attribute to the object URL, so that when the play button is pressed on the audio player, it will play the Blob.

    Finally, we set an onclick handler on the delete button to be a function that deletes the whole clip HTML structure.

    Conclusion

    And there you have it; MediaRecorder should serve to make your app media recording needs easier. Have a play around with it and let us know what you think: we are looking forward to seeing what you’ll build!

  2. PlayCanvas Goes Open Source

    This is a guest post by Will Eastcott of the PlayCanvas engine. As outlined in What Mozilla Hacks is, we constantly cover interesting information about open source and the Open Web, both from external as well as Mozilla authors, so feel free to share with us!

    On March 22nd 2011, Mozilla released Firefox 4.0 which enabled WebGL by default. A month later, we formed PlayCanvas and started building a game engine unlike anything that had gone before. Fast forward three years, and WebGL is everywhere. Only this week, Apple announced support for WebGL in both OS X and iOS 8. So what better time pass on some more exciting news for you:

    The PlayCanvas Engine has been open sourced!

    MozBlog500

    Introducing the PlayCanvas Engine

    The PlayCanvas Engine is a JavaScript library engineered specifically for building video games. It implements all of the major components that you need to write high quality games:

    • Graphics: model loading, per-pixel lighting, shadow mapping, post effects
    • Physics: rigid body simulation, ray casting, joints, trigger volumes, vehicles
    • Animation: keyframing, skeletal blending, skinning
    • Audio engine: 2D and 3D audio sources
    • Input devices: mouse, keyboard, touch and gamepad support
    • Entity-component system: high level game object management

    We had a couple of goals in mind when we originally designed the engine.

    1. It had to be easy to work with.
    2. It had to be blazingly fast.

    Simple Yet Powerful

    As a developer, you want well documented and well architected APIs. But you also want to be able to understand what’s going on under the hood and to debug when things go wrong. For this, there’s no substitute for a carefully hand-crafted, unminified, open source codebase.

    Additionally, you need great graphics, physics and audio engines. But the PlayCanvas Engine takes things a step further. It exposes a game framework that implements an entity-component system, allowing you to build the objects in your games as if they were made of Lego-like blocks of functionality. So what does this look like? Let’s check out a simple example on CodePen: a cannonball smashing a wall:

    playcanvas_codepen

    As you can see from the Pen’s JS panel, in just over 100 lines of code, you can create, light, simulate and view interesting 3D scenes. Try forking the CodePen and change some values for yourself.

    Need For Speed

    To ensure we get great performance, we’ve built PlayCanvas as a hybrid of hand-written JavaScript and machine generated asm.js. The most performance critical portion of the codebase is the physics engine. This is implemented as a thin, hand-written layer that wraps Ammo.js, the Emscripten-generated JavaScript port of the open source physics engine Bullet. If you haven’t heard of Bullet before, it powers amazing AAA games like Red Dead Redemption and GTAV. So thanks to Mozilla’s pioneering work on Emscripten and asm.js, all of this power is also exposed via the PlayCanvas engine. Ammo.js executes at approximately 1.5x native code speed in recent builds of Firefox so if you think that complex physics simulation is just not practical with JavaScript, think again.

    But what about the non-asm.js parts of the codebase? Performance is clearly still super-important, especially for the graphics engine. The renderer is highly optimized to sort draw calls by material and eliminate redundant WebGL calls. It has also been carefully written to avoid making dynamic allocations to head off potential stalls due to garbage collection. So the code performs brilliantly but is also lightweight and human readable.

    Powering Awesome Projects

    The PlayCanvas Engine is already powering some great projects. By far and away, the biggest is the PlayCanvas web site: the world’s first cloud-hosted game development platform.

    For years, we’ve been frustrated with the limitations of current generation game engines. So shortly after starting work on the PlayCanvas Engine, we began designing a new breed of game development environment that would be:

    Accessible
    using any device with a web browser, plug in a URL and instantly access simple, intuitive yet powerful tools.
    Collaborative
    See what you teammates are working on in real-time or just sit back and watch a game as it’s built live before your eyes.
    Social
    Making games is easier with the help of others. Be part of an online community of developers like you.

    PlayCanvas ticks all of these boxes beautifully. But don’t take our word for it – head to https://playcanvas.com and discover a better way to make games.

    In fact, here’s a game we have built using these very tools. It’s called SWOOOP:

    PLAY NOW!

    It’s a great demonstration of what you can achieve with HTML5 and WebGL today. The game runs great in both mobile and desktop browsers, and you are free to deploy your PlayCanvas games to app stores as well. For Google Play and the iOS App Store, there are wrapping technologies available that can generate a native app of your game. Examples of these are Ludei’s CocoonJS and the open source Ejecta project. For Firefox OS, the process is a breeze since the OS treats HTML5 apps as first class citizens. PlayCanvas games will run out of the box.

    Want!

    So if you think this is sounding tasty, where should you go to get started? The entire engine sourcebase is now live on GitHub:

    https://github.com/playcanvas/engine

    Get cloning, starring and forking while it’s fresh!

    Stay in the Loop

    Lastly, I want to give you some useful links that should help you stay informed and find help whenever you need it.

    We’re super excited to see what the open source community will do with the PlayCanvas Engine. So get creative and be sure to let us know about your projects.

    Toodle pip!

  3. Introducing webcompat.com

    For the past few months a small group of contributors inside and outside of Mozilla have been working on webcompat.com. We just recently celebrated moving past the “too broken to share” milestone to the “functional-under-construction.gif” milestone of the project and are eager to share what we’ve been up to.

    There’s a more elaborate description of web compatibility and the site over at http://webcompat.com/about, but the basic premise of this project is: what if you could “file a bug on the internet” in a public space where developers, users and browser vendors could be made aware of compatibility issues?

    You can report issues to browser vendors today, through bug trackers or other feedback mechanisms, some public and some not. But frequently you cannot see the results of such a report—or see what related issues others have reported. It can feel like a black hole experience.

    This is where webcompat.com comes in. We want to provide a simple, open interface to web compatibility issues that affect us all as users of the web, regardless of our browsers or devices of choice. Perhaps more importantly, we hope to foster and enable a community of people who are passionate about a web for everyone to help out.

    So is this a Mozilla-only effort?

    Not at all. Mozilla kindly allows me to work on this project as a part of my full-time responsibilities on the Mozilla Mobile Web Compatibility team—but the project’s goal has always been to create an open resource for all people who use and build things on the internet. We’ve had some positive conversations with some of our browser vendor colleagues that have motivated us to start initial plans around future collaborations. It’s clear to us that Mozilla isn’t the only entity invested in a compatible web.

    OK Neat. What can I do?

    There’s actually quite a bit to do if this type of work is interesting to you.

    Reporting Issues

    Do you happen to notice sites that block you, nag you to install a different browser, or are just plain broken or buggy? Instead of just tweeting sad emoticons or #fail hashtags, you can help make a difference by creating a record of these issues with the form at webcompat.com.

    We’ve also created some browser extensions to make reporting easier—just click the added UI button to open a new tab with some bug report info pre-filled for you. You can grab that for Firefox Desktop, Firefox for Android, Chrome, Opera, and Safari. There’s also a bookmarklet you can install in your bookmark or favorites bar. These are all open source, so feel free to hack away and port them to new platforms.

    Diagnosing Compatibility Issues

    Do you have a knack for figuring out why things are broken and want to hone your internet CSI skills? Have you considered getting a “TypeError: ‘undefined’ is not a function” tattoo? Do you love reading minified code and debugging the entire web stack? We could use your help confirming and diagnosing issues (tattoos not required).

    Site Outreach

    Are you more interested in doing outreach and making sites and developers aware of issues? We need your help too. Stay tuned for better tooling around this role—or hop into the #webcompat irc channel on the Mozilla IRC network and say hi.

    Development

    Do you know or want to get better at JavaScript, Python, CSS, UX, design, GitHub, browser extensions, testing, and more? We’ve got a pile of issues we’re working on over at GitHub and welcome patches, opinions, issues, and feedback.

    What’s coming next?

    Right now two of our biggest priorities are being able to view and interact with issues in the site itself, and creating tools that will enable importing existing web compatibility issues from bug trackers (both open and closed) and exporting compatibility issues on webcompat.com to browser vendors (when the bug lies with the browser, not the site).

    Our project is still in its infancy, but we look forward to your compatibility bug reports and comments!

  4. Pre-orders start today for Flame, the Firefox OS developer phone

    Update 2014-07-15: The pre-order period has ended and the Flame is now available as a standard order, with shipping in 7-10 days.

    The Firefox OS Flame reference device that we announced at end of February is now available for pre-order at everbuying.com for $170 including free shipping.

    Pre-order now.

    To standardize the design, development, and testing of Firefox OS, Mozilla has partnered with a company called Thundersoft to manufacture, distribute, and update a Firefox OS reference phone called the Flame. Until now, there has been no “reference device” and the options for getting something through retail were limited.

    Mid-tier phone hardware

    The Flame is representative of the mid-tier phone hardware Mozilla and its partners are targeting over the coming year. It was designed for our developer and contributor community, so we worked with the manufacturer to keep the price as low as possible. We’re excited that we are able to bring a high quality reference device to our developer community at an affordable price.

    The Flame will also be a great development and testing environment for third party app developers targeting Firefox OS and HTML5. The phone offers a software configurable RAM option that can be made to emulate many different devices that will be commercially available later this year.

    Our partner will provide the Flame with updates to each new major version of Firefox OS and a simple mechanism for switching between different release channels; offering Nightly builds that will keep you at the forefront of Firefox OS development.

    If you’ve held off on getting a Firefox OS phone because they weren’t available in your region or the phones in market didn’t meet your development and testing needs, don’t miss out on the opportunity to pre-order one of these Flame reference phones today.

    Specifications & unlocked!

    The Flame is unlocked from any network and comes with the bootloader unlocked.

    • Qualcomm MSM8210 Snapdragon, 1.2GHZ Dual core processor
    • 4.5” screen (FWVGA 854×480 pixels)
    • Cameras: Rear: 5MP with auto-focus and flash / Front: 2MP
    • Frequency: GSM 850/900/1800/1900MHz
      UMTS 850/900/1900/2100MHz
    • 8GB memory, MicroSD slot
    • 256MB – 1GB RAM (adjustable by developer)
    • A-GPS, NFC
    • Dual SIM Support
    • Battery capacity: 1,800 mAh
    • WiFi: 802.11 b/g/n, Bluetooth 3.0, Micro USB

    NOTE: Once pre-ordered, the Flame will take approximately four weeks before it ships. The Flame ships free to anywhere in the world except for Japan. If you want to pre-order a Flame device certified for use in Japan, please visit here for more information.

    For more information:
    Mozilla Developer Network guide to the Flame reference phone

  5. Developer Tools feedback channels, one week in

    Last week we launched Feedback channels, with the first one out on Developer Tools in Firefox. The feedback channel on UserVoice is the place to go for any ideas or thoughts about the Firefox Developer Tools.

    Looking at the first week for Developer Tools feedback

    During the first week for our Developer Tools feedback channel, we got:

    • 99 new ideas
    • 1382 votes
    • 4080 unique users

    Mozilla has responded to 97% of the ideas, out of which:

    Notable is that many ideas that came up are things that we already have bugs for in Bugzilla and ideas and approaches that have been talked about and considered.

    Thank you!

    So, sincerely, thank you for all the input and constructive ideas and comments! We appreciate it a lot and by having this direct dialog and communication with you, it helps us to build better and more useful Developer Tools!

    Please continue to express your ideas, vote on existing ones and stay on track with the progress!

  6. Introducing TranslationTester and localization support for Open Web Apps

    When building Open Web Apps, one important factor is to make your content available to as many possible, and one way to do that is to make it available in more than one language. Therefore we want to make it as easy as possible for you, both with a complete code repository to get started and updated documentation and possibilities to get translations for your apps.

    We are now also offering a way to have your app set up and to get connected to localizers directly!

    Presenting TranslationTester

    Based on the great work and content by Jason in Localizing Firefox OS Apps and Christie in Localizing the Firefox OS Boilerplate App, I’ve been working on simplifying the set-up for localization.

    Above articles describe many facets and possibilities available, and to complement that, I wanted to be able to offer the most basic code and structure for most use cases: basically, just add translations and you’re good to go!

    This lead to the creation of the TranslationTester code repository, available on GitHub.

    From no translation to localized app in less than 4 minutes!

    Based on the TranslationTester, Will Bamberg has put together a nice and short screencast showing the exact process of going from no translations to having support in just a few minutes!

    In the app-l10n-example repository you have versions of the before and after stages in this screencast, and all of the localization process is described on MDN.

    How to localize an app

    These are the things needed to localize an app. All of them are included in the TranslationTester repository, but included here to show easy it is to get started, or continue to build on the TranslationTester platform for your app.

    Mark up the HTML

    For any element that you want to have its text localized, add a data-l10n-id attribute to the desired element. For example:

    <p data-l10n-id="winter-for-real">Winter for real</p>

    Create translations

    In the locales directory, you have a directory for each language, and can just add new ones for new languages. In each language directoy you create an app.properties file which will contain all your translations for that language (optionally you can also create a manifest.properties file for the name and description of the app, unless you edit it in the main manifest.webapp for your app).

    Include l10n.js

    By including the l10n.js file, the translations for respective language’s app.properties file will be applied to element with the corresponding data-l10n-id attribute.

    Add “name” and “description” translations

    The translations for the name and description for your app could be added to the main manifest.webapp file, or in an optional manifest.properties under each locale’s directory.

    How to view different locales

    To view your app with a different locale, change the language in Firefox/App Manager:

    All platforms
    Set the desired locale to test in JavaScript, e.g:

    document.webL10n.setLanguage("es");
    App Manager/Firefox OS
    Language setting in the Settings app
    Firefox
    Choose language under Preferences > Content > Languages. More information in Set content language in Firefox

    Get help with translations

    We have a Mozilla localization app pilot offered through the Transifex service. As part of that, we’re offering you to add your apps as part of this pilot, to be connected to localizers and quickly offer your app in more languages, to more people.

    Please apply in the Localize My Firefox App form to get started!

  7. How fast is PDF.js?

    Hi, my name is Thorben and I work at Opera Software in Oslo, not at Mozilla. So, how did I end up writing for Mozilla Hacks? Maybe you know that there is no default PDF viewer in the Opera Browser, something we would like to change. But how to include one? Buy it from Adobe or Foxit? Start our own?

    Introducing PDF.js

    While investigating our options we quickly stumbled upon PDF.js. The project aims to create a full-featured PDF viewer in the browser using JavaScript and Canvas. Yeah, it sounds a bit crazy, but it makes sense: browsers need to be good at processing text, images, fonts, and vector graphics — exactly the things a PDF viewer has to be good at. The draw commands in PDFs are a subset of Postscript, and they are not so different from what Canvas offers. Also security is virtually no issue: using PDF.js is as secure as opening any other website.

    Working on PDF.js

    So Christian Krebs, Mathieu Henri and myself began looking at PDF.js in more detail and were impressed: it’s well designed, seems fast and big parts of the code are just wow!

    But we also discovered some problems, mainly with performance on very large or graphics-heavy PDFs. We decided that the best way to get to know PDF.js better and to push the project further, was to help the project and address the major issues we found. This gave us a pretty good understanding of the project and its high potential. We were also very impressed by how much the performance of PDF.js improved while we worked on it. This is an active and well managed project.

    Benchmarking PDF.js

    Of course, our tests gave us the wrong impression about performance. We tried to find super large, awkward and hard-to-render PDFs, but that is not what most people want to view. Most PDFs you actually want to view in PDF.js are fine. But how to test that?

    Well, you could check the most popular PDFs on the Internet – as these are the ones you probably want to view – and benchmark them. A snapshot of 5 to 10k PDFs should be enough … but how do you get them?

    I figured that search engines would be my friend. If you tell them to search for PDFs only, they give you the most relevant PDFs for that keyword, which in turn are probably the most popular ones. And if you use the most searched keywords you end up with a good approximation.

    Benchmarking that many PDFs is a big task. So I got myself a small cluster of old computers and built a nice server application that supplied them with tasks. The current repository has almost 7000 PDFs and benchmarking one version of PDF.js takes around eight hours.

    The results

    Let’s skip to the interesting part with the pretty pictures. This graph

    histogram

    gives us almost all the interesting results at one look. You see a histogram of the time it took to process all the pages in the PDFs in relation to the average time it takes to process the average page of the Tracemonkey Paper (the default PDF you see when opening PDF.js). The User Experience when viewing the Tracemonkey Paper is good and from my tests even 3 to 4 times slower is still okay. That means from all benchmarked pages over 96% (exclude pdfs that crashed) will translate to a good user experience. That is really good news! Or to use a very simple pie chart (in % of pages):

    overview

    You probably already noticed the small catch: around 0.8% of the PDFs crashed PDF.js when we tested them. We had a closer look at most of them and at least a third are actually so heavily damaged that probably no PDF viewer could ever display them.

    And this leads us to another good point: we have to keep in mind that these results just stand here without comparison. There are some PDFs on the Internet that are so complex that there is no hope that even native PDF viewers could display them nice and fast. The slowest tested PDF is an incredibly detailed vector map of the public transport system of Lisbon. Try to open it in Adobe Reader, it’s not fun!

    Conclusion

    From these results we concluded that PDF.js is a very valid candidate to be used as the default PDF viewer in the Opera Browser. There is still a lot of work to do to integrate PDF.js nicely into it, but we are working right now on integrating it behind an experimental flag (BTW: There is an extension that adds PDF.js with the default Mozilla viewer. The “nice” integration I am talking about would be deeper and include a brand new viewer). Thanks Mozilla! We are looking forward to working on PDF.js together with you guys!

    PS: Both the code of the computational system and the results are publicly available. Have a look and tell us if you find them useful!

    PPS: If anybody works at a big search engine company and could give me a list with the actual 10k most used PDFs, that would be awesome :)

    Appendix: What’s next?

    The corpus and the computational framework I described, could be used to do all kinds of interesting things. In the next step, we hope to classify PDFs by used fonts formats, image formats and the like. So you can quickly get PDFs to test a new feature with. We also want to look at which drawing instructions are used with which frequency in the Postscript so we can better optimise for the very common ones, like we did with HTML in browsers. Let’s see what we can actually do ;)

  8. Editable box model, multiple selection, Sublime Text keys + much more – Firefox Developer Tools Episode 31

    A new set of the Firefox Developer Tools features has just been uplifted to the Aurora channel. These features are available right now in Aurora, and will be in the Firefox 31 release in July. This release brings new tools, editor improvements, console and inspector features:

    Editable box model

    The Box Model tab in the Inspector is now editable for easy experimentation. Double-click any of the margin, border, or padding values to change its value for the currently selected element. Enter any valid CSS <length> value and use the Up/Down keys to increment or decrement the value by 1. Alt-Up increments by 0.1 and Shift-Up increments by 10. (development notes)

    Editing the box model

    Eyedropper

    New to the color picker in the Inspector is an Eyedropper tool that grabs the color from any pixel on the page. Select the current color by clicking or pressing Enter. Abort the operation by pressing Esc. Use the Up/Down keys to move by one pixel, and Shift-Up/Shift-Down to move by 10 pixels.

    Eyedropper tool

    You can also use the eyedropper directly to copy a color to the clipboard by accessing it from Web Developer menu, or the toolbar icon that’s enabled by going to the settings panel and checking Available Toolbox Buttons > Grab a color from the page. (development notes)

    Console stack traces

    console.error, console.exception, and console.assert logs in the console now include the full stack from where the call was made. (development notes)

    Stack trace in console.error() printout

    Styled console logs

    On parity with other browser developer tools, you can now add style to console logging with the %c directive.
    (development notes)

    Using %c in console.log() to format output

    Copy as cURL

    Replay any network request in the Network Monitor from the comfort of your own terminal. Right-click a request and select the copy as cURL menu item to copy a cURL command to the clipboard, including arguments for headers and data. (development notes)

    Copy as cURL in Network tool

    Editor – multiple selection, Sublime Text keys

    The source editor used in the developer tools has been upgraded to CodeMirror 4. With that brings:

  9. Multiple selection. Hold down Ctrl/Cmd while selecting to get multiple selections.
  10. Rectangle selection. Hold down Alt to select a column-aligned block of text.
  11. Undo selection. Undo the last selection action with Ctrl-U/Cmd-U and redo with Alt-U/Shift-Cmd-U.
  12. Sublime Text keybindings. To enable, go to about:config in the url bar and set devtools.editor.keymap to sublime.
  13. Multiple selection in action:

    animation of multiple selection in the editor

    development notes

    Canvas Debugger

    Debug animation frames in WebGL and 2d canvas contexts with the newly-landed canvas debugger. The canvas debugger is an experimental feature that has to be enabled in the setting panel. Multiple canvases are not yet supported (bug) as well as animations generated with setInterval (bug). The canvas debugger is described in more in this blog post.
    (development notes)

    Add-on Debugger

    If you develop Firefox add-ons using the Add-on SDK, there’s now a much easier way to debug your add-on’s JavaScript modules. See the blog post for more details. (development notes)

    Firefox 31: Add-on Debugger from Jordan Santell on Vimeo.

    Other features

    • Expand descendants. Hold Alt while double-clicking a node in the Inspector to expand all of its children and descendants. (development notes)
    • Persist network logs. Check Enable persistent logs in the settings panel to keep Network panel logs across reloads and navigations. (development notes)
    • JS warnings on by default. JavaScript warnings now show up in the Console by default. (development notes)
    • Scratchpad View menu. The Scratchpad tool now has a View menu with options for changing font size, hiding line numbers, wrapping text, and highlighting trailing spaces. (development notes)

    Special thanks to the 38 contributors that added all the features and fixes in this release.

    Questions or suggestions? Comment here or shoot feedback to @FirefoxDevTools on Twitter or our brand new feedback channel for Firefox Developer Tools. If you’d like to help out, check out the guide to getting involved.

  14. Launching feedback channels – let us know your ideas for Firefox Developer Tools

    One thing that is vital for us at Mozilla is to listen to developers and users, and care to your needs to make the web a better place. We’re strong believers in constructive communication and discussions, and that’s why I’m happy to announce our latest initiative! Feedback channels for developers.

    We’ve been looking into various ways to make it as easy as possible for developers to express their thoughts and opinions on a number of topics, and also for how we at Mozilla can be as transparent as possible around the projects we work on.

    Making the web a better place isn’t only about trying to figure out what’s right – it’s about listening to people, gather their thoughts and ideas and let that help us in achieving better results.

    Introducing UserVoice

    With that in mind, we’ve been evaluating third party services available as well as our own projects such as Mozilla Support and Firefox Input.

    For this specific use case and to match our requirements, we’ve decided to do it through the UserVoice service. We want to make it dead simple for you to enter ideas and vote for existing ones, both through an easy and intuitive user interface but also with requiring no account registration.

    Currently all you need to do know is enter a name and an e-mail address and you’re good to go! You will also have 10 votes dedicated to your e-mail address/account, which you can dispose as you seem fit for other ideas there.

    Naturally, if we see an abuse of this simplicity, we will have to add an element of moderation to process. We sincerely hope that won’t be needed, though, and that you will act and write respectfully.

    First project: Firefox Developer Tools

    The first project out the door that we will have a feedback channel for is the Developer Tools in Firefox. We have made great great progress with the Developer Tools over the last year – outlined in more detail in Dev Tools articles here on Hacks and the extensive documentation on MDN – but we want to hear the ideas you have to make them even better, what’s important to you, what could be a game changer and much more.

    People from the Developer Tools team will interact directly with you in this feedback channel, giving you a unique opportunity to influence and inspire us, resulting in a better product for you.

    A first test

    This try with UserVoice and feedback channels is our first attempt at this, with the ambition to be able to listen to your thoughts and ideas, and utilize that to make sure we focus on the most important areas.

    If we see that we can make a difference with this initiative and that you appreciate the direct contact and transparency, we will do our best to keep this going and to introduce channels for more topics in the future!

    Will you automatically implement the feature with the highest votes?

    Not automatically. The voting and ideas will help us see what you truly care about and interact with you around those, but we will also need to compare developing time and resources vs. the possible gain we see for a certain feature.

    It will however be of immense help for us weighing ideas, features and items in the roadmap against each other, and a helpful tool for prioritizing our work.

    Working with UserVoice

    Here are the most common things you can do at UserVoice – what they are and what they look like.

    Viewing ideas

    You can easily view a list of ideas, the number of votes they’ve gotten, current state (e.g. Started, Under Review), click the link to see comments and more.

    You also have a quick navigation to sort the listing based on what’s Hot, Top ideas, New or their current status. You can also list them based on your feedback or comments as well.

    Enter an idea

    To enter an idea, simply type the idea in the text box at the top:

    When you start typing, you will get suggestions for existing ideas that might match your topic, highlighting the word(s)in question. Please make sure to vote for existing ideas instead of creating duplicate ones.

    Search existing ideas

    In the right-hand side of the web site, you can also quickly search through existing topics and ideas, and see both matches and their current state.

    Official comment from Mozilla

    It also gives us the possibility to give you a reply to the state of the current feature. Whether it’s already in there, planned or if there’s something that’s not currently on our roadmap.

    Complement to Bugzilla and Stack Overflow

    At Mozilla, we use Bugzilla for tracking bugs and Stack Overflow for developer support. Our feedback channels at UserVoice are intended for conversations, constructive feedback and transparency around what we work on. Therefore, for reporting bugs or getting support for a product or challenge:

    Help us help you

    We really hope you will like this opportunity to have a discussion with us and be able to influence our thinking and work! If this works out well, we’d be glad to take it to other areas as well.

    So help us help you! Let’s work together to make a better web with better products for developing for it.

    Go to our Developer Tools feedback channel and get started!

  15. asm.js performance improvements in the latest version of Firefox make games fly!

    The latest version of Firefox which launched last week includes a major update to the user interface as well as to features like Sync. Another area in which this release brings significant improvements is in asm.js performance, which as we will see below is very important for things like games. To put that aspect of Firefox’s performance in context, we’ll take a look at benchmark results comparing Firefox to other browsers, which show that Firefox is faster at executing asm.js code.

    asm.js speedups

    asm.js is a subset of JavaScript that is very easy to optimize and is particularly useful for porting code in C or C++ to the Web. We’ve blogged about how Firefox can optimize asm.js code using 32-bit floating point operations, which, together with all the other work on optimizing asm.js, allows it to run at around 1.5x slower than the speed of the same C/C++ when compiled natively. So, while not quite native speed yet, things are getting very close. At the time of that blog post those optimizations were only on nightly builds, but they are now reaching hundreds of millions of Firefox users in Firefox 29, which is now the release version of Firefox.

    Another important set of asm.js optimizations concern startup speed. As blogged about by Luke a few months ago, Firefox performs ahead of time (AOT) compilation and can cache the results, for significant speedups in startup times. Those optimizations also shipped to users in Firefox 29.

    Web browser comparisons

    Now that all those optimizations have shipped, it’s interesting to look at up-to-date browser comparisons on asm.js code. The above graph shows the Emscripten benchmark suite running the latest stable versions of Google Chrome, Internet Explorer and Firefox on Windows 8.1. Lower numbers are better in all the results here, which are real-world codebases compiled to asm.js (see notes in the graph).

    Unity, Emscripten and asm.js

    asm.js is a subset of JavaScript, so it is just one of many styles of JavaScript out there. But it represents an important use case. As we announced at GDC, Unity, one of the most popular game creation tools on the market, will support the Web by using Emscripten to compile their engine to asm.js.

    But videos are no substitute for the real thing! You can try the games shown there in your browser right now, with Unity’s recently released Dead Trigger 2 and Angry Bots demos. If you run those in the latest version of Firefox, you’ll see many of the asm.js optimizations mentioned earlier in action. For example, if you visit one of those links more than once then asm.js caching will allow it to avoid recompiling the game (so it starts up faster), and also gameplay will be smoother due to faster asm.js execution.

    Being able to execute asm.js-style code efficiently makes it possible for games like this to run well on the Web, without proprietary, nonstandard plugins. That’s why it’s exciting to see more asm.js optimizations reach Firefox users in Firefox 29. And while benchmark results can sometimes seem like nothing more than abstract numbers, speedups on asm.js benchmarks directly improve things like games, where performance is extremely important and very noticeable.

    (Thanks to Marc Schifer for helping with the benchmark measurements.)