Mozilla

Featured Articles

Sort by:

View:

  1. Pseudo elements, promise inspection, raw headers, and much more – Firefox Developer Edition 36

    Firefox 36 was just uplifted to the Developer Edition channel, so let’s take a look at the most important Developer Tools changes in this release. We will also cover some changes from Firefox 35 since it was released shortly before the initial Developer Edition announcement. There is a lot to talk about, so let’s get right to it.

    Inspector

    You can now inspect ::before and ::after pseudo elements.  They behave like other elements in the markup tree and inspector sidebars. (35, development notes)

    before-after-inspection

    There is a new “Show DOM Properties” context menu item in the markup tree. (35, development notes, documentation about this feature on MDN)

    show-dom-properties

    The box model highlighter now works on remote targets, so there is a full-featured highlighter even when working with pages on Firefox for Android or apps on Firefox OS. (36, development notes, and technical documentation for building your own custom highlighter)

    Shadow DOM content is now visible in the markup tree; note that you will need to set dom.webcomponents.enabled to true to test this feature (36, development notes, and see bug 1053898 for more work in this space).

    We borrowed a useful feature from Firebug and are now allowing more paste options when right clicking a node in the markup tree.  (36, development notes, documentation about this feature on MDN)

    paste-options

    Some more changes to the Inspector included in Firefox 35 & 36:

    • Deleting a node now selects the previous sibling instead of the parent (36, development notes)
    • There is higher contrast for the currently hovered node in the markup view (36, development notes)
    • Hover over a CSS selector in the computed view to highlight all the nodes that match that selector on the page. (35, development notes)

    Debugger / Console

    DOM Promises are now inspectable. You can inspect the promises state and value at any moment. (36, development notes)

    promise-inspection

    The debugger now recognizes and works with eval’ed sources. (36, development notes)

    Eval’ed sources support the //# sourceURL=path.js syntax, which will make them appear as a normal file in the debugger and in stack traces reported by Error.prototype.stack. See this post: http://fitzgeraldnick.com/weblog/59/ for much more information. (36, development notes,  more development notes)

    Console statements now include the column number they originated from. (36, development notes)

    WebIDE

    You are now able to connect to Firefox for Android from the WebIDE.  See documentation for debugging firefox for android on MDN.  (36, development notes).

    We also made some changes to improve the user experience in the WebIDE:

    • Bring up developer tools by default when I select a runtime app / tab (35, development notes)
    • Re-select project on connect if last project is runtime app (35, development notes)
    • Auto-select and connect to last used runtime if available (35, development notes)
    • Font resizing (36, development notes)
    • You can now adding a hosted app project by entering the base URL (eg: “http://example.com”) instead of requiring the full path to the manifest.webapp file (35, development notes)

    Network Monitor

    We added a plain request/response headers view to make it easier to view and copy the raw headers on a request. (35, development notes)

    net-headers-raw

    Here is a list of all the bugs resolved for Firefox 35 and all the bugs resolved for Firefox 36.

    Do you have feedback, bug reports, feature requests, or questions? As always, you can comment here, add/vote for ideas on UserVoice or get in touch with the team at @FirefoxDevTools on Twitter.

  2. Introducing the JavaScript Internationalization API

    Firefox 29 issued half a year ago, so this post is long overdue. Nevertheless I wanted to pause for a second to discuss the Internationalization API first shipped on desktop in that release (and passing all tests!). Norbert Lindenberg wrote most of the implementation, and I reviewed it and now maintain it. (Work by Makoto Kato should bring this to Android soon; b2g may take longer due to some b2g-specific hurdles. Stay tuned.)

    What’s internationalization?

    Internationalization (i18n for short — i, eighteen characters, n) is the process of writing applications in a way that allows them to be easily adapted for audiences from varied places, using varied languages. It’s easy to get this wrong by inadvertently assuming one’s users come from one place and speak one language, especially if you don’t even know you’ve made an assumption.

    function formatDate(d)
    {
      // Everyone uses month/date/year...right?
      var month = d.getMonth() + 1;
      var date = d.getDate();
      var year = d.getFullYear();
      return month + "/" + date + "/" + year;
    }
     
    function formatMoney(amount)
    {
      // All money is dollars with two fractional digits...right?
      return "$" + amount.toFixed(2);
    }
     
    function sortNames(names)
    {
      function sortAlphabetically(a, b)
      {
        var left = a.toLowerCase(), right = b.toLowerCase();
        if (left > right)
          return 1;
        if (left === right)
          return 0;
        return -1;
      }
     
      // Names always sort alphabetically...right?
      names.sort(sortAlphabetically);
    }

    JavaScript’s historical i18n support is poor

    i18n-aware formatting in traditional JS uses the various toLocaleString() methods. The resulting strings contained whatever details the implementation chose to provide: no way to pick and choose (did you need a weekday in that formatted date? is the year irrelevant?). Even if the proper details were included, the format might be wrong e.g. decimal when percentage was desired. And you couldn’t choose a locale.

    As for sorting, JS provided almost no useful locale-sensitive text-comparison (collation) functions. localeCompare() existed but with a very awkward interface unsuited for use with sort. And it too didn’t permit choosing a locale or specific sort order.

    These limitations are bad enough that — this surprised me greatly when I learned it! — serious web applications that need i18n capabilities (most commonly, financial sites displaying currencies) will box up the data, send it to a server, have the server perform the operation, and send it back to the client. Server roundtrips just to format amounts of money. Yeesh.

    A new JS Internationalization API

    The new ECMAScript Internationalization API greatly improves JavaScript’s i18n capabilities. It provides all the flourishes one could want for formatting dates and numbers and sorting text. The locale is selectable, with fallback if the requested locale is unsupported. Formatting requests can specify the particular components to include. Custom formats for percentages, significant digits, and currencies are supported. Numerous collation options are exposed for use in sorting text. And if you care about performance, the up-front work to select a locale and process options can now be done once, instead of once every time a locale-dependent operation is performed.

    That said, the API is not a panacea. The API is “best effort” only. Precise outputs are almost always deliberately unspecified. An implementation could legally support only the oj locale, or it could ignore (almost all) provided formatting options. Most implementations will have high-quality support for many locales, but it’s not guaranteed (particularly on resource-constrained systems such as mobile).

    Under the hood, Firefox’s implementation depends upon the International Components for Unicode library (ICU), which in turn depends upon the Unicode Common Locale Data Repository (CLDR) locale data set. Our implementation is self-hosted: most of the implementation atop ICU is written in JavaScript itself. We hit a few bumps along the way (we haven’t self-hosted anything this large before), but nothing major.

    The Intl interface

    The i18n API lives on the global Intl object. Intl contains three constructors: Intl.Collator, Intl.DateTimeFormat, and Intl.NumberFormat. Each constructor creates an object exposing the relevant operation, efficiently caching locale and options for the operation. Creating such an object follows this pattern:

    var ctor = "Collator"; // or the others
    var instance = new Intl[ctor](locales, options);

    locales is a string specifying a single language tag or an arraylike object containing multiple language tags. Language tags are strings like en (English generally), de-AT (German as used in Austria), or zh-Hant-TW (Chinese as used in Taiwan, using the traditional Chinese script). Language tags can also include a “Unicode extension”, of the form -u-key1-value1-key2-value2..., where each key is an “extension key”. The various constructors interpret these specially.

    options is an object whose properties (or their absence, by evaluating to undefined) determine how the formatter or collator behaves. Its exact interpretation is determined by the individual constructor.

    Given locale information and options, the implementation will try to produce the closest behavior it can to the “ideal” behavior. Firefox supports 400+ locales for collation and 600+ locales for date/time and number formatting, so it’s very likely (but not guaranteed) the locales you might care about are supported.

    Intl generally provides no guarantee of particular behavior. If the requested locale is unsupported, Intl allows best-effort behavior. Even if the locale is supported, behavior is not rigidly specified. Never assume that a particular set of options corresponds to a particular format. The phrasing of the overall format (encompassing all requested components) might vary across browsers, or even across browser versions. Individual components’ formats are unspecified: a short-format weekday might be “S”, “Sa”, or “Sat”. The Intl API isn’t intended to expose exactly specified behavior.

    Date/time formatting

    Options

    The primary options properties for date/time formatting are as follows:

    weekday, era
    "narrow", "short", or "long". (era refers to typically longer-than-year divisions in a calendar system: BC/AD, the current Japanese emperor’s reign, or others.)
    month
    "2-digit", "numeric", "narrow", "short", or "long"
    year
    day
    hour, minute, second
    "2-digit" or "numeric"
    timeZoneName
    "short" or "long"
    timeZone
    Case-insensitive "UTC" will format with respect to UTC. Values like "CEST" and "America/New_York" don’t have to be supported, and they don’t currently work in Firefox.

    The values don’t map to particular formats: remember, the Intl API almost never specifies exact behavior. But the intent is that "narrow", "short", and "long" produce output of corresponding size — “S” or “Sa”, “Sat”, and “Saturday”, for example. (Output may be ambiguous: Saturday and Sunday both could produce “S”.) "2-digit" and "numeric" map to two-digit number strings or full-length numeric strings: “70” and “1970”, for example.

    The final used options are largely the requested options. However, if you don’t specifically request any weekday/year/month/day/hour/minute/second, then year/month/day will be added to your provided options.

    Beyond these basic options are a few special options:

    hour12
    Specifies whether hours will be in 12-hour or 24-hour format. The default is typically locale-dependent. (Details such as whether midnight is zero-based or twelve-based and whether leading zeroes are present are also locale-dependent.)

    There are also two special properties, localeMatcher (taking either "lookup" or "best fit") and formatMatcher (taking either "basic" or "best fit"), each defaulting to "best fit". These affect how the right locale and format are selected. The use cases for these are somewhat esoteric, so you should probably ignore them.

    Locale-centric options

    DateTimeFormat also allows formatting using customized calendaring and numbering systems. These details are effectively part of the locale, so they’re specified in the Unicode extension in the language tag.

    For example, Thai as spoken in Thailand has the language tag th-TH. Recall that a Unicode extension has the format -u-key1-value1-key2-value2.... The calendaring system key is ca, and the numbering system key is nu. The Thai numbering system has the value thai, and the Chinese calendaring system has the value chinese. Thus to format dates in this overall manner, we tack a Unicode extension containing both these key/value pairs onto the end of the language tag: th-TH-u-ca-chinese-nu-thai.

    For more information on the various calendaring and numbering systems, see the full DateTimeFormat documentation.

    Examples

    After creating a DateTimeFormat object, the next step is to use it to format dates via the handy format() function. Conveniently, this function is a bound function: you don’t have to call it on the DateTimeFormat directly. Then provide it a timestamp or Date object.

    Putting it all together, here are some examples of how to create DateTimeFormat options for particular uses, with current behavior in Firefox.

    var msPerDay = 24 * 60 * 60 * 1000;
     
    // July 17, 2014 00:00:00 UTC.
    var july172014 = new Date(msPerDay * (44 * 365 + 11 + 197));

    Let’s format a date for English as used in the United States. Let’s include two-digit month/day/year, plus two-digit hours/minutes, and a short time zone to clarify that time. (The result would obviously be different in another time zone.)

    var options =
      { year: "2-digit", month: "2-digit", day: "2-digit",
        hour: "2-digit", minute: "2-digit",
        timeZoneName: "short" };
    var americanDateTime =
      new Intl.DateTimeFormat("en-US", options).format;
     
    print(americanDateTime(july172014)); // 07/16/14, 5:00 PM PDT

    Or let’s do something similar for Portuguese — ideally as used in Brazil, but in a pinch Portugal works. Let’s go for a little longer format, with full year and spelled-out month, but make it UTC for portability.

    var options =
      { year: "numeric", month: "long", day: "numeric",
        hour: "2-digit", minute: "2-digit",
        timeZoneName: "short", timeZone: "UTC" };
    var portugueseTime =
      new Intl.DateTimeFormat(["pt-BR", "pt-PT"], options);
     
    // 17 de julho de 2014 00:00 GMT
    print(portugueseTime.format(july172014));

    How about a compact, UTC-formatted weekly Swiss train schedule? We’ll try the official languages from most to least popular to choose the one that’s most likely to be readable.

    var swissLocales = ["de-CH", "fr-CH", "it-CH", "rm-CH"];
    var options =
      { weekday: "short",
        hour: "numeric", minute: "numeric",
        timeZone: "UTC", timeZoneName: "short" };
    var swissTime =
      new Intl.DateTimeFormat(swissLocales, options).format;
     
    print(swissTime(july172014)); // Do. 00:00 GMT

    Or let’s try a date in descriptive text by a painting in a Japanese museum, using the Japanese calendar with year and era:

    var jpYearEra =
      new Intl.DateTimeFormat("ja-JP-u-ca-japanese",
                              { year: "numeric", era: "long" });
     
    print(jpYearEra.format(july172014)); // 平成26年

    And for something completely different, a longer date for use in Thai as used in Thailand — but using the Thai numbering system and Chinese calendar. (Quality implementations such as Firefox’s would treat plain th-TH as th-TH-u-ca-buddhist-nu-latn, imputing Thailand’s typical Buddhist calendar system and Latin 0-9 numerals.)

    var options =
      { year: "numeric", month: "long", day: "numeric" };
    var thaiDate =
      new Intl.DateTimeFormat("th-TH-u-nu-thai-ca-chinese", options);
     
    print(thaiDate.format(july172014)); // ๒๐ 6 ๓๑

    Calendar and numbering system bits aside, it’s relatively simple. Just pick your components and their lengths.

    Number formatting

    Options

    The primary options properties for number formatting are as follows:

    style
    "currency", "percent", or "decimal" (the default) to format a value of that kind.
    currency
    A three-letter currency code, e.g. USD or CHF. Required if style is "currency", otherwise meaningless.
    currencyDisplay
    "code", "symbol", or "name", defaulting to "symbol". "code" will use the three-letter currency code in the formatted string. "symbol" will use a currency symbol such as $ or £. "name" typically uses some sort of spelled-out version of the currency. (Firefox currently only supports "symbol", but this will be fixed soon.)
    minimumIntegerDigits
    An integer from 1 to 21 (inclusive), defaulting to 1. The resulting string is front-padded with zeroes until its integer component contains at least this many digits. (For example, if this value were 2, formatting 3 might produce “03”.)
    minimumFractionDigits, maximumFractionDigits
    Integers from 0 to 20 (inclusive). The resulting string will have at least minimumFractionDigits, and no more than maximumFractionDigits, fractional digits. The default minimum is currency-dependent (usually 2, rarely 0 or 3) if style is "currency", otherwise 0. The default maximum is 0 for percents, 3 for decimals, and currency-dependent for currencies.
    minimumSignificantDigits, maximumSignificantDigits
    Integers from 1 to 21 (inclusive). If present, these override the integer/fraction digit control above to determine the minimum/maximum significant figures in the formatted number string, as determined in concert with the number of decimal places required to accurately specify the number. (Note that in a multiple of 10 the significant digits may be ambiguous, as in “100” with its one, two, or three significant digits.)
    useGrouping
    Boolean (defaulting to true) determining whether the formatted string will contain grouping separators (e.g. “,” as English thousands separator).

    NumberFormat also recognizes the esoteric, mostly ignorable localeMatcher property.

    Locale-centric options

    Just as DateTimeFormat supported custom numbering systems in the Unicode extension using the nu key, so too does NumberFormat. For example, the language tag for Chinese as used in China is zh-CN. The value for the Han decimal numbering system is hanidec. To format numbers for these systems, we tack a Unicode extension onto the language tag: zh-CN-u-nu-hanidec.

    For complete information on specifying the various numbering systems, see the full NumberFormat documentation.

    Examples

    NumberFormat objects have a format function property just as DateTimeFormat objects do. And as there, the format function is a bound function that may be used in isolation from the NumberFormat.

    Here are some examples of how to create NumberFormat options for particular uses, with Firefox’s behavior. First let’s format some money for use in Chinese as used in China, specifically using Han decimal numbers (instead of much more common Latin numbers). Select the "currency" style, then use the code for Chinese renminbi (yuan), grouping by default, with the usual number of fractional digits.

    var hanDecimalRMBInChina =
      new Intl.NumberFormat("zh-CN-u-nu-hanidec",
                            { style: "currency", currency: "CNY" });
     
    print(hanDecimalRMBInChina.format(1314.25)); // ¥ 一,三一四.二五

    Or let’s format a United States-style gas price, with its peculiar thousandths-place 9, for use in English as used in the United States.

    var gasPrice =
      new Intl.NumberFormat("en-US",
                            { style: "currency", currency: "USD",
                              minimumFractionDigits: 3 });
     
    print(gasPrice.format(5.259)); // $5.259

    Or let’s try a percentage in Arabic, meant for use in Egypt. Make sure the percentage has at least two fractional digits. (Note that this and all the other RTL examples may appear with different ordering in RTL context, e.g. ٤٣٫٨٠٪ instead of ٤٣٫٨٠٪.)

    var arabicPercent =
      new Intl.NumberFormat("ar-EG",
                            { style: "percent",
                              minimumFractionDigits: 2 }).format;
     
    print(arabicPercent(0.438)); // ٤٣٫٨٠٪

    Or suppose we’re formatting for Persian as used in Afghanistan, and we want at least two integer digits and no more than two fractional digits.

    var persianDecimal =
      new Intl.NumberFormat("fa-AF",
                            { minimumIntegerDigits: 2,
                              maximumFractionDigits: 2 });
     
    print(persianDecimal.format(3.1416)); // ۰۳٫۱۴

    Finally, let’s format an amount of Bahraini dinars, for Arabic as used in Bahrain. Unusually compared to most currencies, Bahraini dinars divide into thousandths (fils), so our number will have three places. (Again note that apparent visual ordering should be taken with a grain of salt.)

    var bahrainiDinars =
      new Intl.NumberFormat("ar-BH",
                            { style: "currency", currency: "BHD" });
     
    print(bahrainiDinars.format(3.17)); // د.ب.‏ ٣٫١٧٠

    Collation

    Options

    The primary options properties for collation are as follows:

    usage
    "sort" or "search" (defaulting to "sort"), specifying the intended use of this Collator. (A search collator might want to consider more strings equivalent than a sort collator would.)
    sensitivity
    "base", "accent", "case", or "variant". This affects how sensitive the collator is to characters that have the same “base letter” but have different accents/diacritics and/or case. (Base letters are locale-dependent: “a” and “ä” have the same base letter in German but are different letters in Swedish.) "base" sensitivity considers only the base letter, ignoring modifications (so for German “a”, “A”, and “ä” are considered the same). "accent" considers the base letter and accents but ignores case (so for German “a” and “A” are the same, but “ä” differs from both). "case" considers the base letter and case but ignores accents (so for German “a” and “ä” are the same, but “A” differs from both). Finally, "variant" considers base letter, accents, and case (so for German “a”, “ä, “ä” and “A” all differ). If usage is "sort", the default is "variant"; otherwise it’s locale-dependent.
    numeric
    Boolean (defaulting to false) determining whether complete numbers embedded in strings are considered when sorting. For example, numeric sorting might produce "F-4 Phantom II", "F-14 Tomcat", "F-35 Lightning II"; non-numeric sorting might produce "F-14 Tomcat", "F-35 Lightning II", "F-4 Phantom II".
    caseFirst
    "upper", "lower", or "false" (the default). Determines how case is considered when sorting: "upper" places uppercase letters first ("B", "a", "c"), "lower" places lowercase first ("a", "c", "B"), and "false" ignores case entirely ("a", "B", "c"). (Note: Firefox currently ignores this property.)
    ignorePunctuation
    Boolean (defaulting to false) determining whether to ignore embedded punctuation when performing the comparison (for example, so that "biweekly" and "bi-weekly" compare equivalent).

    And there’s that localeMatcher property that you can probably ignore.

    Locale-centric options

    The main Collator option specified as part of the locale’s Unicode extension is co, selecting the kind of sorting to perform: phone book (phonebk), dictionary (dict), and many others.

    Additionally, the keys kn and kf may, optionally, duplicate the numeric and caseFirst properties of the options object. But they’re not guaranteed to be supported in the language tag, and options is much clearer than language tag components. So it’s best to only adjust these options through options.

    These key-value pairs are included in the Unicode extension the same way they’ve been included for DateTimeFormat and NumberFormat; refer to those sections for how to specify these in a language tag.

    Examples

    Collator objects have a compare function property. This function accepts two arguments x and y and returns a number less than zero if x compares less than y, 0 if x compares equal to y, or a number greater than zero if x compares greater than y. As with the format functions, compare is a bound function that may be extracted for standalone use.

    Let’s try sorting a few German surnames, for use in German as used in Germany. There are actually two different sort orders in German, phonebook and dictionary. Phonebook sort emphasizes sound, and it’s as if “ä”, “ö”, and so on were expanded to “ae”, “oe”, and so on prior to sorting.

    var names =
      ["Hochberg", "Hönigswald", "Holzman"];
     
    var germanPhonebook = new Intl.Collator("de-DE-u-co-phonebk");
     
    // as if sorting ["Hochberg", "Hoenigswald", "Holzman"]:
    //   Hochberg, Hönigswald, Holzman
    print(names.sort(germanPhonebook.compare).join(", "));

    Some German words conjugate with extra umlauts, so in dictionaries it’s sensible to order ignoring umlauts (except when ordering words differing only by umlauts: schon before schön).

    var germanDictionary = new Intl.Collator("de-DE-u-co-dict");
     
    // as if sorting ["Hochberg", "Honigswald", "Holzman"]:
    //   Hochberg, Holzman, Hönigswald
    print(names.sort(germanDictionary.compare).join(", "));

    Or let’s sort a list Firefox versions with various typos (different capitalizations, random accents and diacritical marks, extra hyphenation), in English as used in the United States. We want to sort respecting version number, so do a numeric sort so that numbers in the strings are compared, not considered character-by-character.

    var firefoxen =
      ["FireFøx 3.6",
       "Fire-fox 1.0",
       "Firefox 29",
       "FÍrefox 3.5",
       "Fírefox 18"];
     
    var usVersion =
      new Intl.Collator("en-US",
                        { sensitivity: "base",
                          numeric: true,
                          ignorePunctuation: true });
     
    // Fire-fox 1.0, FÍrefox 3.5, FireFøx 3.6, Fírefox 18, Firefox 29
    print(firefoxen.sort(usVersion.compare).join(", "));

    Last, let’s do some locale-aware string searching that ignores case and accents, again in English as used in the United States.

    // Comparisons work with both composed and decomposed forms.
    var decoratedBrowsers =
      [
       "A\u0362maya",  // A͢maya
       "CH\u035Brôme", // CH͛rôme
       "FirefÓx",
       "sAfàri",
       "o\u0323pERA",  // ọpERA
       "I\u0352E",     // I͒E
      ];
     
    var fuzzySearch =
      new Intl.Collator("en-US",
                        { usage: "search", sensitivity: "base" });
     
    function findBrowser(browser)
    {
      function cmp(other)
      {
        return fuzzySearch.compare(browser, other) === 0;
      }
      return cmp;
    }
     
    print(decoratedBrowsers.findIndex(findBrowser("Firêfox"))); // 2
    print(decoratedBrowsers.findIndex(findBrowser("Safåri")));  // 3
    print(decoratedBrowsers.findIndex(findBrowser("Ãmaya")));   // 0
    print(decoratedBrowsers.findIndex(findBrowser("Øpera")));   // 4
    print(decoratedBrowsers.findIndex(findBrowser("Chromè")));  // 1
    print(decoratedBrowsers.findIndex(findBrowser("IË")));      // 5

    Odds and ends

    It may be useful to determine whether support for some operation is provided for particular locales, or to determine whether a locale is supported. Intl provides supportedLocales() functions on each constructor, and resolvedOptions() functions on each prototype, to expose this information.

    var navajoLocales =
      Intl.Collator.supportedLocalesOf(["nv"], { usage: "sort" });
    print(navajoLocales.length > 0
          ? "Navajo collation supported"
          : "Navajo collation not supported");
     
    var germanFakeRegion =
      new Intl.DateTimeFormat("de-XX", { timeZone: "UTC" });
    var usedOptions = germanFakeRegion.resolvedOptions();
    print(usedOptions.locale);   // de
    print(usedOptions.timeZone); // UTC

    Legacy behavior

    The ES5 toLocaleString-style and localeCompare functions previously had no particular semantics, accepted no particular options, and were largely useless. So the i18n API reformulates them in terms of Intl operations. Each method now accepts additional trailing locales and options arguments, interpreted just as the Intl constructors would do. (Except that for toLocaleTimeString and toLocaleDateString, different default components are used if options aren’t provided.)

    For brief use where precise behavior doesn’t matter, the old methods are fine to use. But if you need more control or are formatting or comparing many times, it’s best to use the Intl primitives directly.

    Conclusion

    Internationalization is a fascinating topic whose complexity is bounded only by the varied nature of human communication. The Internationalization API addresses a small but quite useful portion of that complexity, making it easier to produce locale-sensitive web applications. Go use it!

    (And a special thanks to Norbert Lindenberg, Anas El Husseini, Simon Montagu, Gary Kwong, Shu-yu Guo, Ehsan Akhgari, the people of #mozilla.de, and anyone I may have forgotten [sorry!] who provided feedback on this article or assisted me in producing and critiquing the examples. The English and German examples were the limit of my knowledge, and I’d have been completely lost on the other examples without their assistance. Blame all remaining errors on me. Thanks again!)

  3. Videos and Firefox OS

    Before HTML5

    Those were dark times Harry, dark times – Rubeus Hagrid

    Before HTML5, displaying video on the Web required browser plugins and Flash.

    Luckily, Firefox OS supports HTML5 video so we don’t need to support these older formats.

    Video support on the Web

    Even though modern browsers support HTML5, the video formats they support vary:

    In summary, to support the most browsers with the fewest formats you need the MP4 and WebM video formats (Firefox prefers WebM).

    Multiple sizes

    Now that you have seen what formats you can use, you need to decide on video resolutions, as desktop users on high speed wifi will expect better quality videos than mobile users on 3G.

    At Rormix we decided on 720p for desktop, 360p for mobile connections, and 180p specially for Firefox OS to reduce the cost in countries with high data charges.

    There are no hard and fast rules — it depends on who your market audience is.

    Streaming?

    The best streaming solution would be to automatically serve the user different videos sizes depending on their connection status (adaptive streaming) but support for this technology is poor.

    HTTP live streaming works well on Apple devices, but has poor support on Android.

    At the time of writing, the most promising technology is MPEG DASH, which is an international standard.

    In summary, we are going to have to wait before we get an adaptive streaming technology that is widely accepted (Firefox does not support HLS or MPEG DASH).

    DIY Adaptive streaming

    In the absence of adaptive streaming we need to try to work out the best video quality to load at the outset. The following is a quick guide to help you decide:

    Wifi or 3G

    Using a certified Firefox OS app you can check to see if the user is on wifi or not.

    var lock    = navigator.mozSettings.createLock();
    var setting = lock.get('wifi.enabled');
     
    setting.onsuccess = function () {
      console.log('wifi.enabled: ' + setting.result);
    }
     
    setting.onerror = function () {
      console.warn('An error occured: ' + setting.error);
    }

    https://developer.mozilla.org/en-US/docs/Web/API/Settings_API

    There is some more information at the W3C Device API.

    Detecting screen size

    There is no point sending a 720p video to a user with a screen smaller than 720p. There are many ways to get the different bounds of a user’s screen; innerWidth and width allow you to get a good idea:

    function getVidSize()
    {
      //Get the width of the phone (rotation independent)
      var min = Math.min($(window).innerHeight(),$(window).innerWidth());
      //Return a video size we have
      if(min < 320)      return '180';
      else if(min < 550) return '360';
      else               return '720';
    }

    http://www.quirksmode.org/m/tests/widthtest.html

    Determining internet speed

    It is difficult to get an accurate read of a user’s internet speed using web technologies — usually they involve loading a large image onto the user’s device and timing it. This has the disadvantage of having to send more data to the user. Some services such as: http://speedof.me/api.html exist, but still require data downloads to the user’s device. (Stackoverflow has some more options.)

    You can be slightly more clever by using HTML5, and checking the time it takes between the user starting the video and a set amount of the video loading. This way we do not need to load any extra data on the user’s device. A quick VideoJS example follows:

    var global_speedcount = 0;
    var global_video = null;
    global_video = videojs("video", {}, function(){
    //Set up video sources
    });
     
    global_video.on('play',function(){
      //User has clicked play
      global_speedcount = new Date().getTime();
    });
     
    function timer()
    {
      var diff = new Date().getTime() - global_speedcount;
      //Remove this handler as it is run multiple times per second!
      global_video.off('timeupdate',timer);
    }
     
    global_video.on('timeupdate',timer);

    This code starts timing when the user clicks play, and when the browser starts to play the video it sends timing information to timeupdate. You can also use this function to detect if lots of buffering is happening.

    Detect high resolution devices

    One final thing to determine is whether or not a user has a high pixel density screen. In this case even if they have a small screen it can still have a large number of pixels (and therefore require a higher resolution video).

    Modernizr has a plugin for detecting hi-res screens.

    if (Modernizr.highresdisplay)
    {
      alert('Your device has a high resolution screen');
    }

    WebP Thumbnails

    Not to get embroiled in an argument, but at Rormix we have seen an average decrease of 30% in file size (WebP vs JPEG) with no loss of quality (in some cases up to 50% less). And in countries with expensive data plans, the less data the better.

    We encode all of our thumbnails in multiple resolutions of WebP and send them to every device that supports them to reduce the amount of data being sent to the user.

    Mobile considerations

    If you are playing HTML5 videos on mobile devices, their behavior differs. On iOS it automatically goes to full screen on iPhones/iPods, but not on tablets.

    Some libraries such as VideoJS have removed the controls from mobile devices until their stability increases.

    Useful libraries

    There are a few useful HTML5 video libraries:

    Mozilla links

    Mozilla has some great articles on web video:

    Other useful Links

  4. Mozilla Introduces the First Browser Built For Developers: Firefox Developer Edition

    Developers are critical to the continued success of the Web. The content and apps they create compel us to come back to the Web every day, whether on a computer or mobile phone.

    In celebration of the 10th anniversary of Firefox, we’re excited to unveil Firefox Developer Edition, the first browser created specifically for developers.

    Ten years ago, we built Firefox for early adopters and developers to give them more choice and control. Firefox integrated WebAPIs and Add-ons to enable people to get the most out of the Web. Now we’re giving developers the whole browser as a hard-hat area, allowing us to bring front and center the features most relevant to them. Having a dedicated developer browser means we can tailor the browsing experience to what developers do every day.

    Because Firefox is part of an open-source, independent community and not part of a proprietary ecosystem, we’re able to offer features other browsers can’t by applying our tools everywhere the Web goes, regardless of platform or device.

    One of the biggest pain points for developers is having to use numerous siloed development environments in order to create engaging content or for targeting different app stores. For these reasons, developers often end up having to bounce between different platforms and browsers, which decreases productivity and causes frustration.

    Firefox Developer Edition solves this problem by creating a focal point to streamline your development workflow. It’s a stable developer browser which is not only a powerful authoring tool but also robust enough for everyday browsing. It also adds new features that simplify the process of building for the entire Web, whether targeting mobile or desktop across many different platforms.

    If you’re an experienced developer, you’ll already be familiar with the installed tools so you can focus on developing your content or app as soon as you open the browser. There’s no need to download additional plugins or applications to debug mobile devices. If you’re a new Web developer, the streamlined workflow and the fact that everything is already set up and ready to go makes it easier to get started building sophisticated applications.

    So what’s under the hood?

    The first thing you’ll notice is the distinctive dark design running through the browser. We applied the developer tools theme to the entire browser. It’s trim and sharp and focused on saving space for the content on your screen. It also fits in with the darker look common among creative app development tools.

    We’ve also integrated two powerful new features, Valence and WebIDE that improve workflow and help you debug other browsers and apps directly from within Firefox Developer Edition.

    Valence (previously called Firefox Tools Adapter) lets you develop and debug your app across multiple browsers and devices by connecting the Firefox dev tools to other major browser engines. Valence also extends the awesome tools we’ve built to debug Firefox OS and Firefox for Android to the other major mobile browsers including Chrome on Android and Safari on iOS. So far these tools include our Inspector, Debugger and Console and Style Editor.

    WebIDE allows you to develop, deploy and debug Web apps directly in your browser, or on a Firefox OS device. It lets you create a new Firefox OS app (which is just a web app) from a template, or open up the code of an existing app. From there you can edit the app’s files. It’s one click to run the app in a simulator and one more to debug it with the developer tools.

    Firefox Developer Edition also includes all the tools experienced Web developers are familiar with, including:

    • Responsive Design Mode – see how your website or Web app will look on different screen sizes without changing the size of your browser window.
    • Page Inspector- examine the HTML and CSS of any Web page and easily modify the structure and layout of a page.
    • Web Console – see logged information associated with a Web page and use Web Console and interact with a Web page using JavaScript.
    • JavaScript Debugger – step through JavaScript code and examine or modify its state to help track down bugs.
    • Network Monitor – see all the network requests your browser makes, how long each request takes and details of each request.
    • Style Editor – view and edit CSS styles associated with a Web page, create new ones and apply existing CSS stylesheets to any page.
    • Web Audio Editor – inspect and interact with Web Audio API in real time to ensure that all audio nodes are connected in the way you expect.

    Give it a try and let us know what you think. We’re keen to hear your feedback.

    More Information:

  5. NFC in Firefox OS

    Firefox OS is being developed in an open collaboration with Mozilla’s partners and community. In that spirit, and over the course of over a year, Mozilla and Deutsche Telekom (DT) teams worked closely together to develop a platform-level support for NFC within Firefox OS. During that time, both teams had regular product and engineering meet-ups for the end-to-end development cycle.

    From proposing the NFC API, to defining the overall architecture, to prototyping and completing a production-level implementation on shipping products, this collaboration model worked so well that it really helped showcase the power of the “open” (Open technology and Open contribution model) for pushing the web forward. After all, this is exactly what Mozilla and Firefox OS stand for.

    In this post, we describe a few basics around Firefox OS NFC implementation.

    NFC Roadmap

    Currently in release 2.0, Firefox OS supports NFC based sharing of contents (contacts, images, videos, URLs), as well as wirelessly reading information stored in NFC enabled tags (tag reading). Our sharing use cases are compatible with NFC enabled devices from other OSes like Android/Windows, so sharing of these contents across these devices would work. Our NFC API (first introduced in v1.3) has been put to use for these sharing use cases in v2.0 with core apps.

    The Overall B2G roadmap is available on the wiki.

    WebNFC API

    The Firefox NFC APIs allow Peer to Peer (P2P) communication between any 2 devices that support NFC Data Type Exchange format (NDEF). NFC passive tags that can present themselves as NDEF compatible tags can also be read and written to. Firefox OS’ NFC implementation is currently for certified applications only, but as stated above, will be opened to marketplace applications as the API is developed to cover more use cases and data formats.

    An example using this API

    The following does P2P communications between 2 NFC devices (from the NFC API docs on MDN):

    // Utility Function for UTF-8 string conversion to Uint8Array.
    // Or ideally, simply add this to your webapp HTML to use NfcUtils:
    // &lt;script defer src="shared/js/nfc_utils.js"&gt;&lt;/script&gt;
    function fromUTF8(str) {
      if (!str) {
        return null;
      }
      var enc = new TextEncoder('utf-8');
      return enc.encode(str);
    }
     
    var tnf     = 1;                                             // NFC Forum Well Known type
    var type    = new Uint8Array(fromUTF8("U"));                 // URL type
    var id      = new Uint8Array(fromUTF8(""));                  // id
    var payload = new Uint8Array(fromUTF8("u0003mozilla.org")); // URL data, with a record prefix 0x3 replacing http://
     
    var ndefRecords = [new MozNDEFRecord(tnf, type, id, payload)];
    var nfcdom = window.navigator.mozNfc;
     
    nfcdom.onpeerready = function(event) {
      // event.detail is a session token
      var nfcPeer = navigator.mozNfc.getNFCPeer(event.detail);
      var req = nfcpeer.sendNDEF(ndefRecords); // push NDEF message to other NFC device.
      req.onsuccess = function(e) {
        console.log("Successfully pushed P2P message");
      };
      req.onerror = function(e) {
        console.log("P2P push failed!");
      };
    };

    More such examples that ship with Firefox OS can be found in Using the NCF API.

    Current Supported data types

    The WebNFC API currently supports NFC Data Exchange Format (NDEF). There are some future plans for Non-NDEF types. From the example above, it is 4 fields, which is defined with 3 optional Uint8Array data types. The TNF and type are used to route the message to the appropriate registered web application(s).

    (Source: http://git.mozilla.org/?p=releases/gecko.git;a=blob_plain;f=dom/webidl/MozNDEFRecord.webidl;hb=refs/heads/v2.0)

    [Constructor(octet tnf, optional Uint8Array type, optional Uint8Array id, optional Uint8Array payload)]
    interface MozNDEFRecord
    {
      /**
       * Type Name Field (3-bits) - Specifies the NDEF record type in general.
       *   tnf_empty: 0x00
       *   tnf_well_known: 0x01
       *   tnf_mime_media: 0x02
       *   tnf_absolute_uri: 0x03
       *   tnf_external type: 0x04
       *   tnf_unknown: 0x05
       *   tnf_unchanged: 0x06
       *   tnf_reserved: 0x07
       */
      [Constant]
      readonly attribute octet tnf;
     
      /**
       * type - Describes the content of the payload. This can be a mime type.
       */
      [Constant]
      readonly attribute Uint8Array? type;
     
      /**
       * id - Identifier is application dependent.
       */
      [Constant]
      readonly attribute Uint8Array? id;
     
      /**
       * payload - Binary data blob. The meaning of this field is application
       * dependent.
       */
      [Constant]
      readonly attribute Uint8Array? payload;
    };

    Note, in upcoming Firefox OS releases, we will be updating the data types slightly to make TNF an enum type instead of an octet.

    Mozilla’s Flame device supports NFC, more devices coming

    Our Flame device supports NFC and we are expecting more commercial devices from our partners soon. Flame device supports NFC chipset from NXP (PN547C2).

    Videos

    Here is a demo video of some of the NFC sharing features based on Firefox OS:

    Core Apps In Flame device that use NFC:

    • Gallery
    • Video
    • Music
    • Settings
    • System browser

    A sample 3rd party App

    Here is an app that Mozillian Dietrich Ayala put together using the NFC tag reading API. BikeCommute is an app that registers an NFC tag to track bike commuters at the Mozilla Portland office. The app is running on a Nexus 4 with Firefox OS 2.2, and is built with Famo.us for UI and PouchDB for data storage and syncing to a remote CouchDB. Currently, the app just reads the user’s email address from a text record written to the tag.

    The next version will add support for running the app on users’ phones, using a local contact (user) instead of a plain text record, and being able to configure the NFC tag from their own device. The plan is to develop leaderboards from the CouchDB data and Mozillians.org integration so we can deploy and compete with other offices and Mozillians everywhere! The source code is available on GitHub and pull requests welcome!

    Here is a Video demo of this app in action:

    More NFC documentation

    So, there it is!

    We are really excited to introduce this new addition to growing list of APIs and features in Firefox OS! We hope developers will take full advantage of all that NFC enables by way of device-to-device sharing and also services like contactless payment planned in future.

    When can developers start using this API?

    Currently this API is available for certified apps. We can’t wait to finish the work to make this API available for privileged apps, so all of you developers can take advantage of this. If you wish to follow along or jump in and help out, feel free to track Bug 1042851. We are targeting to finish the work for the next release v2.2.

    Next in NFC

    In upcoming releases, with the help of our partners, we are focusing on expanding the NFC coverage for supporting Secure elements and services like NFC based payments. More on that in a separate post later. Please stay tuned.

    Here’s to the open web!

  6. New on MDN: Sign in with Github!

    MDN now gives users more options for signing in!

    Sign in with GitHub

    Signing in to MDN previously required a Mozilla Persona account. Getting a Persona account is free and easy, but MDN analytics showed a steep drop-off at the “Sign in with Persona” interface. For example, almost 90% of signed-out users who clicked “Edit” never signed in, which means they never got to edit. That’s a lot of missed opportunities!

    It should be easy to join and edit MDN. If you click “Edit,” we should make it easy for you to edit. Our analysis demonstrated that most potential editors stumbled at the Persona sign in. So, we looked for ways to improve sign in for potential contributors.

    Common sense suggests that many developers have a GitHub account, and analysis confirms it. Of the MDN users who list external accounts in their profiles, approximately 30% include a GitHub account. GitHub is the 2nd-most common external account listed, after Twitter.

    That got us thinking: If we integrated GitHub accounts with MDN profiles, we could one day share interesting GitHub activity with each other on MDN. We could one day use some of GitHub’s tools to create even more value for MDN users. Most immediately, we could offer “sign in with GitHub” to at least 30% (but probably more) of MDN’s existing users.

    And if we did that, we could also offer “sign in with GitHub” to over 3 million GitHub users.

    The entire engineering team and MDN community helped make it happen.

    Authentication Library

    Adding the ability to authenticate using GitHub accounts required us to extend the way MDN handles authentication so that MDN users can start to add their GitHub accounts without effort. We reviewed the current code of kuma (the code base that runs MDN) and realized that it was deeply integrated with how Mozilla Persona works technically.

    As we’re constantly trying to remove technical debt that meant revisiting some of the decisions we’ve made years ago when the code responsible for authentication was written. After a review process we decided to replace our home-grown system, django-browserid, with a 3rd party library called django-allauth as it is a well known system in the Django community that is able to use multiple authentication providers side-by-side – Mozilla Persona and GitHub in our case.

    One challenge was making sure that our existing user database could be ported over to the new system to reduce the negative impact on our users. To our surprise this was not a big problem and could be automated with a database migration–a special piece of code that would convert the data into the new format. We implemented the new authentication library and migrated accounts to it several months ago. MDN has been using django-allauth for Mozilla Persona authentication since then.

    UX Challenges

    We wanted our users to experience a fast and easy sign-up process with the goal of having them edit MDN content at the end. Some things we did in the interface to support this:

    • Remember why the user is signing up and return them to that task when sign up is complete.
    • Pre-fill the username and email address fields with data from GitHub (including pre-checking if they are available).
    • Trust GitHub as a source of confirmed email address so we do not have to confirm the email address before the user can complete signing up.
    • Standardise our language (this is harder than it sounds). Users on MDN “sign in” to their “MDN profile” by connecting “accounts” on other “services”. See the discussion.

    One of our biggest UX challenges was allowing existing users to sign in with a new authentication provider. In this case, the user needs to “claim” an existing MDN profile after signing in with a new service, or needs to add a new sign-in service to their existing profile. We put a lot of work into making sure this was easy both from the user’s profile if they signed in with Persona first and from the sign-up flow if they signed in with GitHub first.

    We started with an ideal plan for the UX but expected to make changes once we had a better understanding of what allauth and GitHub’s API are capable of. It was much easier to smooth the kinks out of the flow once we were able to click around and try it ourselves. This was facilitated by the way MDN uses feature toggles for testing.

    Phased Testing & Release

    This project could potentially corrupt profile or sign-in data, and changes one of our most essential interfaces – sign up and sign in. So, we made a careful release plan with several waves of functional testing.

    We love to alpha- and beta-test changes on MDN with feature toggles. To toggle features we use the excellent django-waffle feature-flipper by James Socol – MDN Manager Emeritus.

    We deployed the new code to our MDN development environment every day behind a feature toggle. During this time MDN engineers exercised the new features heavily, finding and filing bugs under our master tracking bug.

    When the featureset was relatively complete, we created our beta test page, toggled the feature on our MDN staging environment for even more review. We did the end-to-end UX testing, invited internal Mozilla staff to help us beta test, filed a lot of UX bugs, and started to triage and prioritize launch blockers.

    Next, we started an open beta by posting a site-wide banner on the live site, inviting anyone to test and file bugs. 365 beta testers participated in this round of QA. We also asked Mozilla WebQA to help deep-dive into the feature on our stage server. We only received a handful of bugs, which gave us great confidence about a final release.

    Launch

    It was a lot of work, but all the pieces finally came together and we launched. Because of our extensive testing & release plan, we’ve 0 incidents with the launch – no down-time, no stacktraces, no new bugs reported. We’re very excited to release this feature. We’re excited to give more options and features to our incredible MDN users and contributors, and we’re excited to invite each and every GitHub user to join the Mozilla Developer Network. Together we can make the web even more awesome. Sign in now.

    Outlook

    Now that we have worked out the infrastructure and UX challenges associated with multi-account authentication, we can look for other promising authentication services to integrate with. For example, Firefox Accounts (FxA) is the authentication service that powers Firefox Sync. FxA is integrated with Firefox and will soon be integrated with a variety of other Mozilla services. As more developers sign up for Firefox Accounts, we will look for opportunities to add it to our authentication options.

  7. WebIDE, Storage inspector, jQuery events, iframe switcher + more – Firefox Developer Tools Episode 34

    A new set of Firefox Developer Tools features has just been uplifted to the Aurora channel. These features are available right now in Aurora, and will be in the Firefox 34 release in November. This release brings new tools (storage inspector, WebIDE), an updated profiler, and handy enhancements to the existing tools:

    WebIDE

    WebIDE, a new tool for in-browser app development, has been enabled by default in this release. WebIDE lets you create a new Firefox OS app (which is just a web app) from a template, or open up the code for an already created app. From there you can edit the app’s files. It’s one click to run the app in a simulator and one more to debug it with the developer tools. Open WebIDE from Firefox’s “Web Developer” menu. (docs)

    Storage inspector

    There’s a new panel that shows the data your page has stored in cookies, localStorage, sessionStorage, and IndexedDB, which was created mostly by Girish Shama. Enable the Storage panel by checking off Settings > “Default Developer Tools” > “Storage”. The panel is read-only right now, with editing ability planned for a future release. (docs) (development notes) (UserVoice request)

    storage inspector

    jQuery events

    The event listener popup in the Inspector now supports jQuery. This means the popup will display the function you attached with e.g. jQuery.on(), and not the jQuery wrapper function itself. See this post for more info and how to add support for your preferred framework. (development notes)

    JQuery events

    Iframe switcher

    Change the frame you’re debugging using the new frame selection menu. Selecting a frame will switch all of the tools to debug that iframe, including the Inspector, Console, and Debugger. Add the frame selection button by checking off Settings > “Available Toolbox Buttons” > “Select an iframe”. (docs) (development notes)(UserVoice request)

    iframe selection

    Updated profiler

    An updated JavaScript profiler appears in the new “Performance” tab (formerly the “Profiler” tab). New to the profiler are a frame rate timeline and categories for frames like “network” and “graphics”. (docs) (development notes)

    new profiler

    console.table()

    Add a call to console.table() anywhere in your JavaScript to log data to the console using a table-like display. Log any object, array, Map, or Set. Sort a column in the table by clicking on its header. (docs) (development notes)

    console.table

    Selector preview

    Hover over a CSS selector in the Inspector or Style Editor to highlight all the nodes that match that selector on the page. (development notes)

    selector previews

    Other mentions

    • Persistent split console – The split console (opened by pressing ESC) will now open with the tools if you had it open the last time the tools were closed. (development notes)
    • Web audio – AudioParam connections – the Web Audio Editor now displays connections from AudioNodes to AudioParams. (development notes)

    Special thanks to the 41 contributors that added all the features and fixes in this release.

    Comment here, shoot feedback to @FirefoxDevTools on Twitter, or propose changes on the Developer Tools feedback channel. If you’d like to help out, check out the guide to getting involved.

  8. 350 posts on Hacks in 2 years!

    Two years ago, we made a number of changes to the Mozilla Hacks blog. Since then we’ve had over three million unique visitors and 350 quality posts in just less than two years – almost one every second day!

    Part of these changes included:

    • A clear focus on learning about the Open Web & open source – more detail in What Mozilla Hacks is
    • A dedicated Editor, me, working with ensuring consistency, quality & versatility of the articles
    • Articles covering both interesting technologies and possibilities but also learning lessons of how to build exciting solutions and work with a number of different technical opportunities
    • To be a credible and independent go-to resource for developers

    Testimonials

    I’ve gathered a few testimonials from authors behind the most popular articles during this time, with their experiences writing for Mozilla Hacks.

    And you know what? You could write for Hacks too!

    If you have experience with a great solution or idea for the Open Web/open source, just let me know and Mozilla can help you share it with a lot of developers out there! Send an e-mail to robert [at] mozilla [dot] com or talk to me via @robertnyman on Twitter and we’ll talk!

    Here are the thoughts from Peter Cooper of HTML5 Weekly & JavaScript Weekly, Dave Camp, Director of Firefox Developer Tools and Thorben Bochenek from Opera on writing for Mozilla Hacks.

    Austin Hallock from Clay.io:

    My co-founder Zoli and I have both written articles for Mozilla Hacks and we agree that it was a very worthwhile experience. The process of getting our posts on the site was easy and both articles generated more response than we expected. Mozilla has a fantastic mission, and the Mozilla Hacks blog is a great extension of that.

    Their articles:

    Also, thanks to the authors on Mozilla Hacks for their great contributions! And those in that page are only authors with 2 posts and more – in total we have 230 authors all time who have published a post on Hacks!

    Most popular content

    During these two years, these are the most popular posts, written by Mozilla staff:

    The most popular posts, written by 3rd party developers/writers:

    Read & write

    If you like our articles, please spread the word about them! Ask your friends to read too and follow @mozhacks on Twitter for the latest articles and announcements.

    And if you have experiences and learnings to share, let us know and help you with sharing that knowledge!

  9. Enabling Voice Input into the Open Web and Firefox OS

    With the advent of smartphones triggered by iPhone in 2007, Touch became the primary mode of input for interacting with these devices. And now with the advent of wearables (and other hands-free technologies that existed before), Voice is becoming another key method of input. The possibilities of experiences Voice Input enables are huge, to say the least.

    They go beyond just merely interacting with in-vehicle devices, accessories and wearables. Just think of the avenues Voice Input opens up for bringing more technologies to more people. It’s enormous: Accessibility, Literacy, Gaming, VR and the list goes on. There is a social vibe there that definitely resonates with our mission at Mozilla detailed in our Mozilla Manifesto

    How it started

    Both current leading mobile OS/ecosystem providers of today- Apple & Google have their native experiences with Siri and “OK Google” (coupled with Google Now). We really needed an effort to enable Voice Input into the first ecosystem that existed – the open Web. Around MWC 2013 in Barcelona, when Desigan Chinniah introduced me to André Natal – Firefox contributor from Brazil, we had a conversation around this and we instantly agreed to do something about this in whichever way possible. Andre told me about being inspired from a talk by Brendan Eich in BrazilJS, so I did not have much convincing to do. :-)

    First steps

    We had numerous calls and meetings over the past year on the approach and tactics around this. Since “code wins arguments”, the basic work started in parallel with Firefox desktop and FxOS Unagi devices, later switching to Mozilla Flame devices over time. Over a period of the past year, we had several meetings with Mozilla engineering leads on exact approach and decided to break this effort into several smaller phases (“baby steps”).

    The first target was getting Web Speech API implemented, and getting acoustic/language modules integrated with a decoder and giving that a try. Lots of similar minded folks in Mozilla Engineering/QA & community helped along with guidance and code-reviews while Andre moonlighted (on top of his day job) with a very high focus. Things moved fast in past month or so. (Well, to be honest, the only day this effort slowed down was when Team Brazil lost to Germany in FIFA 2014. :-)) Full credit to André for his hard work!

    Where are we?

    Our current thinking is to get a grammar-based (limited commands) app working first and distribute it in our rich & diverse international Mozilla community for accent-based testing and enhancements. Once we have this stablilized, we will get into the phase 2 where we can focus more on natural language processing and get closer to a virtual assistant experience sometime in future that can give users voice based answers. There is lots of work to do there and we are just beginning.

    I will save the rest of the details for later and jump to the current status this month. Where are we so far?

    We now have the Web Speech API ready for testing and we have a couple demos for you to see!

    Desktop: Firefox Nightly on Mac

    Editor’s note: for full effect, start playing the two above videos at the same time.

    Firefox OS demo

    Come, Join us!

    If you want to follow along, please look at the SpeechRTC – Speech enabling the open web wiki and Bug 1032964 – Enabling Voice input in Firefox OS.

    So jump in and help out if you can. We need all of you (and your voices). Remember “Many Voices, One Mozilla”!

  10. PhoneGap Developer App preview for Firefox OS

    The cross-platform team at Mozilla is always looking for ways to improve how developers build apps with open web standards. We consider Cordova and PhoneGap to be great tools to achieve that. We are excited to work on improving support for PhoneGap by getting the PhoneGap Developer App into Firefox OS.

    The PhoneGap Developer App allows you to easily run PhoneGap apps in multiple platforms, without the need to install SDKs or have a developer subscription. The app is available for Android, iOS and Windows Phone at their respective app stores. You can pair your app with multiple devices at a time and any changes you make to your app’s code get propagated to all devices.

    Even though developing for Firefox OS does not require a beefy SDK or a developer subscription, being able to have your changes propagate to multiple devices at once is pretty cool!

    We have started working on support for Firefox OS and you can try it out today. In this post we’re going through the steps needed to get the preview version of the PhoneGap developer app in Firefox OS. As they say, a video is worth 1k images:

    Getting the dependencies

    We will be using a development version of Cordova combined with a development version of the PhoneGap command line interface to get it all working. Luckily, they are very compatible and work well together. You will need to have git, nodejs and npm installed before proceeding.

    Let’s first get all modules we’ll need from github:

      $ git clone https://github.com/apache/cordova-firefoxos.git
      $ git clone https://github.com/apache/cordova-cli.git
      $ git clone https://github.com/apache/cordova-lib.git
      $ git clone -b fxos https://github.com/rodms10/phonegap-app-developer.git
      $ git clone -b fxos https://github.com/rodms10/connect-phonegap.git
      $ git clone https://github.com/phonegap/phonegap-cli.git

    Now let’s set dependencies up:

      $ cd connect-phonegap
      $ npm link
      $ cd ../phonegap-cli
      $ npm link connect-phonegap
      $ npm install
      $ cd ../cordova-lib/cordova-lib
      $ npm link
      $ cd ../../cordova-cli
      $ npm link cordova-lib
      $ npm install
      $ cd ..

    Load the app to your device

    The app is available at phonegap-app-developer/platforms/firefoxos/www/, just point the app manager or webIDE to this path and load it to your device or simulator. Once you start the app, you should see a screen with an IP address. That’s where you enter the address of your server.

    Starting the server

    Let’s get the server going. It will serve your app’s content to the PhoneGap Developer App. Create a new Cordova app:

      $ cordova-cli/bin/cordova create myapp org.app.my "I Heart PhoneGap Dev App"
      $ cd myapp

    Now we need to point cordova to the development version of cordova-firefoxos. In your root app folder, myapp in our case, create a folder named .cordova (with the leading dot) and add a file named config.json to the new folder with the following contents:

    {
      "lib": {
        "firefoxos": {
          "uri": "/&lt;Full/Path/To&gt;/cordova-firefoxos",
          "version": "dev",
          "id": "cordova-firefoxos-dev"
        }
      }
    }

    Make sure you have the correct full path to cordova-firefoxos in the uri property above.

    Time to start the server, we’ll switch back to PhoneGap for that:

      $ ../phonegap-cli/bin/phonegap.js serve

    You should see a line saying [phonegap] listening on 10.0.0.1:3000. Enter that IP in the PhoneGap Developer App and see your app start running there! Easy.

    Ship It!

    If you got this far, you’re probably asking why isn’t the app in the marketplace already! As you saw, to get all up an running we required the development version of the PhoneGap command line interface. Once a new version with our new code is released, we can publish the app and not require development version of the command line.

    If you have any questions leave a comment, find us at the #apps channel on IRC or send us an email at mozilla-cordova@mozilla.org.