App Manager

The App Manager continues to be a priority for the Devtools team and with Firefox 28, in addition to many minor fixes and improvements the team has added two new major features to assist in mobile development. These two features consist of an additional manifest editor and an embedded toolbox.

The Manifest Editor allows a developer to edit their app manifest directly. Using this editor you can add, edit, or delete fields in your app’s manifest.webapp and save them back to disk. This feature provides the developer an integrated experience, as the user never has to leave the tools while debugging and improving an application.

Also new to the App Manager is the embedded toolbox. Now when you debug your app (either using the Simulator or on device) the toolbox will be opened in the App Manager in a separate vertical ‘tab’ that is easily distinguished by the app’s icon:

Web Console

The biggest change in the Web Console is a feature we call ‘split console’; the web console can now be summoned from any other tool. Using split console mode is easy – if you’re working with any other tool and need access to the console, simply hit the Escape key or click on the button to toggle the split console, as you can see in this short screencast:

This isn’t the only improvement to the console though. In addition to split console we’ve also made the following changes:

• CSS warnings are off by default – CSS warnings can often add hundreds of messages to the console as a page loads.
• The console will now offer autocomplete suggestions for the current scope the debugger is in.
• Message timestamps can be toggled on/off (the default is off)
• console.exception() and console.assert() have been added to the console api.
• We added a dark theme for the console

Debugger

In the debugger we’ve added two key features that developers have been asking for. First off we’ve added the ability to pretty-print minified JS files:

This feature is particularly handy if you minify your JavaScript or need to step through a minified library served from a remote site – just click on the minification button (picture above) to get a much more nicely formatted version of the file.

In addition to this we’ve added a feature to the debugger that allows you to inspect the value of a variable while debugging. For any variable, simply hover over it or click on it to bring up a pop-up that displays the current value:

The pop-up allows you to inspect not only simple types like the boolean picture above, but also objects and DOM nodes:

Inspector

Following along from the addition of various tooltips to the Inspector in Firefox 27, we’ve added some additional features in the inspector:

• A Color picker tooltip in the rule view
• We updated style the style of tooltips to better match the dark and light themes
• The display timeout for tooltips was tuned based on your feedback

Here’s a screenshot to give you an idea what the color-picker tooltip looks like:

Wrapping up

That’s a huge list, but there were a couple of additional changes I’d like to call out as well:

• The Network Monitor now has a clear button, which should be of particular use to developers creating apps that make heavy use of web APIs:

  

• As a bonus for Add-on or Platform hackers, we’ve expanded the functionality of the Browser Debugger and re-named it the Browser Toolbox. When you open the toolbox you’ll see a full range of tools including not only the debugger but also the Console, Inspector, Style Editor, Profiler, Network Monitor and Scratchpad:

Browser add-ons are clearly an important web browser feature, at least on the desktop platform, and for a long time Firefox was the browser add-on authors’ preferred target. When Google launched Chrome, this trend on the desktop browsers domain was pretty clear, so their browser provides an add-on api as well.

Most of the Web DevTools we are used to are now directly integrated into our browser, but they were add-ons not so long time ago, and it’s not strange that new web developer tools are born as add-ons.

Web DevTools (integrated or add-ons) can motivate web developers to change their browser, and then web developers can push web users to change theirs. So, long story short, it would be interesting and useful to create cross-browser add-ons, especially web devtools add-ons (e.g. to preserve the web neutrality).

With this goal in mind, I chose Ember Inspector as the target for my cross-browser devtool add-ons experiment, based on the following reasons:

• It belongs to an emerging and interesting web devtools family (web framework devtools)
• It’s a pretty complex / real world Chrome extension
• It’s mostly written in the same web framework by its own community
• Even if it is a Chrome extension, it’s a webapp built from the app sources using grunt
• Its JavaScript code is organized into modules and Chrome-specific code is mostly isolated in just a couple of those

Plan & Run Porting Effort

Looking into the ember-extension git repository, we see that the add-on is built from its sources using grunt:



The extension communicates between the developer tools panel, the page and the main extension code via message passing:



Using this knowledge, planning the port to Firefox was surprisingly easy:

• Create new Firefox add-on specific code (register a devtool panel, control the inspected tab)
• Polyfill the communication channel between the ember_debug module (that is injected into the inspected tab) and the devtool ember app (that is running in the devtools panel)
• Polyfill the missing non-standard inspect function, which open the DOM Inspector on a DOM Element selected by a defined Ember View id
• Minor tweaks (isolate remaining Chrome and Firefox specific code, fix CSS -webkit prefixed rules)

In my opinion this port was particularly pleasant to plan thanks to two main design choices:

• Modular JavaScript sources which helps to keep browser specific code encapsulated into replaceable modules
• Devtool panel and code injected into the target tab collaborate exchanging simple JSON messages and the protocol (defined by this add-on) is totally browser agnostic

Most of the JavaScript modules which compose this extension were already browser independent, so the first step was to bootstrap a simple Firefox Add-on and register a new devtool panel.

Create a new panel into the DevTools is really simple, and there’s some useful docs about the topic in the Tools/DevToolsAPI page (work in progress).

Register / unregister devtool panel

From https://github.com/tildeio/ember-extension/blob/master/dist_firefox/lib/main.js

Devtool panel definition

From https://github.com/tildeio/ember-extension/blob/master/dist_firefox/lib/devtool-panel.js#L26

Then, moving to the second step, adapt the code used to create the message channels between the devtool panel and injected code running in the target tab, using content scripts and the low level content worker from the Mozilla Add-on SDK, which are well documented on the official guide and API reference:

• Content Script Guide
• SDK/content/workers API



• DevTool Panel Content Script
• ember_extension.js FirefoxAdapter
• TargetTab Content Script
• ember_debug.js FirefoxAdapter

DevTool Panel Workers

From https://github.com/tildeio/ember-extension/blob/master/dist_firefox/lib/devtool-panel.js

Inject ember_debug

From https://github.com/tildeio/ember-extension/blob/master/dist_firefox/lib/devtool-panel.js

Finally hook browser specific code needed to activate the DOM Inspector on a defined DOM Element:

Inspect DOM element request handler

From https://github.com/tildeio/ember-extension/blob/master/dist_firefox/lib/devtool-panel.js#L178

Evaluate its features and dive into the exchanged messages

At this point one could wonder: how much useful is a tool like this?, Do I really need it? etc.

I must admit that I’ve started and completed this port without being an experienced EmberJS developer, but to be able to check if all the original features were working correctly on Firefox and to really understand how this browser add-on helps EmberJS developers during app development/debugging phases (its most important use cases), I’ve started to experiment with EmberJS and I have to say that EmberJS is a very pleasant framework to work with and Ember Inspector is a really important tool to put into our tool belts.

I’m pretty sure that every medium or large sized JavaScript framework need this kind of DevTool; clearly it will never be an integrated one, because it’s framework-specific and we will get used to this new family of DevTool Add-ons from now on.

List Ember View, Model Components and Routes

The first use case is being able to immediately visualize Routes, Views/Components, Models and Controllers our EmberJS app instantiate for us, without too much webconsole acrobatics.

So its immediately available (and evident) when we open its panel on an EmberJS Apps active in the current browser tab:



Using these tables we can then inspect all the properties (even computed ones) defined by us or inherited from the ember classes in the actual object hierarchy.

Using an approach very similar to the Mozilla Remote Debugging Protocol from the integrated DevTools infrastructure (e.g. even when we use devtools locally, they exchange JSON messages over a pipe), the ember_debug component injected into the target tab sends the info it needs about the instantiated EmberJS objects to the devtool panel component, each identified by internally generated reference IDs (similar to the grips concept from the Mozilla Remote Debugging Protocol.



Logging the exchanged messages, we can learn more about the protocol.

Receive updates about EmberJS view tree info (EmberDebug -> DevtoolPanel):

Request inspect object (DevtoolPanel -> EmberDebug):

Receive updates about the requested Object info (DevtoolPanel -> EmberDebug):

Reach every EmberJS object in the hierarchy from the webconsole

A less evident but really useful feature is “sendToConsole”, to be able to reach any object/property that we can inspect from the webconsole, from the tables described above.

When we click the >$E link, which is accessible in the right split panel:



The ember devtool panel asks to ember_debug to put the defined object/property into a variable accessible globally in the target tab and named $E, then we can switch to the webconsole and interact with it freely:



Request send object to console (DevtoolPanel -> EmberDebug):

Much more

These are only some of the feature already present in the Ember Inspector and more features are coming in its upcoming versions (e.g. log and inspect Ember Promises).

If you already use EmberJS or if you are thinking about trying it, I suggest you to give Ember Inspector a try (on both Firefox or Chrome, if you prefer), it will turn inspecting your EmberJS webapp into a fast and easy task.

Integrate XPI building into the grunt-based build process

The last challenge in the road to a Firefox add-on fully integrated into the ember-extension build workflow was xpi building for an add-on based on the Mozilla Add-on SDK integrated into the grunt build process:

Chrome crx extensions are simply ZIP files, as are Firefox XPI add-ons, but Firefox add-ons based on the Mozilla Add-on SDK needs to be built using the cfx tool from the Add-on SDK package.

If we want more cross-browser add-ons, we have to help developers to build cross-browser extensions using the same approach used by ember-extension: a webapp built using grunt which will run into a browser add-on (which provides glue code specific to the various browsers supported).

So I decided to move the grunt plugin that I’ve put together to integrate Add-on SDK common and custom tasks (e.g. download a defined Add-on SDK release, build an XPI, run cfx with custom parameters) into a separate project (and npm package), because it could help to make this task simpler and less annoying.



Build and run Ember Inspector Firefox Add-on using grunt:

Following are some interesting fragments from grunt-mozilla-addon-sdk integration into ember-extension (which are briefly documented in the grunt-mozilla-addon-sdk repo README):

Integrate grunt plugin into npm dependencies: package.json

Define and use grunt shortcut tasks: Gruntfile.js

Configure grunt-mozilla-addon-sdk tasks options

Conclusion

Especially thanks to the help from the EmberJS/EmberInspector community and its maintainers, Ember Inspector Firefox add-on is officially merged and integrated in the automated build process, so now we can use it on Firefox and Chrome to inspect our EmberJS apps!

Stable:

• Firefox (from AMO): https://addons.mozilla.org/en-US/firefox/addon/ember-inspector/
• Chrome (from Chrome WebStore): https://chrome.google.com/webstore/detail/ember-inspector/bmdblncegkenkacieihfhpjfppoconhi

Latest Build

• Firefox: http://ember-extension.s3.amazonaws.com/xpi-latest.xpi
• Chrome: grunt-mozilla-addon-sdk

In this article we’ve briefly dissected an interesting pattern to develop cross-browser devtools add-ons, and introduced a grunt plugin that simplifies integration of Add-on SDK tools into projects built using grunt: https://npmjs.org/package/grunt-mozilla-addon-sdk

Thanks to the same web first approach Mozilla is pushing in the Apps domain creating cross-browser add-ons is definitely simpler than what we thought, and we all win :-)

Happy Cross-Browser Extending,
Luca

]]> https://hacks.mozilla.org/2013/12/ember-inspector-on-a-firefox-near-you-2/feed/ 0 https://hacks.mozilla.org/2013/12/application-layout-with-css3-flexible-box-module/ https://hacks.mozilla.org/2013/12/application-layout-with-css3-flexible-box-module/#comments Mon, 16 Dec 2013 18:31:50 +0000 https://hacks.mozilla.org/?p=23361 It has become very easy to create fluid application layouts thanks to the CSS3 Flexible Box Layout Module. In this article we are going to implement a simple application layout, which fills the whole screen, resizes with the browser window and comes with the additional bonus of a draggable splitter.

Instead of the classic <div> elements let’s also use some HTML5 structural tags. This will not only make the code more semantic, but also more convenient to work with, since we can directly address the elements with a CSS type selector without having to rely on id attributes or parent-child relationships.

Take a look at the complete demo to see how it works.

First step: Add Vertical Boxes

We start with only three tags (<header>, <main> and <footer>) in the body.



<!DOCTYPE html> <html> <head> <title>CSS3 Application Layout</title> </head>   <body> <header></header> <main></main> <footer></footer> </body> </html>

Let’s add the CSS to make these three elements fill the space vertically. This is achieved by setting the CSS display property of the <body> to flex and the flex-direction property to column. This tells the browser to lay out the body’s children (<header>, <main> and <footer>) as vertical flexible boxes.

How this available space is distributed can be controlled with the flex shorthand property. You can read about it on MDN. In this application layout though, we don’t want the size to shrink or expand proportionally. Instead the <header> and the <footer> element should have a fixed height, whereas the <main> should just fill the remaining space by setting its flex property to auto.

html, body { height: 100%; width: 100%; padding: 0; margin: 0; }   body { display: flex; flex-direction: column; }   header { height: 75px; }   main { flex: auto; }   footer { height: 25px; }

Show demo

Second Step: Horizontal Boxes

Let’s add three more elements (<nav>, <article> and <aside>) inside the <main> element. But this time we want them to fill the space inside the <main> element horizontally instead of vertically.



<body> <header></header> <main> <nav></nav> <article></article> <aside></aside> </main> <footer></footer> </body>

This is achieved by setting the display property of the <main> element also to flex, but the flex-direction property to row (this is the default). The <nav> and <aside> element should have a fixed width, while the <article> should just fill the remaining space: this is achieved in the same kind of manner as before:

main { display: flex; flex-direction: row; flex: auto; }   nav { width: 150px; }   article { flex: auto; }   aside { width: 50px; }

Show demo

Thats all. Resize your browser window and enjoy the flexible application layout.

Next step: CSS Refinements

But wait. When there is a lot of content, an element can become smaller than specified and also scrollbars can appear.



Therefore, we need to add a min-width property to all elements where we added a width property. We also should set the overflow property of the <body> and the <main> element to hidden as well as the overflow-y of <article> and <aside> to auto to only show scrollbars where we want them.

body { overflow: hidden; display: flex; flex-direction: column; }   header { height: 75px; min-height: 75px; }   footer { height: 25px; min-height: 25px; }   main { display: flex; flex-direction: row; flex: auto; border: solid grey; border-width: 1px 0; overflow: hidden; }   nav { width: 150px; min-width: 150px; }   article { border: solid grey; border-width: 0 0 0 1px; flex: auto; overflow-x: hidden; overflow-y: auto; }   aside { width: 50px; min-width: 50px; overflow-x: hidden; overflow-y: auto; }

Note: This does probably not work in Safari yet. You might be able to get it to work by using the -webkit- prefix.

Final Step: Throw a little JavaScript into the Mix

As the final step, we want the user to be able to resize the <aside> element when dragging it with the mouse. For that we add a <div> element as a splitter, which will serve as the drag handle.

<body> <header></header> <main> <nav></nav> <article></article> <div class="splitter"></div> <aside></aside> </main> <footer></footer> </body>

We set the width of the handle to 4px and give the cursor attribute a value of col-resize, to show the user that this element can be resized East-West.

.splitter { border-left: 1px solid grey; width: 4px; min-width: 4px; cursor: col-resize; }

All what’s left now is to add a little JavaScript, that enables moving the splitter.

var w = window, d = document, splitter;   splitter = { lastX: 0, leftEl: null, rightEl: null,   init: function(handler, leftEl, rightEl) { var self = this;   this.leftEl = leftEl; this.rightEl = rightEl;   handler.addEventListener('mousedown', function(evt) { evt.preventDefault(); /* prevent text selection */   self.lastX = evt.clientX;   w.addEventListener('mousemove', self.drag); w.addEventListener('mouseup', self.endDrag); }); },   drag: function(evt) { var wL, wR, wDiff = evt.clientX - splitter.lastX;   wL = d.defaultView.getComputedStyle(splitter.leftEl, '').getPropertyValue('width'); wR = d.defaultView.getComputedStyle(splitter.rightEl, '').getPropertyValue('width'); wL = parseInt(wL, 10) + wDiff; wR = parseInt(wR, 10) - wDiff; splitter.leftEl.style.width = wL + 'px'; splitter.rightEl.style.width = wR + 'px';   splitter.lastX = evt.clientX; },   endDrag: function() { w.removeEventListener('mousemove', splitter.drag); w.removeEventListener('mouseup', splitter.endDrag); } };   splitter.init(d.getElementsByClassName('splitter')[0], d.getElementsByTagName('article')[0], d.getElementsByTagName('aside')[0]);

Note: For some reason the resizing doesn’t work in IE11 (, Safari?) or Chrome 31. It seems that it has something to do with the display: flex; property value.

]]> https://hacks.mozilla.org/2013/12/application-layout-with-css3-flexible-box-module/feed/ 3 https://hacks.mozilla.org/2013/12/css-variables-in-firefox-nightly/ https://hacks.mozilla.org/2013/12/css-variables-in-firefox-nightly/#comments Sun, 15 Dec 2013 05:22:03 +0000 https://hacks.mozilla.org/?p=23483 As reported by Cameron McCormack, Firefox Nightly (version 29) now supports CSS variables. You can get a quick overview in this short screencast:

You can define variables in a context with a var- prefix and then implement them using the var() instruction. For example:

:root { var-companyblue: #369; var-lighterblue: powderblue; }   h1 { color: var(companyblue); } h2 { color: var(lighterblue); }

<h1>Header on page</h1> <h2>Subheader on page</h2>

This defines the two variables companyblue and lighterblue for the root element of the document which results in (you can try it here using Firefox Nightly):



Variables are scoped, which means you can overwrite them:

:root { var-companyblue: #369; var-lighterblue: powderblue; } .partnerbadge { var-companyblue: #036; var-lighterblue: #cfc; }   h1 { color: var(companyblue); } h2 { color: var(lighterblue); }

<h1>Header on page</h1> <h2>Subheader on page</h2>   <div class="partnerbadge"> <h1>Header on page</h1> <h2>Subheader on page</h2> </div>

Using these settings, headings inside an element with a class of partnerbadge will now get the other blue settings:



Variables can be any value you want to define and you can use them like any other value, for example inside a calc() calculation. You can also reset them to other values, for example inside a media query. This example shows many of these possibilities.

:root { var-companyblue: #369; var-lighterblue: powderblue; var-largemargin: 20px; var-smallmargin: calc(var(largemargin) / 2); var-borderstyle: 5px solid #000; var-headersize: 24px; } .partnerbadge { var-companyblue: #036; var-lighterblue: #369; var-headersize: calc(var(headersize)/2); transition: 0.5s; }   @media (max-width: 400px) { .partnerbadge { var-borderstyle: none; background: #eee; } }   /* Applying the variables */ body {font-family: 'open sans', sans-serif;}   h1 { color: var(companyblue); margin: var(largemargin) 0; font-size: var(headersize); } h2 { color: var(lighterblue); margin: var(smallmargin) 0; font-size: calc(var(headersize) - 5px); }   .partnerbadge { padding: var(smallmargin) 10px; border: var(borderstyle); }



Try resizing the window to less than 400 pixels to see the mediaQuery change in action.

An initial implementation of CSS Variables has just landed in Firefox Nightly, which is currently at version 29 and after the February 3 merge, in Firefox Aurora. There are still a few parts of the specification which still need to be supported before the can go into the release cycle of Firefox Beta and Firefox. Cameron has the details on that:

The only part of the specification that has not yet been implemented is the CSSVariableMap part, which provides an object that behaves like an ECMAScript Map, with get, set and other methods, to get the values of variables on a CSSStyleDeclaration. Note however that you can still get at them in the DOM by using the getPropertyValue and setProperty methods, as long as you use the full property names such as "var-theme-colour-1".

The work for this feature was done in bug 773296, and my thanks to David Baron for doing the reviews there and to Emmanuele Bassi who did some initial work on the implementation. If you encounter any problems using the feature, please file a bug!

For now, have fun playing with CSS variables in Nightly and tell us about issues you find.

]]> https://hacks.mozilla.org/2013/12/css-variables-in-firefox-nightly/feed/ 39 https://hacks.mozilla.org/2013/12/monster-madness-creating-games-on-the-web-with-emscripten/ https://hacks.mozilla.org/2013/12/monster-madness-creating-games-on-the-web-with-emscripten/#comments Thu, 12 Dec 2013 15:00:31 +0000 https://hacks.mozilla.org/?p=23253

When our engineering teams at Trendy Entertainment & Nom Nom Games decided on the strategy of developing one of our new Unreal Engine 3 games — Monster Madness Online — as a cross-platform title, we knew that a frictionless multiplayer web browser version would be central to this experience. The big question, however, was determining what essential technologies to utilize in order to bring our game onto the web. As a C++ oriented developer, we determined quickly that rewriting the game engine from the ground-up was out of the question. We’d need a solution that would allow us to port our existing code in an efficient manner into a format usable in the browser…

TL;DR? Watch the video!


Playing the Field

We looked hard at the various options in front of us: FlasCC (a GCC Flash compiler), Google’s NaCl, a custom native C++ extension, or Mozilla’s Emscripten & asm.js.

In our tests, Flash ran slowly and had inconsistent behaviors between Pepper (Chrome) and Adobe’s plugin version. Combined with the increasingly onerous plugin requirement, we opted to look elsewhere for a more seamless, forward-thinking approach.

NaCl had the issue of requiring a walled-garden distribution site that would separate us from direct contact with our users, and also being processor-specific. pNaCL eliminated the walled-garden requirement and added dynamic code compilation support, but still had the issues of being processor-specific code necessitating in our view device-specific testing, and a potentially long startup time as the code would be linked on first run. Finally, only working in Chrome would be a dealbreaker for our desire to have our game run in all major browsers.

A custom plugin/extension with C++ would require lots of testing & maintenance efforts on our part to run across different browsers, processor architectures, and operating systems, and such an installation requirement would likely scare away many potential players.

As it turned out, for our team’s purposes the Emscripten compiler & asm.js proved to be the best solution to these challenges, and when combined with a set of other new-ish Web API’s, enabled the browser to become a fully featured platform for instant high-end 3D gaming. This just took a little trial & error to figure out exactly how we’d piece it together…and that’s what we’ll be reviewing here!

First Steps into a Brave New World

We Trendy game engineers are primarily old-school C++ programmers, so it was something of a revelation that Emscripten could compile our existing application (built on Epic Game’s Unreal Engine 3) into asm.js optimized Javascript with little to no changes.

The primary Unreal Engine 3-specific code tweaks that were necessary to get the project to compile & run with Emscripten, were essentially… 1, 2, 3:

1. // Esmcripten needs 4 byte alignment FNameEntry* Allocate( INT Size ) { #if EMSCRIPTEN Size = Align( Size, 4 ); #endif   ...... }   2. // Script execution: llvm needs aligned data #if EMSCRIPTEN #define XFER(T) { T Temp;   if (!Ar.IsLoading()) { appMemcpy(&Temp, &Script(iCode), sizeof(T)); }   Ar << Temp;   if (!Ar.IsSaving()) { appMemcpy(&Script(iCode), &Temp, sizeof(T)); }   iCode += sizeof(T); } #else #define XFER(T) { Ar << *(T*)&Script(iCode); iCode += sizeof(T); } #endif   3. // This function needs to synchronously complete IO requests for single-threaded Emscripten IO to work, so added a ServiceRequestsSynchronously() to the end of it which flushes & blocks till the IO request finishes. FAsyncIOSystemBase::QueueIORequest()

No really, that was about it! Within a day of fiddling with it, we had our game’s Javascript ‘executable’ compiled & running in the browser. Crashing, due to not having a graphics API implememented — but running with log output! Thankfully, we already had Unreal Engine3’s OpenGL ES2 version of the rendering subsystem ready to utilize, so porting the renderer to WebGL only took another day.

WebGL appeared to essentially have a superset of features compared to OpenGL ES2, so the shaders and methods used matched up by simply changing some API calls. In fact, we were able to do improvements by making use of WebGL’s floating point render targets for certain postprocessing effects, such as edge outlining and dynamic shadows.


Postprocessing makes everything prettier!

But how’s it run?

Now we had something rendering in the browser, and with a quick change to capture input, we could start playing the game and analyzing its performance. What we found was very encouraging: straight ‘out of the box’, in Firefox the asm.js version of the game was getting nearly 33% of the performance of the native executable. And this was comparing a single-threaded web application to the multi-threaded native executable (so really, not a fair comparison! ;). This was about 2x the performance we saw with our quick Flash port (which we still utilize as a fallback for older browsers that don’t yet support asm.js, though we eventually hope to deprecate entirely).

Its performance in Chrome was less astonishing, more towards 20% of native performance, but still within our target margins: namely, can it run on a 2011-model Macbook Air at 45-60 FPS (with Vsync disabled)? The answer, thankfully, was yes. We hope Google will continue to improve the performance of asm.js on their browser over time. But as it currently stands, we believe unless you’re making the browser version of ‘Crysis’ with this tech (which may not be far off), it seems you have enough performance even in Chrome to do most kinds of web games.


60 FPS on an old Macbook Air

Putting the Pieces into Place

So within a week from starting, we had turned our Unreal Engine 3 PC game into a well-running, graphically-rich web game. But where to take it from here? Well, it’d still need: Audio, Networking, Streaming, and Storage. Let’s discuss the various techniques used for each of these systems.

Audio

This was a no-brainer, as there is only really one robust standardized web audio system apart from Flash: WebAudio. Again, this API matched up pretty well to its mobile cousin, OpenSL, for which we already had an integration. So once we switched out the various calls, we had .

There was an apparent issue in Mac Chrome where sounds flagged “looping” would sometimes never become destroyed, so we implemented a Chrome-specific hack to manually loop the sound, and filed a bug report with Google. Ah well, one thing we’ve seen with browser API’s is there’s not a 100% guarantee that every browser will implement the functionality to perfect specification, but it gets the job done!

Networking

This proved a little trickier. First, we investigated WebRTC as used in the Bananabread demo, but WebRTC of course is for browser-to-browser communication which is actually not what we were looking to do. Our online game service uses a server-and-client architecture with centralized infrastructure, and so WebSockets is the API to utilize in that case. The tricky part is that we have to handle all the WebSockets incoming and outgoing data in JavaScript buffers, and then pass that along to the “C++” Emscripten-compiled game.

With some callbacks, this worked out, but we also had to take our UDP game server code and place the WebSockets TCP-style layer onto it — some iteration was necessary to get the packets to be formatted in exactly the way that WebSockets expects, but once we did that, our browser game was communicating with our backend-hosted Linux dedicated game servers with no problems!

Streaming & Storage

One advantage to being on the web is easy access to the browser’s asynchronous downloading functionalities to stream-in content. We certainly made use of this with our game, with the initial download clocking in at under 10 MB. Everything else streams in on-demand as you play using standard browser http download requests: Textures, Sound Effects, Music, even Skeletal Meshes and Level Packages. But the bigger question is how to reliably store this content. We don’t want to just rely on the browser cache, since it’s not good for guaranteed immediate gameplay loading as we can’t pre-query whether something exists on disk in the regular browser cache or not.

For this, we used the IndexedDB API, which lets us asynchronously save and retrieve data objects from a secure abstracted storage location. It works in both Chrome and Firefox, though it’s still finicky as the database can occasionally become corrupted (perhaps if terminated during async writes) and has to be regenerated. In the worst case, this simply results in a re-download of content the user already had received.

We’re currently looking into this issue, but that aside, IndexedDB certainly works well and has the advantage of providing our application standard file IO functionality, useful to store content that we download. (UPDATE: Firefox Nightly build as of 12/10 seems to automatically reset the IndexedDB storage if this happens and it may not recur.)

Play it Now, and Embrace the Future!

While we still have more profiling and tweaking to, as we’re just now starting to use Firefox’s VTune support to symbolically profile the asm.js performance within the browser. Even still, we’re pretty pleased with where the things currently stand. But don’t take our word for it, please try it yourselves right here, no installation or sign-up required:

Try our demo test anonymously In-browser Here!
(Please bear with us if our game servers limit access under load, we’re still testing our backend scalability!)

We at Trendy envision a day when anybody can play any game no matter where they are or what device they happen to have, without friction or gateways or middlemen. With the right combination of these cutting-edge web technologies, that day can be today. We hope other enterprising game developers will join us in reaching players directly through the web, which thanks to Emscripten & asm.js, may well become the most powerful and far-reaching “game console” of all!

]]> https://hacks.mozilla.org/2013/12/monster-madness-creating-games-on-the-web-with-emscripten/feed/ 8 https://hacks.mozilla.org/2013/12/the-side-projects-of-mozillians-jsfiddle-and-meatspac-es/ https://hacks.mozilla.org/2013/12/the-side-projects-of-mozillians-jsfiddle-and-meatspac-es/#comments Wed, 11 Dec 2013 13:22:39 +0000 https://hacks.mozilla.org/?p=23210 At Mozilla, we are happy to get the chance to work with a lot of talented people. Therefore, as an on-going series, we wanted to take the opportunity to highlight some of the exciting projects Mozillians work on in their spare time.

JSFiddle

JSFiddle is a tool to write web examples (in HTML, JavaScript and CSS) called ‘fiddles’. They can be saved and shared with others or embedded in a website which is perfect for blogs, documentation or tutorials. Created by Piotr Zalewa.



Piotr: I wanted a tool that could help me check if my frontend code was working. I was active on the MooTools scene at the time and we needed a tool to support our users who had questions about the framework and specific bugs to solve. The community is the best motivation. There are about 2,000 developers creating and watching fiddles right now! Many big projects are using JSFiddle for docs (MooTools, HighCharts) or bug requests (jQuery).

I’m always logged in on the #mootools IRC channel and one day we had a small competition to see who could be the first to answer support questions with only one line of JavaScript code. A user asked a non-trivial question which needed to be answered with both HTML and JavaScript. Our usual workflow was to write an HTML file, run it locally in the browser, copy the code to a Pastebin site then share the link. No one knew of a tool that could do this. The next day I had a prototype created in the evening and it was well accepted. The working but ugly version was completed shortly after. Oskar Krawczyk joined as a designer and the project was ready to be shown to the world.

It started as Django and MySQL on the server side with MooTools as a frontend framework. Since then the only major change was adding Memcache. Currently we run JSFiddle on 12 servers sponsored by DigitalOcean. 2 database servers, 3 application servers, 2 Memcache, then static files and development servers. I would ideally like to have the database structured in a way that would be easier to scale. The database is huge and updating tables takes a lot of time.

JSFiddle was designed in the time when most of the JavaScript libraries were running under one framework only. We want to allow users to mix frameworks and add more languages. At the moment you can write in HTML, JavaScript, Coffeescript, CSS and SCSS but I would like to support more languages. We’ve got a full hat of ideas to be implemented but I think it’s better to provide improvements than promises.

Meatspac.es

Meatspac.es is a single public channel chat app that generates animated GIFs of users from their camera once they submit a new message. Created by Jen Fong with GIF library support added by Sole Penadés.



Jen: I’ve been working on various quirky chat apps that involved some form of embedded media so this was an idea I had about getting users to interact beyond typing by posing for the camera and doing a little movement. I also really like GIFs and the fact that they work everywhere. I had been playing with WebRTC here and there and Sole was working on her RTCamera app when I thought: “Could we combine the two worlds? Chat and GIFs?”.

For the web server I used Nginx which proxies to a long running Node process using Express. The messages and GIFs are stored temporarily in LevelDB with a TTL (time-to-live) that deletes the message, including the GIFs stored as Base64 blobs, after 10 minutes. On the client-side, it uses jQuery, some GIF library files and updates with WebSockets with an AJAX fallback.

The biggest challenge of the project was surprisingly not code related! It was largely keeping up with all the craziness when a flood of people started using the chat, tweeting at me and contacting me. I first mentioned it publicly at ‘RealTimeConf’ in Portland a few weeks prior then started tweeting about it. After that a bunch of people checked it out, and someone posted it on Hacker News where even more people came (around 8,000 people on the heaviest day). It was mentioned on Twitter and various sources for a few days after.

People can be really creative during their GIF creation. It was also interesting to watch people give each other humorous ‘-bro’ nicknames; both women and men. They would always ask others what their name should be rather than giving themselves a name.

I am now working on a similar app but for one to many GIF chatting for Firefox OS called chatspaces. Anyone who is interested in contributing can watch the repository and check the README for what to contribute.

]]> https://hacks.mozilla.org/2013/12/the-side-projects-of-mozillians-jsfiddle-and-meatspac-es/feed/ 0 https://hacks.mozilla.org/2013/12/the-mozilla-developer-network-has-a-new-face/ https://hacks.mozilla.org/2013/12/the-mozilla-developer-network-has-a-new-face/#comments Mon, 09 Dec 2013 20:26:07 +0000 https://hacks.mozilla.org/?p=23284 Last summer the Mozilla Developer Network (MDN) underwent a massive platform change, moving from a hosted third-party solution to our own custom Django application code-named Kuma. That move laid the ground work for our latest major MDN upgrade: a complete front-end redesign, included many new features as well as usability and accessibility enhancements. Let me provide you with a quick overview of what you can expect to see on the new MDN and what features we’re cooking up for the future!

New MDN Features

Increased Commitment to Search

The majority of MDN users are looking to find documentation the moment they land on the MDN homepage, so we’ve placed search front and center:



We’ve also added search filters to the mix, allowing users to narrow down search results to their specific needs:



From a technical perspective, we’ve moved to Elasticsearch for search, allowing us to continue making indexing and filtering improvements, as well as add new search features at will. We anticipate fine-tuning search as we receive feedback so we’ll continue the push to get you to better documentation faster.

Ease in Navigation

Getting from document to document was a pain point in the previous design, so we’ve fixed that in two ways. The first was creating Content Zones, a method for creating navigation for a given topic. We’ve started with the most prominent parts of MDN, including App Center, Firefox, Firefox OS, Firefox Marketplace, Add-ons, and Persona:



Content Zones

MDN’s new Content Zones provide a complete collection of documentation about a given topic, encompassing the very basics of a topic to API details and advanced techniques. We’ll be kicking off with the following zones:

Firefox OS

Highlights of the Firefox OS zone include:

• A detailed Platform Guide
• Build and Install details
• Hacking Firefox OS
• App Design & Development

Firefox Marketplace

Highlights of this zone include:

• App submission and review
• App publishing and monetization
• Marketplace API information

App Center

Highlights of the App Center zone include:

• Quickstart Guide
• Design and Build tips
• App publishing guidelines
• API references

Persona

Highlights of the Persona zone include:

• Guide to using Persona on your site
• Becoming an identify provider
• Details on the Persona project

Firefox

Highlights of the Firefox zone include:

• A complete Firefox Add-on overview
• Information on Firefox internals
• Detailed instructions for building Firefox and contributing

Add-ons

Highlights of the Add-ons zone include:

• XUL extension information
• Best practice tips
• Theming
• Add-on publishing guidelines

“See Also” Links

We’ve also implemented “See Also” links which may appear in any wiki page, linking to documents which may be relevant based on the document you’re currently viewing.

Both the zone subnavigation and “See Also” link sidebar widgets are built from basic link lists in the wiki document, so adding links and shuffling navigation is easy for anyone looking to contribute to MDN. These link lists can also be built using MDN’s macro language, Kumascript, and our writing team has done a great job automating “See Also” links so that contributors can save on the manual labor of hunting down other relevant documents.

Top level navigation

In the top level navigation, you will have access to five distinct areas:

• The above-mentioned Content Zones
• Web Platform, including direct links to more information on technologies, references and guides
• Developer Program – To be able to help developers and establish long-term relationships and channels, we have created the Mozilla Developer Program. We have a lot of plans and ideas for iteratively expanding the Program, and we want you involved as we do so! So, sign up! You will get a membership, be able to subscribe to our newsletter and get access to features over time as we roll them out.
• Tools – more information on the Firefox Developer Tools and their features
• Demos, being a direct link to the Demo Studio



Enhanced Kumascript Macro Features

Kumascript, MDN’s dynamic macro language, was also outfitted with the ability to read external RSS feeds. At present MDN is using the feed reader capability to pull forum posts from StackOverflow and blog posts from the Mozilla Hacks blog. Check out the MDN:Common macro to view the fetchJSONResource and fetchHTTPResource methods which aid in displaying feed content in wiki documents.

Future Features

This visual redesign is just the beginning of our push to make MDN more dynamic and usable. The MDN development and UX teams have plenty more coming in 2014. Here are a few peeks into what you can expect to see!

Dynamic Search Filtering

To improve the efficiency in user search, we plan to implement hashtag-prefixed text filtering which may be added in the initial search — doing so will prevent the need for additional filtering when the user lands on the search results page.



Holly Habstritt Gaal has detailed this query system in detail on her blog. Check out her blog post to see implementation details.

Docs Navigator

So you’ve completed a search and you click the first link you thought would be applicable, but you want to move onward and view other results. Instead of backing out to the search results page again, the wiki page (if the user came from search) will display a doc navigator to move to the next or previous result, or you can view the entire list of results from your original search:



Just another handy way of finding what you need faster!

Demo Studio and Dev Derby Redesign

A redesign to the MDN Demo Studio and Dev Derby will be coming shortly. We have an excellent design in review and we hope to roll that out in early 2014.

If you have a suggestion or find any bugs within the new MDN, please let us know.

Look forward to more from MDN in 2014 and beyond. The MDN platform promises to expand and improve the way we view, write, and experience documentation and web technologies!

]]> https://hacks.mozilla.org/2013/12/the-mozilla-developer-network-has-a-new-face/feed/ 57 https://hacks.mozilla.org/2013/12/holiday-gaming-competition-with-mozilla-and-goo-technologies/ https://hacks.mozilla.org/2013/12/holiday-gaming-competition-with-mozilla-and-goo-technologies/#comments Mon, 09 Dec 2013 12:24:59 +0000 https://hacks.mozilla.org/?p=23282 The web has come a long way, and especially lately it is amazing what great opportunities this means for game creators. To use open technologies to reach as many users as possible across multiple web browsers and platforms.

Looking at what WebGL, Web Audio API, Emscripten, asm.js and more have brought to the table, and what we’ve covered in articles here on Mozilla Hacks about gaming, WebGL, Audio etc, we are very excited to see the progress for gaming on the web.

Thinking about all this, we wanted to find an interesting way to encourage you to see what kind of games you could create! Together with Goo Technologies, we’re bringing you Goo’s Game Creator Challenge.



You will be using the Goo platform to create your games, which consists of:

• Goo Engine – a 3D JavaScript gaming engine entirely built on WebGL/HTML5
• Goo Create – a visual editing tool running on top of the engine

Categories

There are three different categories you can enter in:

• Best Amateur Interactive Game Scene – Create a visually stunning interactive game scene using only the built in object primitives (simple 3D shapes, such as cubes, spheres, and so forth, that can be manipulated and combined together to create characters or other objects in a scene). This category is for those with little to no previous experience of game creation.
• Best Desktop Game – Create an immersive game level for the desktop browser using Goo Create in combination with the Goo Engine JavaScript API.
• Best Mobile Game – Create an immersive game level for the mobile Web using Goo Create in combination with Goo Engine JavaScript API – extra credit will be awarded for cross-platform games that also work on desktop.

Getting started

If you need inspiration to get started, a good place to start is Build an interactive scene in Goo Create or look at the Creating a Goon in Goo Create video:

Other good resources are:

• Build JetPack Santa using primitives in Goo Create
• Use Goo Create together with Goo Engine JS API for a desktop- or mobile game

Also, if you need help, you can go to the forums for Goo Technologies.

Prizes

As with any challenge, there are prizes! For the people entering the Best Amateur Interactive Game Scene category, the top 5 contributions will win:

• $1,000 in cash
• 5 year access to Goo Create Pro ($2900 value)

For the Best Desktop Game and Best Mobile Game categories, the prizes are:

• A trip for two people to the GDC Conference in San Francisco, scheduled for March 17 – 21, 2014 or to GamesCom in Cologne, August 13-17 (maximum value $10,000) or $5,000 cash, at the winner’s option.
• 5 year access to Goo Create Pro ($2900 value)
• Guest post on the Mozilla Apps blog

Judges

Judging the challenge are people from Rovio (know for Angry Birds and more), Mozilla (Brendan Eich, CTO and inventor of JavaScript, Vlad Vukicevic, Engineering Director and inventor of WebGL) and Goo Technologies (Rikard Herlitz, CTO). They will pick the winners by January 17th.

Deadline for entry

Deadline for submitting entries is Tuesday, January 14 at midnight PT. Submit your entry by sending the url of your finished scene or game to mozgoocreate@mozilla.com.

For more information:

• Read more details, guidance on the three categories and FAQ
• Read the Official Rules
• Read the Mozilla Privacy Policy and the Goo Privacy Policy
]]> https://hacks.mozilla.org/2013/12/holiday-gaming-competition-with-mozilla-and-goo-technologies/feed/ 0 https://hacks.mozilla.org/2013/12/protecting-your-firefox-os-app-code/ https://hacks.mozilla.org/2013/12/protecting-your-firefox-os-app-code/#comments Thu, 05 Dec 2013 16:05:34 +0000 https://hacks.mozilla.org/?p=23216 One question we get asked quite often about developing web applications using HTML5, CSS3, and JavaScript, is about how to conceal code. Reasons range from preventing cheating, protecting intellectual property, hiding encryption algorithms and others. Since JavaScript is executed on the client side, even on Firefox OS, the users can get access the code if they have the skills and spend the time. But don’t get the wrong idea! Applications on Android, iOS and most mobile platforms can be reverse engineered with effort and the right tools.

We will present some measures that make it harder for copycats to reveal your application’s code. In various degrees they will increase the effort, skills and tools required to get the code.

Compressing your code

This is the starting point for most developers. Compressing your code reduces the file size by removing what the browser doesn’t need to run your application. It starts by stripping comments and unnecessary white space. This is a good first step as comments usually reveal a lot of information on how the code is working.

The next level is shortening names of local variables and functions, the so called mangling. As this removes the meaning on those symbols, it further reduces readability.

More advanced compression tools, like UglifyJS, will actually parse your JavaScript to analyze and transform it. This allows not just renaming local symbols but also rewriting conditions and loops into shorter expressions, joining variable declarations (hoisting) and removing unreachable code. The resulting file is not only smaller and less readable but in many cases also faster.

You can take this one step further with the Google Closure optimizer in Advanced mode. The tool allows settings for all the aforementioned transformations. Advanced mode is the most aggressive and requires greater consideration when used, to ensure that the output runs just as the input. It will rename, inline and even remove functions, properties and variables, so you need to make sure to either reference all symbols correctly or annotate them. The extra work pays off as the result is highly compressed and even gains a performance boost. As an example, let’s see what can happen to this code:

document.onload = function() { var shareImage = document.querySelector("#share-image");   function share() { var imgToShare = document.querySelector("#image-to-share");   if (!imgToShare.naturalWidth) { alert("Image failed to load, can't be shared"); return; }   // Create dummy canvas var blobCanvas = document.createElement("canvas"); blobCanvas.width = imgToShare.width; blobCanvas.height = imgToShare.height;   // Get context and draw image var blobCanvasContext = blobCanvas.getContext("2d"); blobCanvasContext.drawImage(imgToShare, 0, 0);   // Export to blob and share through a Web Activitiy blobCanvas.toBlob(function (blob) { var sharingImage = new MozActivity({ name: "share", data: { type: "image/*", number: 1, blobs: [blob] } }); }); }; shareImage.onclick = share; };

Once we’ll run this code through Google Closure, the code will look like:

var b=document.querySelector("#share-image"), c=document.querySelector("#image-to-share");b&&c&& (b.onclick=function(){if(0<c.naturalWidth){ var a=document.createElement("canvas");a.width=c.width; a.height=c.height;a.getContext("2d").drawImage(c,0,0); a.c(function(a){new MozActivity({name:"share", data:{type:"image/*",b:1,a:[a]}})})} else alert("Image failed to load, can't be shared")});

Of course, the further you are writing custom logic, the more the code will be obfuscated, as the tools cannot change names from reserved word, or core functions. There are many tools online, and you can choose the one that fit your need. Most web frameworks offer interchangeable libraries and build steps that bundle and compress your JavaScript sources. For smaller projects your IDE, like Sublime Text, can be extended with packages available.

Using Emscripten

Did you remember that we said JavaScript is interpreted code? Well, you can bend that rule. Mozilla’s Emscripten tool compiles C and C++ to JavaScript. The output isn’t JavaScript as you know it. It behaves more like native code and manages its own memory (stack and heap) in pre-allocated typed arrays. This reduces garbage collection and makes inspecting the memory harder. The output also benefits from asm.js, a highly optimizable subset of JS, that runs on all modern browsers. If you are moving your app from native to the web you can just convert your existing libraries. But even if that doesn’t apply, moving your protected code into a C/C++ library that you compile to JavaScript is the best option to get blazing performance with the added benefits of obfuscation.

Using a web service

Compression and compilation isn’t the safest solution as people still have access to your code: it’s just a bit harder to understand the logic. On the other side, one technique that moves the code to a place you have more control over is to use a web service. Your secret logic would run as a server-side script that your application will call when needed. Someone might be able to inspect the in- and outgoing data but there is no way they will be able to view your logic without access to your server. The major downside is that this feature will be only available when the user is online.

Let’s see a simple example on how you can achieve this with a Firefox OS app, and a Node.js script: let’s say we would like to hide our super exceptional fantastic code to do the addition to two numbers. Here is a simple node.js script (the goal is not to teach node.js, but show you a working basic example):

"use strict"; var express = require('express'); var app = express();   // Set up express to parse body app.use(express.json()); app.use(express.urlencoded()); app.use(express.multipart());   // Only one route for web service app.post('/', function(req, res) { // Convert parameters from request body var firstNumber = parseInt(req.body.firstNumber, 10); var secondNumber = parseInt(req.body.secondNumber, 10);   if (isNaN(firstNumber) || isNaN(secondNumber)) { // Bad request, fail with 400 status return req.send(400); }   var result = firstNumber + secondNumber; res.send({'result': result}); });   app.listen(8000);

Now in your app, you’ll call this script by sending the values you want to add, and the script will return you the result. Of course, users can see the call you make, and the results you’ll get, but they won’t be able to see the magic behind the hidden function.

"use strict"; var firstNumber = 4; var secondNumber = 5;   var data = new FormData(); data.append("firstNumber", firstNumber); data.append("secondNumber", secondNumber);   var xhr = new XMLHttpRequest({mozSystem: true}); xhr.open("POST", "http://fharper-nodejs.nodejitsu.com", true);   xhr.onload = function() { // Parse the response and do something with it var response = JSON.parse(xhr.response); console.log(response.result); };   xhr.onerror = function() { console.log("Magic failed"); };   xhr.send(data);

This code is just showing the web service pattern and not best practices. As any server, it still needs security measures to prevent exploits. Make sure you mask the request and response data by using a secure HTTPS connection. Control client access to your service by checking the Referrer and Origin HTTP headers but don’t trust what you see as they can be faked. You can limit replaying request with malicious parameters by using adding a secret time-limited or single-use token. As you can see, having your code in a web service is keeping it off the client but opens different attack vectors you need to consider.

Be proactive, open your code

Of course, the best approach is to open your code: make your application open source, choose the right license, and show to the rest of the world that you were the first to create this piece of art. Choose the technology you want: you will make it harder to copy your code, but there is never a way to hide your idea. You need to be proactive: you need to be one of the best in your domain. Show your expertise, innovate, add features faster than the competitor do, be relevant for your market, and believe in what you do. How many knock offs of Angry Birds can you find on the web or on any marketplace/store? I don’t have the exact number, but I can definitely say with a lot of confidence, that it’s many! Rovio had an amazing idea, and they were the first to make it a reality: they are always innovating with brand new levels, new games with the characters we like, and more. You, your children, myself, we want to play to the original one, not the copy. Replace Angry Birds with any app or games you like, and you’ll understand what I mean.

You have access to an amazing platform that open the web like never before: take advantage of it, build that app you want, and make your idea a reality!

]]> https://hacks.mozilla.org/2013/12/protecting-your-firefox-os-app-code/feed/ 21 https://hacks.mozilla.org/2013/12/how-the-manana-app-was-built/ https://hacks.mozilla.org/2013/12/how-the-manana-app-was-built/#comments Tue, 03 Dec 2013 09:40:23 +0000 https://hacks.mozilla.org/?p=23160 We saw Firefox OS as a great opportunity and a challenge to deliver a product true to the values of open web and best standards. We found it exciting to be able to deliver an app that will offer a smooth UX even for people using lower end devices. As many of the users have a very limited data plan we decided that a proper reading app offering the possibility to enjoy their favourite articles offline and without unnecessary distractions would be a key product. And so we decided to build Manana.

Here is the story of how we did it.



On a technological side, we decided to go vanilla but with a little help from ZeptoJS to handle the DOM manipulation. We also integrated Web Activities to hook into the sharing options in the browser, and to share the selected links via email. We are using also the Readability API to have a nice formatted version of the link. The app is simple and that’s how we wanted it to be. but we have a great plus: it is internationalized in English, Spanish, Portuguese, Polish, German, Italian. Hungarian and Greek are coming soon.

You can install Manana from the Firefox Marketplace.



Building the layout and managing transitions between the views

To provide a branded UI/UX, we use a small framework built in-house. With some clever CSS properties and a few lines of JavaScript (to handle the visibility of the Views) the whole thing was quite easy to set up. Basically, we have each View arranged like cards in a deck. Every time we need one of them, we pull it out while the old one goes in the deck again (becoming hidden). When the animation is ended, we capture the animationend event, so right after that we remove the class that brings the animation properties and at the same time we add a current class, which lets the card stay in place and be visible. The old one is hidden automatically since that’s its default state.



Below you will find a small example of how it works with a moveRight animation.



Optimize CSS3 transitions

Every smartphone comes with at least a small GPU, so that the CPU doesn’t have to do all the graphical calculations. It’s very much in fashion to take advantage of the Hardware Acceleration and so we do. There are some CSS3 properties that are automatically handled by the GPU, but others are not – usually the ones that are related to 3D. This is because the element (tied to the animated property) is promoted to a new layer that doesn’t suffer from repaints imposed by other properties (like left, for example. But this can change in the future).

tl;dr Where you can, avoid expensive CPU work, especially on big or complicated elements. Here is a small extract of how we implemented such properties.

/* moveLeftIn directive */ .moveLeftIn { animation: 0.5s moveLeftIn cubic-bezier(…) forwards; /* 1 */ backface-visibility: hidden; /* 2 */ transform: translateY(0) translateX(99%); /* 3 */ /* 1. `forwards` let the animation stop at the last keyframe */ /* 2. avoid some possible glitches caused by 3D rendering */ /* 3. take advantage of the GPU with 3D-related properties */ }   /* moveLeftIn animation */ @keyframes moveLeftIn { 0% { transform: translateY(0) translateX(99%); z-index: 999; } 100% { transform: translateY(0) translateX(0); z-index: 999; } }

Hooking to the Firefox OS specific Web Activities

One of our favourite APIs of the WebAPIs, is Web Activities. An application emits an Activity (for example, save bookmark) then the user is prompted to chose among other applications that handles that activity. At the moment it is possible to register an application as an activity handler through the app manifest (declaration registration), but in the future this could be done inside the app itself (dynamic registration).



Saving a link from the browser to Manana is an example of Web Activity (save-bookmark). Registering an activity is trivial:

manifest.webapp

"activities": { "save-bookmark": { "disposition": "inline", "index.html#/savebookmark", "returnValue": true } },

returnValue is of great importance, otherwise the handler application (Manana in our case) will never return to the caller (Browser). The handling in the app is also very easy:

navigator.mozSetMessageHandler("activity", function(activity) { ... check if activity is savebookmark   var data = activity.source.data;   ... do stuff with activity data   activity.postResult("saved"); });



The applications included in your phone are part of Gaia, the UI of Firefox OS, and you can actually browse the source. They provide a lot of examples on how to use WebAPIs. If you are interested in the save-bookmark activity, have a look at the Homescreen app source code.

Manana users can also share their resources via email, directly from the app. To achieve that, we’ve implemented the email Web Activity. It’s easy to pre-populate the email subject and body, just follow the mailto URI scheme and you are done.

DB.Get(id, function (rs) { var subject = rs.title, body = rs.url, shareActivity = new MozActivity({ name: "new", data: { type: "mail", url : "mailto:?body=" + encodeURIComponent(body) + "&subject=" + encodeURIComponent(subject) } }); });

Retrieving and storing data with IndexedDB

IndexedDB is a client side NoSQL DB. It has some nice features which make it easier working with app updates and is easy to integrate in your app, even if you’re using a framework like AngularJS. Setting up models is fairly easy and extensively documented on the web and in Gaia Apps.

You don’t have a quota limit, although the Firefox OS Simulator seems to have a limit when dealing with datasets larger than 5mb. Each database is tied to an app and a version. Only one version can run at a time, which makes it easier not to break compatibility, especially if you rely on some external API.

Since it is asynchronous, dealing with IndexedDB operations requires having callback. Below is an example on how we deal with the action of adding a link:

1. Call Link.Add function, passing a link object and a success callback
2. Link.Add get a DB connection by calling getDB
   • In the case a DB does not exist, the onupgradeneeded callback is executed
     • Create DB
     • Create collections
     • Create indexes
   • getDB check if the connection is already open, otherwise it opens a new one
3. Link.Add fills the missing property of links with default values
4. We call the objectStore put method to save/update new data into the collection
   • If the Link object we are going to save already contains a primary key attribute, IndexedDB does an update
   • Otherwise, an insert
5. If the put method is successful it calls a success callback, returning the primary key in event.target.result

Below is an example of DB action defined in our models: (for the sake of brevity we omitted error checking code)

Add: function(data, onsuccess) { getDB(function(db) { var transaction = db.transaction(["links"], "readwrite"), objectStore = transaction.objectStore("links"), request = objectStore.put(data);   request.onsucces = function(e) { onsuccess(e.target.result) // e.target.result is the id of the new Link } }); },

This is how you call it from the other part of the app:

var newLink = {url: "http://zombo.com", title: "Welcome to zombo com"}; Link.Add(newLink, function(linkId) { window.location.hash = "#/read/:" + linkId; });

Enhance localization with Markdown support

Localization is a key feature in our app. Manana currently supports 6 languages (soon it will be 8). There is no official recommendation, but after some research we decided to use the same technique that Gaia is using. This is a great feature of working with Firefox OS: the whole source code is there, and you can get inspiration by reading it.

L10n has been used in two different cases: to translate the text content of DOM elements and to translate internal help pages. To make the Localization process as simple as possible, we avoided putting HTML tags in the strings and in the internal help pages; we used markdown instead. By design, we avoided having strings in the Javascript code, all messages and alerts are contained in the HTML.

To support markdown we added some glue between the l10n.js library and marked.js.

To load the help resources, Manana first does a lookup in the current installation to find the file named <resourceName>.<locale>.md. If the lookup fails, the fallback is the English version of the resource. Once the file is retrieved succesfully, the content is applied to the marked function to generate the HTML, and then added to the DOM.

function resourceInit($oldView, $newView, resource) { var fallback = "/locales/resources/" + resource + ".en.md", localized = "/locales/resources/" + resource + "." + getLanguage() + ".md", renderMd = function (txt) { $(".js-subpage", $newView).html(marked(txt)); }, renderError = function (txt) { $(".js-subpage", $newView).text("ouch, cannot find: " + resource); };   getFile(localized, renderMd, function () { getFile(fallback, renderMd, renderError); }); }

A note on retrieving local files with a XMLHttpRequest

Retrieving files asynchronously from your local app is as easy as doing a XMLHttpRequest, but there is a case that took us some time to sort out. If the local file you are retrieving is missing, the request won’t fire the success callback (of course), but neither the error callback. It will just hang there forever.

After some research, we noticed a try { ... } catch block in the l10n.js library. Basically, “in Firefox OS with the app:// protocol, trying to XHR a non-existing URL will raise an exception.”

function getFile (url, success, error) { var xhr = new XMLHttpRequest();   xhr.open("get", url, true); xhr.setRequestHeader("Content-Type", "text/plain;charset=UTF-8");   xhr.onreadystatechange = function() { if (xhr.readyState == 4) { if (xhr.status == 200 || xhr.status === 0) success(xhr.responseText); else error && error(); } };   xhr.onerror = error; xhr.ontimeout = error;   // Here it is! try { xhr.send(null); } catch (e) { error && error(); } }

Keeping the UI synchronized with the Database

One of the most challenging aspect of an HTML app is synchronization of state between a data source (db, file, api, etc) and the UI. Frameworks like Backbone and Angular make your life easier but they are not easy beasts to tame. Going vanilla is fun, mostly because JavaScript gives you a lot of flexibility when deciding how to do things.

We really like OO programming by message passing, so we came up with a message based architecture. The body element acts as our global dispatcher while view root element is the local dispatcher. Global events get registered when the application starts and live as long as the app. Local events get registered when entering a view and live until we leave that view.

An example of global events are the CRUD operations we want to perform on our model.

Each action on the UI which changes the state of models is tied to a global event, registered on document.body. On entering a new view, we call a constructor function, which executes some initialization code.

var events = { // global events are never unbinded global: { "save-link": function() { ... }, "remove-link": function() { ... }, ...   "update-link-collecetion": function { ... },   ... },   // view specific events are unbinded everytime we leave that view home: { "update-link-collection-complete": function(e) { var $caller = e.detail.$caller;   ... } },   ... }   function homeInit($oldView, $newView, dispatch) { // $oldView is the previous view // $newView is the current view   // dispatch in an helper which wrap // document.body.dispatchEvent(new CustomEvent(...)) dispatch("update-link-collection") }

Leaving a view calls a destructor on that view, and unbinds all the local events. At the end of the day we can open the app source code without spending hours trying to figure out how things work because of framework related black magic.

Integration with git

When it comes to Git, our hearts melt. We love git. Seriously. Git and Vim are the best pieces of software ever written. Love.

Ok, lets get serious again.

Our application is composed from two components, the app itself and an API Server. The API server is a Golang API proxy to Readability API. So our app directory layout looks like this:

manana/
 |- distrib.sh
 |- srv/
 |- app/

distrib.sh is a simple bash script which perform three simple tasks:

• Install a post-{commit, merge, rebase} hook, if not already installed
• Update app/js/revision.js writing the current git revision hash.
• Create a clean zip containing the app, ready for the app store or your device.

This makes our lives easier.

Conclusion. What Next?

Manana relies on the Readability API to get the stripped down version of the web page. Readability is doing a really good job, but we’d like to experiment with other solutions, like parsing the page directly in the device. It’s a hard job, but it’d be cool to do it on the client side, and some work has already been done.

We had a great time developing for Firefox OS. The most important thing for us, techwise, is that we developed a webapp, not an OS-specific app. We can use all the know-how we gained in the past years, all the great open source libraries we want, and we can even use the browser debug tools we use every day.

To help newcomers in building a Firefox OS app, we have created a list of resources we found indispensable.

]]> https://hacks.mozilla.org/2013/12/how-the-manana-app-was-built/feed/ 8 https://hacks.mozilla.org/2013/11/firefox-and-firecat-as-a-platform-for-ethical-hacking/ https://hacks.mozilla.org/2013/11/firefox-and-firecat-as-a-platform-for-ethical-hacking/#comments Thu, 28 Nov 2013 09:19:46 +0000 https://hacks.mozilla.org/?p=23129 Some years ago – in early 2007, while working as freelancers – we were challenged to do a penetration test on a web application. It was really simple but had a condition-based methodology, and therefore was impossible to use any automated tool; we could only use a web browser.

Our solution was to convert Firefox into a Platform for Ethical Hacking.

How did we do it?

We searched the Web, looking for a solution that satisfied the requirements and obligations to not use automated tools. We found many, a lot of them being add-ons and extensions. Many of them were unknown, unnoticed and unofficial.

Having compiled some useful findings, our next challenge was to create something that maintained the essence of Mozilla Firefox: open, free and easy for everyone. The project was named FireCAT (Firefox Catalog of Auditing exTensions)



What is FireCAT?

FireCAT is a mind map organized into different categories and subcategories with a variety of add-ons available. These are themed around application security auditing and assessment, adding news functionality, customizing your browser and providing the ability to adapt it to different user’s need.

Current Status

The latest version is 2.0, and was presented at Black Hat Arsenal during 2011. From the web site it is possible to list the content and browse the categories, directly accessing the add-ons on Mozilla Add-ons; you can also download the entire mind map with the categories:



What’s New?

• Current Version: 2.0
• Add-ons: > 90
• Categories: 7
• Subcategories: 19

FireCAT v2.0 – Firefox Catalog of Auditing exTensions


• Information Gathering
  • Whois
  • Location Info
  • Enumeration & Fingerprint
  • Data Mining
  • Googling & Spidering
• Proxies & Web Utilities
• Editors
• Network Utilities
  • Intrusion Detection System
  • Sniffers
  • Wireless
  • Passwords
  • Protocols & Applications
• Misc
  • Tweaks & Hacks
  • Encryption / Hashing
  • Antivirus & Malware Scanner
  • Anti Spoof
  • Anti-phishing / Pharming / Jacking
  • Automation
  • Logs / History
  • Backup & Synchronization
  • Protection
• IT Security Related
• Application Auditing

What can we find in the categories?

The 7 main categories are as follows:

Information Gathering

An important part of the process of conducting vulnerability assessment or penetration testing is related to obtaining information in a passive or active way about our objective, to be analyzed later.

From the Firefox platform, we could identify IP addresses, query DNS, verify HTTP headers, generate reports of domains via Whois, use different search engines, and identify applications and versions from Web Server, modules, etc.

Example



“Show IP” shows the IP address(es) of the current page in the status bar. It also allows querying of custom services by IP (right mouse button) and hostname (left mouse button), like whois, netcraft, etc.

Proxies & Web Utilities

Sometimes it’s necessary to use proxies to connect with different hosts or through networks for exaple using TOR to hide our real IP, analyze time responses, detect load balancer, etc.

Example

The User Agent Switcher extension adds a menu and a toolbar button to switch the user agent of the browser:



Editors

From the ubiquitous FireBug to different JavaScript or XML editors, the available editors can help us to visualize and modify content as required.

Example



The JavaScript Deobfuscator will show you what JavaScript gets to run on a web page, even if it is obfuscated and generated on the fly.

Network Utilities

In this section you can use and connect with applications and services such as FTP, DNS and databases in real time.

Miscellaneous

This section contains a lot of useful add-ons, for example tools to convert your ASCII into md5, sha1, base64, hex, etc., and methods to identify the WOT (Web of Trust) in a web site, guage reputation and ratings, modify your HTTP Referer, automate actions through Macros, and make backups of your configuration and installed addons.

Example



The Certificate Patrol add-on reveals when certificates are updated, so you can ensure it was a legitimate change.

IT Security Related

Sometimes it is necessary to use search engines to find information within some Web sites, content related to errors, configurations, and news vulnerabilities by CVE.

Application Auditing

The HackBar is probably the best known addon in this group, but I recommend you take a deeper look because many of them are useful. For example, Exploit-Me’s suite to perform vulnerability analysis of cross-site scripting and SQL Injection.

Example



The SQLite Manager allows you to manage any SQLite database on your computer.

Recommended add-ons

The top ten recommended add-ons would be:

• Firebug
• Web Developer
• HackBar
• FoxyProxy
• Exploit Me
• Tamper Data
• iMacros for Firefox
• SQLite Manager
• Live HTTP Headers
• PassiveRecon

Final Words

I invite you to explore the different add-ons and extensions available, identifying which ones are most interesting and helpful for your work, and their relative performance and compatibilities. Firefox was a great help during our work!

If you decide to install a large amount of add-ons, memory usage might be a problem: consider this carefully.

Feel free to send us your comments and recommendations for new options to add to the mind map!

]]> https://hacks.mozilla.org/2013/11/firefox-and-firecat-as-a-platform-for-ethical-hacking/feed/ 2 https://hacks.mozilla.org/2013/11/implementing-in-app-payments-in-your-firefox-os-app/ https://hacks.mozilla.org/2013/11/implementing-in-app-payments-in-your-firefox-os-app/#comments Tue, 26 Nov 2013 20:29:36 +0000 https://hacks.mozilla.org/?p=21950 At Mozilla, we have been working on bringing payments to the web. We think it is important that developers have an easy way to monetize their apps. It also gives us the chance to offer deeper platform integration, allowing things like carrier billing in addition to credit/debit cards.

We’ve talked about in-app payments before in navigator.mozPay() For Web Payments, but this is an overview from an app developer’s standpoint. This also includes a few updates, such as real payments being live in the Firefox Marketplace. Payments are different than simply being a paid app, which charges the user once on the initial purchase. In-app payments is a different workflow, and allows you to charge the customer for specific items within your app.

Architecture Overview

Payments are only enabled on Firefox OS devices for now. We are working on enabling them across other platforms, even desktop Firefox, but that is a ways off. For the rest of this article, we will assume the Firefox OS environment.

The MDN page for in-app payments has a good overview of the process, but here’s an even quicker overview. You need a server to generate the payments requests, because otherwise a hacker could easily change the prices to 0. The server creates JWT tokens, passes them to your app, and your app calls navigator.mozPay with the token. The user will be taken to a built-in payments workflow, and when finished, your server will be notified. The previous Hacks article on payments goes into the lower-level mechanics and reasons for payments more thoroughly.

It’s up to you to handle the rest of the workflow: updating the app when an item is successfully purchased, linking purchases to accounts, etc.

Getting a Simulation Key

In order to use payments, you’ll need use the app key and secret generated for you by the Marketplace. If you just want to simulate payments, you can easily grab a testing key and secret. The only thing you need to do is create an account if you haven’t done so already.

You will only be able to simulate payments with that key. When you are ready to use real money, you need to submit your app to the Marketplace, setup payments, and obtain a real key and secret. All of this is explained in more detail below.

Setting up the Server-Side Code

The first thing you need to do is set up the server-side code. A “mozpay” module exists in a few different languages which implements the payment workflow for you. If it does not exist in your programming language, you’ll need to use a JWT module and do some more legwork. Check the rather simple code from mozpay-js for an example of what you need to do.

I wrote a game for Firefox OS called Webfighter which uses node.js as the backend, so I used mozpay-js on the server-side. Webfighter is a shooter with a store where you can buy more weapons and ships, and it’s a good reference for developers to see actual code for working with payments. Examples below are taken from Webfighter. You can check out Webfighter’s source here.

First, require and configure the mozpay module:

var pay = require('mozpay'); var settings = require('settings');   pay.configure({ mozPayKey: settings.payKey, mozPaySecret: settings.paySecret, mozPayAudience: 'marketplace.firefox.com', mozPayType: 'mozilla/payments/pay/v1', mozPayRoutePrefix: '/mozpay' });

The settings module is just a file with JSON that specifies configuration of the app. It’s nice to separate the configuration so that you can easily swap it out in different environments, like development and production.

Use the key and secret that you got from the Marketplace. You should only change the audience and type options if you are using a different payment server than Mozilla’s. The routePrefix is optional and specifies the URL prefix for postbacks from the payment server.

Next you need to define a route that creates the data used for the payment transaction. The client is the one making the actual payments request (so that you don’t have to handle credit card details, and it’s possible to even be charged through carrier billing). You just need to create an object that represents a purchase. This example uses express.js as the http server.

var purchaseQueue = {};   app.post('/sign-jwt', function(req, res) { var token = 'o' + Date.now(); var item = getItem(req.body.name);   var jwt = pay.request({ id: item.name, name: item.name, description: item.description, icons: { '64': settings.url + item.icon }, pricePoint: item.price, productData: token, postbackURL: settings.url + 'mozpay/postback', chargebackURL: settings.url + 'mozpay/chargeback', simulate: { result: 'postback' } });   // Keep track of which JWT objects we are waiting on purchaseQueue[token] = 'processing';   res.send(JSON.stringify({ jwt: jwt, token: token }); });

Notice the settings object again; it’s just a dict that has some configuration properties about our app. settings.url is the base URL where our app is hosted, so that all of the passed URLs are absolute.

We call getItem to get the details about an item (which we don’t define here). I found it easiest to define a JSON file containing info about all the available items. You can see the JSON for available items in Webfighter on GitHub.

There are several fields for a purchase request. id is just a unique identifer, and we simply use the product name. Other fields describe the product, such as icons which lists icons of various sizes. The pricePoint is a number that maps to a specific price for each available region, and you should look at the price point table to find the one you need. For example, a price point of 10 indicates a price of $0.99 in the US.

The simulate parameter tells the payments system to just simulate the request. If it’s passed, no real charges are made and you will be taken through a simulated workflow. This is useful for development. Just remember to remove this flag in production.

You can store anything you want to in productData, which you can later use when handling events. Here I generated an arbitrary token to track requests, and I store it in the global object purchaseQueue. We use this when we handle events.

Next, subscribe to the postback event from the mozpay module:

pay.on('postback', function(data) { // the payment was successful var req = data.request; purchaseQueue[req.productData] = 'success'; });

You can also subscribe to the chargeback event. Chargebacks happen when a customer disputes a charge and gets the money back. This could happen a month later, and is difficult to process. Webfighter keeps track of chargebacks, but the app does not check for them and revoke items. You would need to check periodically or on startup if the user still has access to all purchased items.

pay.on('chargeback', function(data) { var req = data.request; purchaseQueue[req.productData] = 'chargeback'; });

That’s it for the server-side.

Using mozPay On the Client Side

On the client-side, you need to post to /sign-swt and use mozPay to initiate the purchase. In the following code, we check if mozPay is available and if it’s not we just give the user the item. Remember, it’s just a test app! Once a purchase is initiated, we start polling the server until we get a successful purchase.

function buy(name) { // Purchase an item by requesting a JWT object from the // server, and posting it to the mozPay API $.post('/sign-jwt', { name: name }, function(res) { if(navigator.mozPay) { var req = navigator.mozPay([res.jwt]);   req.onerror = function() { console.log('mozPay error: ' + this.error.name); clearPolling(); };   // Poll to see when the payment is complete startPolling(); } else { alert('in-app payments unavailable, so giving it to you for free'); onPurchase(name); } }); }

Here are the polling functions which wait until the server responds with a successful purchase. Note that we only check for a successful purchase. The user will be notified of any errors within the native payment screen, and when they close the screen an error event is fired on the mozPay request object, which we handle and stop the polling. See the above code with req.onerror.

function pollQueue(token, name) { // Poll the server every second to see the status of our // payment request   $.get('/purchaseQueue?token=' + token, function(res) { if(res == 'success') { onPurchase(name); clearPolling(); } }); }   var pollTimer; function startPolling() { pollTimer = setInterval(function() { pollQueue(res.token, name); }, 1000); }   function clearPolling() { if(pollTimer) { clearInterval(pollTimer); } pollTimer = null; }

We call onPurchase when an item is successfully purchased and should be enabled for the user. You should create that function and write your app-specific code.

Real payments: submitting app to the Firefox Marketplace

When you are ready to run real payments, remove the simulate field when the server creates the payment request, and submit your app to the Firefox Marketplace. It needs to be submitted so that you can hook up real payments.

Get paid: configuring bank account details and payment options

Once your app is submitted, edit your app and click on “Compatibility & Payments” on the left side and select “Paid / In-App”. Under “Payment Accounts”, and add your bank account. One thing that tripped me up was needing the SWIFT code for my bank. In the future, Marketplace will hopefully autofill it, but for now you can look it up on the theswiftcodes.com.

Under “Price & Countries” you can set the price for your app. You may just want “Free with in-app payments”, which is what Webfighter is. Additionally, you can selectively choose which region your app is available in. Apps with payments are only available in regions with them enabled.



Use a real key & secret

Now click the “In-App Payments” link in the left sidebar. You’ll see a screen where you can grab a new key & secret for real payments. Use these in the configure call on the server. Never share your secret.

At this time of writing, carrier billing is only available in specific countries. Check out the Payments Status page for where it’s available. When it is not available, the user will be presented with a credit card screen.

Go make some money with your app!

We want you to be successful in participating in the app ecosystem and economy. Let us know what your experience is like! We are happy to answer any questions over on the marketplace mailing list.

]]> https://hacks.mozilla.org/2013/11/implementing-in-app-payments-in-your-firefox-os-app/feed/ 17 https://hacks.mozilla.org/2013/11/firefox-os-security-part-2-user-experience-and-security-updates/ https://hacks.mozilla.org/2013/11/firefox-os-security-part-2-user-experience-and-security-updates/#comments Thu, 21 Nov 2013 16:59:18 +0000 https://hacks.mozilla.org/?p=23110 When presenting Firefox OS to people, security is a big topic. Can an operating system built on web technologies be secure? What has Mozilla built in to avoid drive-by downloads and malware? How can a browser-based app be secure without making the UX suffer by asking the user to react to a lot of “do you want to allow this?”. In this, the second of a two part video series Christian Heilmann (@codepo8), principal evangelist of Mozilla, talks to Michael Coates (@_mwc), chair of @OWASP Board about the user experience when it comes to installing and using apps and the security update model for Firefox OS.

You can watch the video on YouTube.

Firefox OS was built on top of the technologies that power the Web. Following Mozilla’s security practices and knowledge from over 10 years of securing Firefox, Firefox OS is engineered as a multi-tiered system that protects users while delivering the power of the mobile web. The design ensures users are in control of their data and developers have APIs and technologies at their disposal to unlock the power of the Web.

Additional links for more information:

• Firefox OS security architecture
• Firefox OS app permission model
• Application security
• Data sandboxing
• Firefox OS release schedule
]]> https://hacks.mozilla.org/2013/11/firefox-os-security-part-2-user-experience-and-security-updates/feed/ 5 https://hacks.mozilla.org/2013/11/handling-click-to-activate-plugins-using-javascript/ https://hacks.mozilla.org/2013/11/handling-click-to-activate-plugins-using-javascript/#comments Thu, 21 Nov 2013 16:03:28 +0000 https://hacks.mozilla.org/?p=23072 From Firefox 26 onwards — and in the case of insecure Flash/Java in older Firefox versions — most plugins will not be automatically activated. We therefore can no longer plugins starting immediately after they have been inserted into the page. This article covers JavaScript techniques we can employ to handle plugins, making it less likely that affected sites will break.

Using a script to determine if a plugin is installed

To detect if a plugin is actually installed, we can query navigator.mimeTypes for the plugin MIME type we intend to use, to differentiate between plugins that are not installed and those that are click-to-activate. For example:

function isJavaAvailable() { return 'application/x-java-applet' in navigator.mimeTypes; }

Note: Do not iterate through navigator.mimeTypes or navigator.plugins, as enumeration may well be removed as a privacy measure in a future version of Firefox.

Using a script callback to determine when a plugin is activated

The next thing to be careful of is scripting plugins immediately after instances are created on the page, to avoid breakage due to the plugin not being properly loaded. The plugin should make a call into JavaScript after it is created, using NPRuntime scripting:

function pluginCreated() { document.getElementById('myPlugin').callPluginMethod(); }

<object type="application/x-my-plugin" data="somedata.mytype" id="myPlugin"> <param name="callback" value="pluginCreated()"> </object>

Note that the “callback” parameter (or something equivalent) must be implemented by your plugin. This can be done in Flash using the flash.external.ExternalInterface API, or in Java using the netscape.javascript package.

Using properties on the plugin to determine when it activated

When using a plugin that doesn’t allow us to specify callbacks and we can’t modify it, an alternative technique is to test for properties that the plugin should have, using code constructs like so:

<p id="myNotification">Waiting for the plugin to activate!</p> <object id="myPlugin" type="application/x-my-plugin"></object>

window.onload = function () { if (document.getElementById('myPlugin').myProperty !== undefined) { document.getElementById('myNotification').style.display = 'none'; document.getElementById('myPlugin').callPluginMethod(); } else { console.log("Plugin not activated yet."); setTimeout(checkPlugin, 500); } }

Making plugins visible on the page

When a site wants the user to enable a plugin, the primary indicator is that the plugin is visible on the page, for example:



If a page creates a plugin that is very small or completely hidden, the only visual indication to the user is the small icon in the Firefox location bar. Even if the plugin element will eventually be hidden, pages should create the plugin element visible on the page, and then resize or hide it only after the user has activated the plugin. This can be done in a similar fashion to the callback technique we showed above:

function pluginCreated() { // We don't need to see the plugin, so hide it by resizing var plugin = document.getElementById('myPlugin'); plugin.height = 0; plugin.width = 0; plugin.callPluginMethod(); }

<!-- Give the plugin an initial size so it is visible --> <object type="application/x-my-plugin" data="somedata.mytype" id="myPlugin" width="300" height="300"> <param name="callback" value="pluginCreated()"> </object>

Note: For more basic information on how plugins operate in Firefox, read Why do I have to click to activate plugins? on support.mozilla.org.

]]> https://hacks.mozilla.org/2013/11/handling-click-to-activate-plugins-using-javascript/feed/ 0 https://hacks.mozilla.org/2013/11/the-pond-building-a-multi-platform-html5-game/ https://hacks.mozilla.org/2013/11/the-pond-building-a-multi-platform-html5-game/#comments Wed, 20 Nov 2013 23:47:32 +0000 https://hacks.mozilla.org/?p=23046

Introducing The Pond

The Pond is a multi-platform HTML5 game (source code) that explores minimalistic design and resolution independent gameplay. The Pond isn’t about reaching a high score, or about buying weapon upgrades. It’s about relaxing and exploring a beautiful world.

It is available on all these platforms/in all these stores:

• 
• 
• 
• 
• 
• 

In making The Pond I came across many performance obstacles which I will explore in detail (especially when optimizing the codebase for mobile).

Tools

Before I begin, I would like to mention the two tools that made coding The Pond both efficient and highly enjoyable: Light Table and CocoonJS.

Light Table is an IDE (still in alpha) which provides an integrated development environment for real-time javascript code injection. This means that javascript edited within the editor can be previewed without reloading the page. If we look at the shape of the fish in the game we notice that it is comprised of Bézier curves. Instead of trying to find an editor for creating Bézier curves, I simply estimated a basic shape and modified the variables in real-time until I was satisfied with it’s look and feel.

CocoonJS on the otherhand provides a canvas optimized compatibility layer for improved performance on mobile devices. Not only does it optimize, it also provides an interface for exporting our application to many devices (Android, iOS, Amazon (android), Pokki, and Chrome Web Store).

Physics

The Pond may seem simple on the outside, but on the inside it’s full of performance optimizations and responsive features. As we resize the game, it updates and re-optimizes itself to render less objects and spawn less fish, and if that’s not enough the framerate degrades smoothly to keep physics in check. This is thanks to the use of a fixed interval physics time step. Gameprogrammingpatterns.com provides a good explanation for how to do this and why it matters, but honestly the code makes the most sense:

var MS_PER_UPDATE = 18; // Time between physics calculations var lag = 0.0; // accumulate lag over frames var previousTime = 0.0; // used for calculating the time delta   // main game loop function draw(time) { requestAnimFrame(draw); // immidiately queue another frame lag += time - previousTime; // add time delta previousTime = time;   var MAX_CYCLES = 18; // prevent infinite looping/hanging on slow machines   // physics calculations while(lag >= MS_PER_UPDATE && MAX_CYCLES) {   // user input, movement, and animation calculations physics(); lag -= MS_PER_UPDATE; MAX_CYCLES--; }   // if we exhausted our cycles, the client must be lagging if(MAX_CYCLES === 0) {   // adaptive quality lowerQuality(); }   // if 5 frames behind after update, jump // this prevents an infinite input lag from ocurring if(lag/MS_PER_UPDATE > 75) { lag = 0.0; }   // draw to canvas paint(); }

What’s important to notice here is that physics is not calculated based on the time delta, instead it’s calculated at a fixed 18ms interval. This is important because it means that any client lag will not be reflected in physics calculations, and that slower machines will simply lose framerate.

Dynamic Quality

The next optimization we notice is the lowerQuality() function, which adaptively decreases the render quality of the game. The way this works is simply by re-sizing the drawing canvas (it’s still full screen, it simply gets streched out), which in-turn leads to reduced spawns and collisions.

function resizeWindow() {   // quality is a global variable, updated by lowerQuality() $canv.width = window.innerWidth * quality/10 $canv.height = window.innerHeight * quality/10 ctx = $canv.getContext('2d') ctx.lineJoin = 'round'   // resize HUD elements, and reduce spawning if(GAME.state === 'playing') { GAME.spawner.resize($canv.width, $canv.height) GAME.levelBar.resize($canv.width, $canv.height) GAME.levelBalls.resize($canv.width, $canv.height) } else { if(ASSETS.loaded) drawMenu() } }

Spawning

Now, we’ve been talking about reducing spawning to improve performance so let me explain how that happens. The spawning algorithm works by creating a virtual grid sized based on the window size. As the player travels from one grid zone to another, the adjacent zones are populated with enemies:



Spawner.prototype.spawn = function(zone) { // spawn 1-3 fish per 500sqpx, maybe larger maybe smaller than player // 0.5 chance that it will be bigger/smaller var mult = this.width*this.height/(500*500) for(var i=0, l=(Math.floor(Math.random()*3) + 1) * mult; i < l; i++) {   // spawn coordinates random within a zone var x = zone[0]+Math.floor(this.width*Math.random()) - this.width/2 var y = zone[1]+Math.floor(this.height*Math.random()) - this.height/2 var size = Math.random() > 0.5 ? this.player.size + Math.floor(Math.random() * 10) : this.player.size - Math.floor(Math.random() * 10)   // spawn a new fish var fish = new Fish(true, x, y, size, Math.random()*Math.PI*2-Math.PI, Math.random()*Math.PI)   this.fishes.push(fish) } return zone }

The last piece of the puzzle is removing enemies as they move far enough away:

// if far enough away from player, remove if(distance(fish, player) > Math.max($canv.width, $canv.height) * 2) { fish.dead = true }

Collisions

The next performance optimization lies with the collision code. Colliding irregularly shaped objects can be extremely difficult and resource intensive. One option is to do color based collision (scan for overlapping colors), but that is much too slow. Another option might be to mathematically calculate Bézier curve collisions, however this is not only CPU intensive, it is also quite difficult to code. I finally opted for an approximation approach using circles. Basically I calculate the position of circles within each fish and detect circle collision among the fish. Boolean circle collision is extremely efficient, as it simply requires measuring the distance between objects. This ends up looking like this (debug mode):



Fish.prototype.collide = function (fish) {   // the fish has been killed and is being removed or it is far away if (this.dying || fish.dying || distance(this, fish) > this.size * 5 + fish.size*5) { return false }   // there are 6 circles that make up the collision box of each fish var c1, c2 for (var i=-1, l = this.circles.length; ++i < l;) { c1 = this.circles[i] for (var j=-1, n = fish.circles.length; ++j < n;) { c2 = fish.circles[j]   // check if they touch if(distance(c1, c2) <= c2.r + c1.r) { return true } } }   return false }

We also avoid unnecessary collision calculations by only checking the fish that are visible (or near-visible):

if(Math.abs(fish2.x - player.x) < $canv.width && Math.abs(fish2.y - player.y) < $canv.height) { // check }

Drawing

After getting the physics+ out of the way, it’s time to optimize drawing operations. Many games use sprite maps for animation (Senshi for example) which can be highly optimized. Unfortunately our fish are dynamically generated so we must find other ways to optimizing drawing. First lets use Chrome’s javascript profiler to identify bottlenecks:



What we see here is that stroke is using a lot of resources. Truth be told, fill used to be there too. This is because both were called heavily when drawing fish. The game looked a bit like this:



After removing fill I saw a huge performance increase, and the game looked much better. The reason the drawImage function is up there as well is because I take advantage of offscreen canvas rendering. Each fish is drawn on its own offscreen canvas which is then rendered onto the larger visible canvas. This is also what allowed me to easily explode the fish into particles by reading pixel data:

Fish.prototype.toParticles = function(target) { var particles = []   // read canvas pixel data var pixels = this.ctx.getImageData(0,0,this.canv.width, this.canv.height).data for(var i = 0; i < pixels.length; i += 36 * Math.ceil(this.size/20) * (isMobile ? 6 : 1)) { var r = pixels[i] var g = pixels[i + 1] var b = pixels[i + 2]   // black pixel - no data if(!r && !g && !b){ continue }   // Math to calculate position var x = i/4 % this.canv.width - (this.canv.width/2 + this.size) var y = Math.floor(i/4 / this.canv.width) - (this.canv.height/2) var relativePos = rot(x, y, this.dir) x=this.x + relativePos[0] y=this.y + relativePos[1]   var col = new Color(r, g, b) var dir = directionTowards({x: x, y: y}, this) particles.push(new Particle(x, y, col, target, Math.PI*Math.random()*2 - Math.PI, this.size/20)) } return particles }

The End

In the end the performance optimizations paid off and made the game feel more polished and playable even on lower-end mobile devices.

If you enjoyed this post, I regularly blog about my development projects over at http://zolmeister.com.

The Pond awaits exploring…

]]> https://hacks.mozilla.org/2013/11/the-pond-building-a-multi-platform-html5-game/feed/ 8 https://hacks.mozilla.org/2013/11/launching-developer-q-a-on-stack-overflow/ https://hacks.mozilla.org/2013/11/launching-developer-q-a-on-stack-overflow/#comments Tue, 19 Nov 2013 21:56:42 +0000 https://hacks.mozilla.org/?p=23091 One thing that is very important for us at Mozilla is the need to directly interact with you developers and help you with challenges and issues while developing using open technologies. We are now happy to announce our presence on Stack Overflow!

Stack Overflow is one of – if not the – most well-known question and answer sites among developers, and we believe it’s very important to go where developers are and choose to have their discussions while developing.

Initially our main focus is on Firefox OS app development and Open Web Apps in general. There are three tags that we are monitoring:

• firefox
• firefox-os
• html5-apps

All tags and posts are available at our landing page on Stack Overflow: http://stackoverflow.com/r/mozilla

Take part!

We’d love for you to participate with both questions and answers, so we together can collaborate and make the Open Web and open technologies the first choice for developers!



A little extra motivation might be that if you answer a lot of questions, you’ll show up in the Top Users page for that tag, e.g. http://stackoverflow.com/tags/firefox-os/topusers

Resources that might be good for you about HTML5, Apps, Developer Tools and more are:

• HTML5 on the Mozilla Developer Network (MDN)
• App Center on MDN
• Firefox OS on MDN
• Firefox Developer Tools on MDN

See you on Stack Overflow!

]]> https://hacks.mozilla.org/2013/11/launching-developer-q-a-on-stack-overflow/feed/ 18 https://hacks.mozilla.org/2013/11/firefox-os-security-part-1-the-web-security-model/ https://hacks.mozilla.org/2013/11/firefox-os-security-part-1-the-web-security-model/#comments Thu, 14 Nov 2013 16:06:08 +0000 https://hacks.mozilla.org/?p=23080 When presenting Firefox OS to people, security is a big topic. Can an operating system built on web technologies be secure? What has Mozilla built in to avoid drive-by downloads and malware? In this two part video series Christian Heilmann (@codepo8), principal evangelist of Mozilla, talks to Michael Coates (@_mwc), chair of @OWASP Board about all things security in Firefox OS.

Firefox OS was built on top of the technologies that power the Web. Following Mozilla’s security practices and knowledge from over 10 years of securing Firefox, Firefox OS is engineered as a multi-tiered system that protects users while delivering the power of the mobile web. The design ensures users are in control of their data and developers have APIs and technologies at their disposal to unlock the power of the Web.

Watch the following video where we talk more about the security design and controls present in Firefox OS. In this, the first of two videos on Firefox OS security, we’ll cover items such as the multi-tiered architecture, the permission model, run time decision making, protection of users data and the update model. You can watch the video on YouTube.

Additional links for more information:

• Firefox OS security architecture
• Firefox OS app permission model
• Application security
• Data sandboxing
• Firefox OS release schedule
]]> https://hacks.mozilla.org/2013/11/firefox-os-security-part-1-the-web-security-model/feed/ 14 https://hacks.mozilla.org/2013/11/introducing-the-whiteboard-drum-webrtc-and-web-audio-api-magic/ https://hacks.mozilla.org/2013/11/introducing-the-whiteboard-drum-webrtc-and-web-audio-api-magic/#comments Thu, 14 Nov 2013 07:18:08 +0000 https://hacks.mozilla.org/?p=22982 Browser functionality has expanded rapidly, way beyond merely “browsing” a document. Recently, Web browsers finally gained audio processing abilities with the Web Audio API. It is powerful to the point of building serious music applications.

Not only that, but it is also very interesting when combined with other APIs. One of these APIs is getUserMedia(), which allows us to capture audio and/or video from the local PC’s microphone / camera devices. Whiteboard Drum (code on GitHub) is a music application, and a great example of what can be achieved using Web Audio API and getUserMedia().

I demonstrated Whiteboard Drum at the Web Music Hackathon Tokyo in October. It was a very exciting event on the subject of Web Audio API and Web MIDI API. Many instruments can collaborate with the browser, and it can also create new interfaces to the real world.



I believe this suggests further possibilities of Web-based music applications, especially using Web Audio API in conjunction with other APIs. Let’s explain how the key features of Whiteboard Drum work, showing relevant code fragments as we go.

Overview

First of all, let me show you a picture from the Hackathon:



And a easy movie demo:

As you can see, Whiteboard Drum plays a rhythm according to the matrix pattern on the whiteboard. The whiteboard has no magic; it just needs to be pointed at by a WebCam. Though I used magnets in the demo, you can draw the markers using pen if you wish. Each row represents the corresponding instruments of Cymbal, Hi-Hat, Snare-Drum and Bass-Drum, and each column represents timing steps. In this implementation, the sequence has 8 steps. The color blue will activate the grid normally, and the red will activate with accent.

The processing flow is:

1. The whiteboard image is captured by the WebCam
2. The matrix pattern is analysed
3. This pattern is fed to the drum sound generators to create the corresponding sound patterns



Although it uses nascent browser technologies, each process itself is not so complicated. Some key points are described below.

Image capture by getUserMedia()

getUserMedia() is a function for capturing video/audio from webcam/microphone devices. It is a part of WebRTC and a fairly new feature in web browsers. Note that the user’s permission is required to get the image from the WebCam. If we were just displaying the WebCam image on the screen, it would be trivially easy. However, we want to access the image’s raw pixel data in JavaScript for more processing, so we need to use canvas and the createImageData() function.

Because pixel-by-pixel processing is needed later in this application, the captured image’s resolution is reduced to 400 x 200px; that means one rhythm grid is 50 x 50 px in the rhythm pattern matrix.

Note: Though most recent laptops/notebooks have embedded WebCams, you will get the best results on Whiteboard Drum from an external camera, because the camera needs to be precisely aimed at the picture on the whiteboard. Also, the selection of input from multiple available devices/cameras is not standardized currently, and cannot be controlled in JavaScript. In Firefox, it is selectable in the permission dialog when connecting, and it can be set up from “contents setup” option of the setup screen in Google Chrome.

Get the WebCam video

We don’t want to show these parts of the processing on the screen, so first we hide the video:

<video id="video" style="display:none"></video>

Now to grab the video:

video = document.getElementById("video"); navigator.getUserMedia=navigator.getUserMedia||navigator.webkitGetUserMedia||navigator.mozGetUserMedia; navigator.getUserMedia({"video":true}, function(stream) { video.src= window.URL.createObjectURL(stream); video.play(); }, function(err) { alert("Camera Error"); });

Capture it and get pixel values

We also hide the canvas:

<canvas id="capture" width=400 height=200 style="display:none"></canvas>

Then capture our video data on the canvas:

function Capture() { ctxcapture.drawImage(video,0,0,400,200); imgdatcapture=ctxcapture.getImageData(0,0,400,200); }

The video from the WebCam will be drawn onto the canvas at periodic intervals.

Image analyzing

Next, we need to get the 400 x 200 pixel values with getImageData(). The analyzing phase analyses the 400 x 200 image data in an 8 x 4 matrix rhythm pattern, where a single matrix grid is 50 x 50 px. All necessary input data is stored in the imgdatcapture.data array in RGBA format, 4 elements per pixel.

var pixarray = imgdatcapture.data; var step; for(var x = 0; x < 8; ++x) { var px = x * 50; if(invert) step=7-x; else step=x; for(var y = 0; y < 4; ++y) { var py = y * 50; var lum = 0; var red = 0; for(var dx = 0; dx < 50; ++dx) { for(var dy = 0; dy < 50; ++dy) { var offset = ((py + dy) * 400 + px + dx)*4; lum += pixarray[offset] * 3 + pixarray[offset+1] * 6 + pixarray[offset+2]; red += (pixarray[offset]-pixarray[offset+2]); } } if(lum < lumthresh) { if(red > redthresh) rhythmpat[step][y]=2; else rhythmpat[step][y]=1; } else rhythmpat[step][y]=0; } }

This is honest pixel-by-pixel analysis of grid-by-grid loops. In this implementation, the analysis is done for the luminance and redness. If the grid is “dark”, the grid is activated; if it is red, it should be accented.

The luminance calculation uses a simplified matrix — R * 3 + G * 6 + B — that will get ten times the value – meaning – which means getting the value of range 0 to 2550 for each pixel. And the redness R – B is a experimental value because all that is required is a decision of Red or Blue. The result is stored in the rhythmpat array, with a value of 0 for nothing, 1 for blue or 2 for red.

Sound generation through the Web Audio API

Because the Web Audio API is a very cutting edge technology, it is not yet supported by every web browser. Currently, Google Chrome/Safari/Webkit-based Opera and Firefox (25 or later) support this API. Note: Firefox 25 is the latest version released at the end of October.

For other web browsers, I have developed a polyfill that falls back to Flash: WAAPISim, available on GitHub. It provides almost all functions of the Web Audio API to unsupported browsers, for example Internet Explorer.

Web Audio API is a large scale specification, but in our case, the sound generation part requires just a very simple use of the Web Audio API: load one sound for each instrument and trigger them at the right times. First we create an audio context, taking care of vendor prefixes in the process. Prefixes currently used are webkit or no prefix.

audioctx = new (window.AudioContext||window.webkitAudioContext)();

Next we load sounds to buffers via XMLHttpRequest. In this case, different sounds for each instrument (bd.wav / sd.wav / hh.wav / cy.wav) are loaded into the buffers array:

var buffers = []; var req = new XMLHttpRequest(); var loadidx = 0; var files = [ "samples/bd.wav", "samples/sd.wav", "samples/hh.wav", "samples/cy.wav" ]; function LoadBuffers() { req.open("GET", files[loadidx], true); req.responseType = "arraybuffer"; req.onload = function() { if(req.response) { audioctx.decodeAudioData(req.response,function(b){ buffers[loadidx]=b; if(++loadidx < files.length) LoadBuffers(); },function(){}); } }; req.send(); }

The Web Audio API generates sounds by routing graphs of nodes. Whiteboard Drum uses a simple graph that is accessed via AudioBufferSourceNode and GainNode. AudioBufferSourceNode play back the AudioBuffer and route to destination(output) directly (for normal *blue* sound), or route to destination via the GainNode (for accented *red* sound). Because the AudioBufferSourceNode can be used just once, it will be newly created for each trigger.



Preparing the GainNode as the output point for accented sounds is done like this.

gain=audioctx.createGain(); gain.gain.value=2; gain.connect(audioctx.destination);

And the trigger function looks like so:

function Trigger(instrument,accent,when) { var src=audioctx.createBufferSource(); src.buffer=buffers[instrument]; if(accent) src.connect(gain); else src.connect(audioctx.destination); src.start(when); }

All that is left to discuss is the accuracy of the playback timing, according to the rhythm pattern. Though it would be simple to keep creating the triggers with a setInterval() timer, it is not recommended. The timing can be easily messed up by any CPU load.

To get accurate timing, using the time management system embedded in the Web Audio API is recommended. It calculates the when argument of the Trigger() function above.

// console.log(nexttick-audioctx.currentTime); while(nexttick - audioctx.currentTime < 0.3) { var p = rhythmpat[step]; for(var i = 0; i < 4; ++i) Trigger(i, p[i], nexttick); if(++step >= 8) step = 0; nexttick += deltatick; }

In Whiteboard Drum, this code controls the core of the functionality. nexttick contains the accurate time (in seconds) of the next step, while audioctx.currentTime is the accurate current time (again, in seconds). Thus, this routine is getting triggered every 300ms – meaning look ahead to 300ms in the future (triggered in advance while nextticktime – currenttime < 0.3). The commented console.log will print the timing margin. While this routine is called periodically, the timing is collapsed if this value is negative.

For more detail, here is a helpful document: A Tale of Two Clocks – Scheduling Web Audio with Precision

About the UI

Especially in music production software like DAW or VST plugins, the UI is important. Web applications do not have to emulate this exactly, but something similar would be a good idea. Fortunately, the very handy WebComponent library webaudio-controls is available, allowing us to define knobs or sliders with just a single HTML tag.



NOTE: webaudio-controls uses Polymer.js, which sometimes has stability issues, causing unexpected behavior once in a while, especially when combining it with complex APIs.

Future work

This is already an interesting application, but it can be improved further. Obviously the camera position adjustment is an issue. Analysis can be more smarter if it has an auto adjustment of the position (using some kind of marker?), and adaptive color detection. Sound generation could also be improved, with more instruments, more steps and more sound effects.

How about a challenge?

Whiteboard Drum is available at http://www.g200kg.com/whiteboarddrum/, and the code is on GitHub.

Have a play with it and see what rhythms you can create!

]]> https://hacks.mozilla.org/2013/11/introducing-the-whiteboard-drum-webrtc-and-web-audio-api-magic/feed/ 4 https://hacks.mozilla.org/2013/11/live-editing-webgl-shaders-with-firefox-developer-tools/ https://hacks.mozilla.org/2013/11/live-editing-webgl-shaders-with-firefox-developer-tools/#comments Tue, 12 Nov 2013 17:19:17 +0000 https://hacks.mozilla.org/?p=23013 If you’ve seen Epic Games’ HTML5 port of ‘Epic Citadel’, you have no doubt been impressed by the amazing performance and level of detail. A lot of the code that creates the cool visual effects you see on screen are written as shaders linked together in programs – these are specialized programs that are evaluated directly on the GPU to provide high performance real-time visual effects.

Writing Vertex and Fragment shaders are an essential part of developing 3D on the web even if you are using a library, in fact the Epic Citadel demo includes over 200 shader programs. This is because most rendering is customised and optimised to fit a game’s needs. Shader development is currently awkward for a few reasons:

• Seeing your changes requires a refresh
• Some shaders are applied under very specific conditions

Here is a screencast that shows a how to manipulate shader code using a relatively simple WebGL demo:

Starting in Firefox 27 we’ve introduced a new tool called the ‘Shader Editor’ that makes working with shader programs much simpler: the editor lists all shader programs running in the WebGL context, and you can live-edit shaders and see immediate results without interrupting any animations or state. Additionally editing shaders should not impact WebGL performance.

Enabling the Shader Editor

The Shader Editor is not shown by default, because not all the web pages out there contain WebGL, but you can easily enable it:

1. Open the Toolbox by pressing either F12 or Ctrl/Cmd + Shift + I.
2. Click on the ‘gear’ icon near the top edge of the Toolbox to open the ‘Toolbox Options’.
3. On the left-hand side under ‘Default Firefox Developer Tools’ make sure ‘Shader Editor’ is checked. You should immediately see a new ‘Shader Editor’ Tool tab.

Using the Shader Editor

To see the Shader Editor in action, just go to a WebGL demo such as this one and open the toolbox. When you click on the shader editor tab, you’ll see a reload button you will need to click in order to get the editor attached to the WebGL context. Once you’ve done this you’ll see the Shader Editor UI:



• On the left you have a list of programs, a vertex and fragment shader corresponds to each program and their source is displayed and syntax highlighted in the editors on the right.
• The shader type is displayed underneath each editor.
• Hovering a program highlights the geometry drawn by its corresponding shaders in red – this is useful for finding the right program to work on.
• Clicking on the eyeball right next to each program hides the rendered geometry (useful in the likely case an author wants to focus solely on some geometry but not other, or to hide overlapping geometry).
• The tool is responsive when docked to the side.

Editing Shader Programs

The first thing you’ll notice about Shader program code is that it is not JavaScript. For more information on how Shader programs work, I highly recommend you start with the WebGL demo on the Khronos wiki and/or Paul Lewis’ excellent HTML5 Rocks post. There also some great long standing tutorials on the Learning WebGL blog. The Shader Editor gives you direct access to the programs so you can play around with how they work:

• Editing code in any of the editors will compile the source and apply it as soon as the user stops typing;
• If an error was made in the code, the rendering won’t be affected, but an error will be displayed in the editor, highlighting the faulty line of code; hovering the icon gutter will display a tooltip describing the error.



Learn more about the Shader Editor on the Mozilla Developer Network.

Here is a second screencast showing how you could directly edit the shader programs in the Epic Citadel demo:

]]> https://hacks.mozilla.org/2013/11/live-editing-webgl-shaders-with-firefox-developer-tools/feed/ 17 https://hacks.mozilla.org/2013/11/make-your-firefox-os-app-feel-alive-with-video-and-audio/ https://hacks.mozilla.org/2013/11/make-your-firefox-os-app-feel-alive-with-video-and-audio/#comments Thu, 07 Nov 2013 14:44:21 +0000 https://hacks.mozilla.org/?p=22949 Firefox OS applications aren’t just about text: there is no better way to make your app feel alive than adding some videos or audio to it. Let’s explore different ways we can use as developers to enhance our mobile masterpiece.

Audio and video HTML tags

Since we are talking about HTML, it makes total sense to think about using the <audio>, and <video> tag to play those media in your Firefox OS app. If you want to add a video in your application, just use this code.

<video src="http://v2v.cc/~j/theora_testsuite/320x240.ogg" controls> Your browser does not support the video element. </video>

In this code example, the user will see a video player with controls, and will have the opportunity to start the video. If your application is running in a browser not supporting the video tag, the user will see the text between the tag. It’s still a good practice to do so, even if your primary target is a Firefox OS app, because since it uses HTML5, someone may access it from another browser if it’s a hosted app. Note that you can use other attributes for this element.

As for the audio tag, it’s basically the same.

<audio id="demo" src="/music/audio.mp3" autoplay loop></audio>

In this example, the audio will start automatically, and will play the audio file, in a loop, from the relative path: it’s perfect for background music if you are building a game. Note that you can add other attributes to this element too.

Of course, using those elements without JavaScript give you basic features, but no worries, you can programmatically control them with code. Once you have your HTML element, like the audio example you just saw, you can use JavaScript to play, pause, change the volume, and more.

document.querySelector("#demo").play(); //Play the Audio document.querySelector("#demo").pause(); //Pause the Audio document.querySelector("#demo").volume+=0.1; //Increase Volume document.querySelector("#demo").volume-=0.1; //Decrease Volume

You can read more on what you can do with those two elements in the Mozilla Developer Network documentation. You also want to give a closer look to the supported format list.

Use audio while the screen is locked

Maybe you are building a podcast app, or at least you need to be able to play audio while the screen is locked? There is a way to do it by using the audio tag. You simply need to add the mozaudiochannel attribute with the value of content to your actual tag.

<audio mozaudiochannel="content" preload="none" src="http://upload.wikimedia.org/wikipedia/en/4/45/ACDC_-_Back_In_Black-sample.ogg" autoplay></audio>

Actually, it’s not quite true as this code won’t work as is. You also need to add a permission to the manifest file.

"permissions": { "audio-channel-content":{ "description":"Use the audio channel for the music player" } }

Having the manifest line above will authorize your application to use the audio channel to play music, even when the screen is locked. Having said that, you probably realize that this code is specific to Firefox OS for now. I intentionally put the end of the last sentence in bold as it’s one thing you need to understand about Firefox OS: we had to create some APIs, features or elements to give the power HTML deserve for developers, but we are working with the W3C to make those standards. In the case that the standards won’t be the same as what we created, we’ll change it to reflect it.

Firefox OS Web activities

Finally, something very handy for Firefox OS developers: the Web Activities. They define a way for applications to delegate an activity to another (usually user-chosen) application. They aren’t standardized, at the time of writing. In the case that will be interesting to us, we’ll use the Web Activity call open, to open music or video files. Note that for video, you can also use the view activity that basically does the same. Let’s say I want to open a remote video when someone clicks on a button with the id open-video: I’ll use the following code in my JavaScript to make it happen.

var openVideo = document.querySelector("#open-video"); if (openVideo) { openVideo.onclick = function () { var openingVideo = new MozActivity({ name: "open", data: { type: [ "video/webm", "video/mp4", "video/3gpp", "video/youtube" ], url: "http://v2v.cc/~j/theora_testsuite/320x240.ogg" } }); } }

In that situation, the video player of Firefox OS will open, and play the video: it’s that easy!

In the end…

You may or may not need to use those tricks in your app, but adding videos or audio can enhance the quality of your application, and make it feel alive. At the end, you have to give a strong experience to your users, and it’s what will make the difference between a good and a great app!

]]> https://hacks.mozilla.org/2013/11/make-your-firefox-os-app-feel-alive-with-video-and-audio/feed/ 5