Introducing node-firefox

NOTE: we presented this project last Sunday at FOSDEM, but not everyone could make it to Brussels, so here’s a post explaining what node-firefox is and how can it help you superturbocharge your Firefox OS app development!

At Mozilla we’re always looking for ways in which we can make developers’ lives easier. When aspiring app developers told us that it was cumbersome to get started writing Open Web Apps, we worked on turning App Manager into a more beginner friendly environment, which in turn gave way to WebIDE. This tool simplifies many actions that were slow and tedious before, such as creating a new app, downloading and installing simulators or running and debugging apps.

But there was still a segment of developers that felt left out: power users! They already have their node.js-based build toolchains, with tasks such as asset optimisation, code hinting, or test running. They often also use tools such as Browserify, and perhaps they don’t even write JavaScript, favouring alternatives such as CoffeeScript instead, but all these goodies require you to build the app or website before you push it again to your device or reload the browser.

Essentially, we were telling these developers to leave their beloved command line (or editor shortcuts!) to go to WebIDE and click a button to deploy the app, and then go back to their editor of choice. And they most unanimously answered: “But we don’t like to click! We like the terminal!”

How can we make this more efficient?

People didn’t like this because it implied changing contexts. It is inefficient, we are engineers, and if there one thing that engineers like more than building new things it is probably optimising and streamlining processes.

Since we already have a build script, the only step that is left in order to get our apps onto the runtime is deploying, and that’s what we are using WebIDE for. So the obvious question would be: can we do whatever WebIDE is doing to deploy, but programmatically?

Servers and actors

Every Firefox runtime has something called the remote debugger server. This is not enabled by default, for obvious security reasons, but when enabled, clients can connect to it and take advantage of its various functionalities, such as installing apps, accessing the console, etc. And this is what WebIDE does internally.

Each of these functionalities is provided by an actor. For example, suppose we want to list the installed apps. We would…

  • first find the webApps actor
  • then run the getAll command
  • and get a list of apps in response

Another example would be installing a packaged app. The steps would be:

  • first zip the app contents, using whatever library or way you like
  • then get the webApps actor
  • call the uploadPackage command in the webApps actor with the contents of the ZIP file
  • the result of this call is a File actor
  • call the install command in the webApps actor with the returned File actor
  • done!

Therefore all the magic for installing apps is not in WebIDE—it is in the servers! We can take advantage of this magic programmatically, but building a client from scratch involves establishing TCP connections and parsing packets, which is not what you want to be doing: you want to write apps and push them to devices instead!

Despair not, as node-firefox will abstract that for you. It is not a monolithic piece of code, but a series of node.js modules, each one of them performing a different task, hosted on its own separate repository and published to the npm registry like good module citizens. You can use as many of them as you need in your scripts or task runners, and thus you can finally build and run your app without ever leaving the command line.

Show, don’t tell

But enough of talking and describing; let’s see how to write a script that starts a simulator!

First install the module in your project, using npm:

npm install --save node-firefox-start-simulator

And this would be the script:

var startSimulator = require('node-firefox-start-simulator');
 
startSimulator({ version: '2.2' })
  .then(function(simulator) {
    console.log('Listening in port', simulator.port);
  });

That’s it! With just a few lines of code you are able to programmatically start a version 2.2 simulator. If you don’t care about the version, just don’t pass in any option to startSimulator, and it will start the first simulator it finds:

startSimulator().then(function(simulator) {
  // your code
});

We can also see this in action. Here’s us starting a simulator, installing an app and launching it, all from a node.js script:

Starting simulator, running app from node.js

The code for this example is actually the example for the node-firefox-uninstall-app module. Each of the node-firefox modules come with an examples folder so you can get started rather quickly.

As we mentioned at the beginning, many web developers that move to app development want to keep using their task runners, so we also wrote an example of how to use node-firefox with gulp.

Let’s run the default-one task. This starts a simulator, deploys an app, and for a bit more of a challenge, also keeps watching for CSS changes. If you edit and save any of the app’s stylesheets, the file watcher will detect the change, and send the new file contents to the runtime, which will replace them on the fly, without having to stop, push and relaunch the whole app. Look at me changing the background colour from austere dark blue to the timeless Paul Rouget pink!

Starting simulator, launching app with gulp

Live CSS reloading is really great to build and experiment with UI interfaces. Not having to reload the app and then navigate to the particular layout you want to work in saves lots of time—I wish I’d had that when I was programming Android apps!

But we can outdo this. The default-all task will do the same as default-one, but for all the simulators installed in your system, so you can see the effect of your CSS changes in all the simulators at the same time:

Starting all simulators, launching app and live CSS reload with gulp.

Unfortunately there is a bug in the 2.1 and 2.2 simulators, and those don’t reload the stylesheet changes, but it’s been filed and will be fixed.

What can we do so far?

The current set of modules lets you find ports where runtimes are listening, find and start simulators; connect to runtimes; find, install, uninstall and launch apps, and reload stylesheets.

Philosophy

You might have noticed a pattern already, but just in case it wasn’t evident enough, we are trying to write deliberately simple modules. Each module should perform only one action, return a Promise and use as few dependencies as possible.

Small modules are easier to understand, use, and test. Also, most of the future Web APIs are designed to work with Promises, and we want to write code for the future, not for the past. In addition, reducing the number of dependencies also makes it easier for new people to get started on contributing to a module, as there are fewer new unfamiliar elements to understand.

Finally, since all the modules work the same way, when you know how to use one module you know how to use the rest—the only thing that changes is the parameters, and the result.

Dream ideas (or: what we cannot do yet)

There’s a number of things that we’d like to see happen in the future. Some people call them features, but I call them ‘dream ideas.’

A recurrent one is the WebCLI: a counterpart equivalent to WebIDE, where everything you can do with WebIDE could be done with a command line tool. I keep switching back and forth between “this is a good idea” and “perhaps we don’t need this at all and a library of tasks will be enough”, but people seemed to like the idea, so maybe it’s not that bad!

Another great feature would be the ability to attach the DevTools debugger to an app that was launched from the command line but that just crashed. Launching apps from the command line is great, but command line debuggers are not that exciting! Why not use the best of both worlds?

Or maybe it would be neat to control any browser from the command line, interfacing via Valence!

And finally, there is my favourite dream idea: Firefox OS custom editions. Imagine if we could just write a script that would create an empty Firefox OS slate, pull in our favourite apps and settings, and generate a whole Firefox OS image that we could then flash to devices. And since it is not a binary blob but a script, we could just publish it on its repository, and other people could remix and build their own Firefox OS based editions.

How do we get there?

There’s still a long way ahead of us, and lots of areas that need work. Perhaps the most urgent task is to get better multiplatform support. Currently we can only interact with runtimes through the network, but no physical devices. Also, support on platforms other than Mac OS is largely lacking.

Testing is another important aspect. If we test early, often and profusely we will be able to detect problems such as the CSS bug I stumbled upon when building the gulp demo. We want to have these modules running on several platforms and connecting to other different platforms, including physical devices.

Of course we need more modules, and more examples! To make sure no two people start writing the same module, we are discussing and proposing new modules in the top project issue tracker. And we’d love to see more examples, or even just better examples that hook existing functionality in other node modules with our code. For example, one could add manifest validation via the firefox-app-validator-manifest module.

And, as always, we need you. We are not you, so we cannot know what you need or what thoughts cross your mind. And we certainly cannot use software the same way you use it either. We need your input and your contributions!

We’re looking forward to seeing what you create with node-firefox. File issues, or talk to us on irc if you have questions. We hang out mostly in the #apps and #devtools channels in irc.mozilla.org.

Thanks

It would be dishonest not to thank Nicola Greco, whom I mentored last summer when he was interning at Mozilla. He came up with the initial idea of building individual node modules that would help you develop Firefox OS apps. Go check out his final intern presentation, as it’s really entertaining and illustrative!

Many thanks to all the (infinitely patient) DevToolers Ryan Stinnet, Alexandre Poirot, Jeff Griffiths and Dave Camp, who helped us find our way around remote servers and actors and whatnot, and huge thanks to Heather Arthur who wrote firefox-client and made writing node-firefox way more pleasant than it would have been otherwise.

About Soledad Penadés

Sole works at the Developer Tools team at Mozilla, helping people make amazing things on the Web, preferably real time. Find her on #devtools at irc.mozilla.org

More articles by Soledad Penadés…


13 comments

  1. Hervé

    That looks promising. But can you use it for packaging HTML5 games as with node-webkit ? https://github.com/nwjs/nw.js/wiki/How-to-package-and-distribute-your-apps

    February 5th, 2015 at 09:00

  2. Soledad Penades

    No, this is not meant to generate self contained executables but to help you drive browsers/runtimes.

    February 5th, 2015 at 09:43

  3. Ivan Dejanovic

    I must admit this looks awesome. As a person how just spent two weeks trying to stream line a maven build process of an old Flex application I can definitely see how this can be beneficial. Keep up the good work in supporting FirefoxOS.

    February 5th, 2015 at 12:15

  4. Fritz

    Yes! Thank you

    February 5th, 2015 at 15:08

  5. Ben Adams

    Can you use Desktop Boot2Gecko with a manifest to create a desktop app that you can sick in stores (steam etc)

    February 5th, 2015 at 18:07

    1. sole

      I have no idea, sorry. Perhaps with Cordova.

      February 9th, 2015 at 04:01

  6. Moez Bouhlel

    That’s great! I liked the idea of WebCLI/Valence. For long time, I wanted to execute some js code against the current tab DOM tree without living the terminal. I know that FF DevTools console is the same. But switching between Firefox and the terminal for just one instruction many times a day is a little painful. I Hope that WebCLI will support accessing JavaScript objects of opened tabs on Firefox browser too. And I’m willing to help implement this feature too.

    February 5th, 2015 at 23:40

  7. Lucian

    This will make the development of https://sublevel.net web app for Firefox OS so much easier. Thanks for your efforts!

    February 6th, 2015 at 02:43

  8. Martin

    node-firefox is a very misleading name. I was expecting something like node-webkit, just with firefox instead of webkit.

    February 6th, 2015 at 04:26

    1. Ivan

      Me too Martin!

      The current process of build ‘firefox desktop apps’ is painful… I can’t force the user to install the app under Firefox…

      I would be SO happy if I can distribute executables ‘without dependencies’, like node-webkit…

      February 8th, 2015 at 11:48

      1. Soledad Penades

        I understand where you’re going but what we are working on is to have everything on the web. There is work going on an W3C standard for “apps” i.e. sort of persistent web-based apps. You won’t need to install any particular browser in the future to get app-like features: http://www.w3.org/2008/webapps/manifest/

        February 9th, 2015 at 04:05

  9. Miguel Mota

    Awesome work Mozilla!

    February 7th, 2015 at 01:40

  10. Bhumi

    Very nice article, very informative.

    February 10th, 2015 at 03:35

Comments are closed for this article.