Mozilla

Geolocation Articles

Sort by:

View:

  1. geolocation with open street maps

    This demo was created by René-Luc D’Hont. He created this demo for the 35 days project with open source software and open data from various projects. His company, 3Liz, specializes in open source GIS application development.

    Three days ago we had a post from Doug Turner describing how Geolocation works in Firefox 3.5. René-Luc has taken the geolocation functionality in Firefox 3.5 and blended it together with data from OpenStreetMap and a few other sources of free data. You can try the demo below. Don’t forget to click the Share Location button in the drop down when it appears on the site.

    Assuming that it was able to find your location, you should see where you are with a red marker. A blue circle surrounds the red marker indicating the accuracy of your location information. Note that since this information is based on a combination of your IP address and possibly local WiFi access points, its accuracy can vary.

    This demo also tries to pull in information from other sources about your local area. Each set of information is shown as a layer. These layers are:

    • The base layer is the map itself, provided by OpenStreetMap. OpenStreetMap is a project to create and provide free geographic data, such as street maps, to anyone who wants them. Much like Wikipedia, it’s possible for anyone to edit the maps and add their own information.
    • The next layer is based on articles in Wikipedia. In some articles, like Mountain View or Montpellier, you can find coordinates. GeoNames provides a web service to query Wikipedia’s articles by location. With this demo you can discover Wikipedia articles about things and places around you.
    • The last layer is based on GeoNames. GeoNames is a geographical database covering all countries and contains over eight million placenames. In this demo you can see things from the GeoNames database like cities, villages, lakes, parks, or even hotels.

    The map and layers are built using OpenLayers, a free JavaScript library that you can use to put a dynamic map on any web page.

    Resources

    We’ve also included a couple more screenshots of places that have data already loaded.

    geolocation-2

    geolocation-3

  2. geolocation in Firefox 3.5

    This post is from Doug Turner, one of the engineers who is behind the Geolocation support in Firefox 3.5.

    Location is all around us. As of this writing, I am in a coffee shop in Toronto, Canada. If I type google into the url bar, it takes me to www.google.ca, the Canadian version of Google, based on my IP address. And when I want to find the closest movie theater to where I am located, I typically just type in my postal code. That information is often stored with the site so that it’s easier to find the movie theater next time. In these two situations, having the web application automatically figure out where I am is much more convenient. In fact, I have no idea what the postal code is for Toronto. I know how to find it, but that is a lot of work to simply tell a web application where I am.

    Firefox 3.5 includes a simple JavaScript API that allows you to quickly geo-enable your web application. It allows users to optionally share their location with websites without having to type in a postal code. What follows is a general overview of how to use geolocation in Firefox 3.5, how it works, and examines some of the precautions you should take when using geolocation.

    The Basics

    Getting the user’s location is very easy:

    function showPosition(position) {
        alert(position.coords.latitude + “ “ +
        position.coords.longitude);
    }
     
    navigator.geolocation.getCurrentPosition(showPosition);

    The call to getCurrentPosition gets the users current location and puts it into an alert dialog. Location is expressed in terms of longitude and latitude. Yes, it’s that simple.

    When you ask for that information the user will see a notification bar like this:

    Their options are to allow or not, and to remember or not.

    Handling Errors

    It is important to handle two error cases in your code:

    First, the user can deny or not respond to the request for location information. The API allows you to set an optional error callback. If the user explicitly cancels the request your error callback will be called with an error code. In the case where the user doesn’t respond, no callback will be fired. To handle that case you should include a timeout parameter to the getCurrentPosition call and you will get an error callback when the timer expires.

    navigator.geolocation.getCurrentPosition(successCallback,
                                             errorCallback,
                                             {timeout:30000});

    With this code your errorCallback function will be called if the user cancels. It will also be called if 30 seconds pass and the user hasn’t responded.

    Second, the accuracy of the user’s location can vary quite a bit over time. This might happen for a few reasons:

    • Different methods for determining a person’s location have different levels of accuracy.
    • The user might choose not to share his or her exact location with you.
    • Many GPS devices have limited accuracy depending on the view of the sky. Over time if your view of the sky gets worse, so can the accuracy.
    • Many GPS devices can take several minutes to go from a very rough location to a very specific location, even with a good view of the sky.

    These cases can and will happen, and supporting changes in accuracy is important to provide a good user experience.

    If you want to monitor the location as it changes, you can use the watchPosition callback API:

    navigator.geolocation.watchPosition(showPosition);

    showPosition will be called every time there is a position change.

    Note that you can also watch for changes in position by calling getCurrentPosition on a regular basis. But for power savings and performance reasons we suggest that you use watchPosition when you can. Callback APIs generally save power and are only called when required. This will make the browser more responsive, especially on mobile devices.

    For more information, please take a look at the API draft specification which has other examples which may be useful.

    Under the Hood

    There are a few common ways to get location information. The most common are local WiFi networks, IP address information, and attached GPS devices. In Firefox 3.5 we use local WiFi networks and IP address information to try and guess your location.

    There are a few companies that drive cars around listening for WiFi access points spots and recording the access point’s signal strength at a specific point on the planet. They then throw all of this collected data into a big database. Then they have algorithms that allow you to ask the question “If I see these access points, where am I?”. This is Firefox 3.5’s primary way of figuring out your location.

    But not everyone has a WiFi card. And not every place has been scanned for WiFi. In that case Firefox will use your local IP address to try to figure out your location using a reverse database lookup that roughly maps your IP address to a location. IP derived locations often have much lower accuracy than a WiFi derived location. As an example, here in Toronto, a location from WiFi is accurate to within 150 meters. The same location using just an IP address is about 25 km.

    Privacy

    Protecting a user’s privacy is very important to Mozilla – it’s part of our core mission. Within the realm of data that people collect online, location can be particularly sensitive. In fact, the EU considers location information personally identifiable information (PII) and must be handled accordingly (Directive 95/46/EC). We believe that users should have strict control over when their data is shared with web sites. This is why Firefox asks before sharing location information with a web site, allows users to easily “forget” all of the places that the user has shared their location with, and surfaces sharing settings in Page Info.

    Firefox does what it can to protect our users’ privacy but in addition the W3C Geolocation working group has proposed these privacy considerations for web site developers and operators:

    • Recipients must only request location information when necessary.
    • Recipients must only use the location information for the task for which it was provided to them.
    • Recipients must dispose of location information once that task is completed, unless expressly permitted to retain it by the user.
    • Recipients must also take measures to protect this information against unauthorized access.
    • If location information is stored, users should be allowed to update and delete this information.
    • The recipient of location information must not retransmit the location information without the user’s consent. Care should be taken when retransmitting and use of HTTPS is encouraged.
    • Recipients must clearly and conspicuously disclose the fact that they are collecting location data, the purpose for the collection, how long the data is retained, how the data is secured, how the data is shared if it is shared, how users may access, update and delete the data, and any other choices that users have with respect to the data. This disclosure must include an explanation of any exceptions to the guidelines listed above.

    Obviously these are voluntary suggestions, but we hope that it forms a basis for good web site behavior that users will help enforce.

    Caveats

    We have implemented the first public draft of the Geolocation specification from the W3C . Some minor things may change, but we will encourage the working group to maintain backwards compatibly.

    The only issue that we know about that may effect you is the possible renaming of enableHighAccuracy to another name such as useLowPower. Firefox 3.5 includes the enableHighAccuracy call for compatibility reasons, although it doesn’t do anything at the moment. If the call is renamed, we are very likely to include both versions for compatibility reasons.

    Conclusion

    Firefox 3.5 represents the first step in support for Geolocation and a large number of other standards that are starting to make their way out of the various working groups. We know that people will love this feature for mapping applications, photo sites and sites like twitter and facebook. What is most interesting to us is knowing that people will find new uses for this that we haven’t even thought of. The web is changing and location information plays a huge role in that. And we’re happy to be a part of it.

  3. (r)evolution number 5

    We’ve just launched Firefox 3.5, and we’re incredibly proud. Naturally, we have engaged in plentiful Mozilla advocacy — this site is, amongst other things, a vehicle for showcasing the latest browser’s new capabilities. We like to think about this release as an upgrade for the whole World Wide Web, because of the new developer-facing features that have just been introduced into the web platform. When talking about some of the next generation standards, the appearance of the number “5” is almost uncanny — consider HTML5 and ECMAScript 5 (PDF). The recent (and very welcome) hype around HTML5 in the press is what motivates this article. Let’s take a step back, and consider some of Mozilla’s web advocacy in the context of events leading up to the release of Firefox 3.5.

    Standardization of many of these features often came after much spirited discussion, and we’re pleased to see the prominent placement of HTML5 as a key strategic initiative by major web development companies. Indeed, exciting new web applications hold a great deal of promise, and really showcase what the future of the web platform holds in store for aspiring developers. Many herald the triumphant arrival of the browser as the computer, an old theme that gets bolstered with the arrival of attractive HTML5 platform features that are implemented across Safari, Chrome, Opera, and of course, Firefox (with IE8 getting an honorable mention for having both some HTML5 features and some ECMAScript, 5th Edition features).

    Call it what you will — Web 5.0, Open Web 5th Generation (wince!), or, (R)evolution # 5, the future is now. But lest anyone forget, HTML5 is not a completed standard yet, as the W3C was quick to point out. The editor doesn’t anticipate completion till 2010. The path taken from the start of what is now called HTML5 to the present-day era of (very welcome) hype has been a long one, and Mozilla has been part of the journey from the very beginning.

    For one thing, we were there to point out, in no uncertain terms, that the W3C had perhaps lost its way. Exactly 5 summers ago (again, with that magic number!), it became evident that the W3C was no longer able to serve as sole custodian of the standards governing the open web of browser-based applications, so Mozilla, along with Opera, started the WHATWG. Of course, back then, we didn’t call it HTML5, and while Firefox itself made a splash in 2004, the steps taken towards standardization were definitive but tentative. Soon, other browser vendors joined us, and by the time the reconciliation with W3C occurred two years later, the innovations introduced into the web platform via the movement initiated by Mozilla had gained substantial momentum.

    The net result is a specification that is not yet complete called “HTML5″ which is implemented piecemeal by most modern browsers. The features we choose to implement as an industry are in response to developers, and our modus operandi is (for the most part) in the open. Mozilla funds the HTML5 Validator, producing the first real HTML5 parser, which now drives W3C’s markup validation for HTML5. That parser has made its way back into Firefox. It’s important to note that capabilities that are of greatest interest (many of which are showcased on this blog) are not only developed within the HTML5 specification, but also as part of the W3C Geolocation WG, the Web Apps WG, and the CSS WG.

    The release of Firefox 3.5, along with updates to other modern browsers, seems to declare that HTML5 has arrived. But with the foresight that comes with having been around this for a while, we also know that we have a lot of work ahead of us. For one thing, we’ve got to finish HTML5, or at least publish a subset of it that we all agree is ready for implementation, soon. We’ve also got to ensure that accessibility serves as an important design principle in the emerging web platform, and resolve sticky differences here. Also, an open standard does not an open platform make, as debates about web fonts and audio/video codecs show. We’ve got a lot of work ahead of us, but for now, 5 years after the summer we started the ball rolling, we’re enjoying the hype around (R)evolution Number 5.

  4. Who moved my geolocation?

    One of the questions we often get when we are talking about Firefox OS is: “What about the GPS on some devices”? You may have noticed that on some devices, the GPS position is not quite accurate or can take a long time to report even when you are outside. Let me start by explaining how it works first. After, we’ll see what the issue is right now, and how we can, as developers, continue to work our magic, and create awesome applications using geolocation.

    How the devices give you geolocation

    Most smartphones use two techniques to help you get the longitude and latitude of the phone: the GPS itself, but also something called A-GPS (Assisted GPS) servers. When you are outside, the GPS connects with satellite signals, and gets you the coordinates of the device: latitude and longitude. It works well as long as the GPS can connect with satellites, but it can take some time to achieve this, or to give you something more accurate.

    To help the device achieve its goal faster, often you’ll get a location from an A-GPS server: this is why most of the time, you’ll first get a location within maybe 50 meters, and if you wait a little longer, you’ll get something more accurate. It’s also why, when you are using dedicated GPS devices (like the one you use for hiking or geocaching), it takes longer: they just use the GPS, and need to connect to more satellite, no assisted GPS connection.

    What about Firefox OS?

    Mozilla doesn’t provide any Firefox OS images; we provide source code to chip manufacturer, and OEMs like Geeksphone. These parties customize various parts and create binary images for devices. The final Firefox OS image is mostly representative of what we have in the public source repositories, but with some modifications. This is an important distinction because the configuration of some parts (like Linux config, device setup, etc.) is not in Mozilla’s hands.

    With that in mind, some devices have configuration problems for A-GPS. We are actively working with OEMs to solve that issue, but it’s not something we can fix by ourselves. Once it is fixed for specific devices with A-GPS problems, we’ll let you know about the procedure to fix your device on this blog.

    But I need geolocation for my application

    There are many ways to develop applications needing geolocation information even with this little A-GPS issue. First, you can use the simulator to test your application. There is a nice little option right in the simulator to let you emulate any coordinates you need.

    Screenshot of the Firefox OS Simulator

    Of course, while the simulator is perfect for the development part and the first series of tests, you’ll need to test your new masterpiece on a real device. If you are using Linux or OS X (I’m working on a solution for Windows users), our friend, Doug Turner, created a mock location provider which you can install on your (rooted) phone to do some tests. It can hardcode the latitude and longitude that Firefox OS itself returns to your phone. You can change those coordinates by editing MockGeolocationProvider.js file in the components folder of the project. Of course, you could hardcode this yourself in your code, but you won’t be able to see how well your code handles what the device returns to you.

    Last but not least, you can also use free services like freegeoip.net. It’s a database that you can use to detect geolocation from IP addresses. It’s not perfect, but it’s a good start to give a more accurate location to the user and a good fallback solution for any applications. You never know when there will be a problem with A-GPS or GPS.

    Best practices for apps using GPS

    There are a couple of things you need to keep in mind when you are building an application that needs geolocation. First, you need to think about the accuracy of the result you’ll receive. What you need to know is that using getCurrentPosition tries to return a result as fast as possible: sometimes it means using wifi or the IP address to get the result. When using the GPS device, it may take minutes before it connects to satellites, so in that situation, you have two choices:

    1. You can get the accuracy of the result, in meters, by getting accuracy for the coordinates returned by getCurrentPosition (see code below);
    2. Alternatively, you can define a HighAccuracy option when you call getCurrentPosition (see code below).
    var options = {
        enableHighAccuracy: true,
        timeout: 5000,
        maximumAge: 0
    };
     
    function success(pos) {
        var crd = pos.coords;
     
        console.log('Your current position is:');
        console.log('Latitude : ' + crd.latitude);
        console.log('Longitude: ' + crd.longitude);
        console.log('More or less ' + crd.accuracy + ' meters.');
    };
     
    function error(err) {
        console.warn('ERROR(' + err.code + '): ' + err.message);
    };
     
    navigator.geolocation.getCurrentPosition(success, error, options);

    You also need to think about the fact that the user may move, so you need to re-estimate the user’s coordinates every so often, depending on what you are trying to achieve. You can do this either manually or by using the watchPosition method of the geolocation API in Firefox OS.

    var watchID = navigator.geolocation.watchPosition(function(position) {
        do_something(position.coords.latitude, position.coords.longitude);
    });

    In that situation, if the position changes, either because the devices moves or because more accurate geolocation information arrives, your function will be called, and you’ll be able to handle the new information.

    If you want more information about how to use geolocation in your application, you can always check the Mozilla Developer Network documentation on using geolocation. If you have any questions about using geolocation in your Firefox OS application, please leave a question in the comments’ section.

  5. Webinar: Geolocation with Remy Sharp

    Update 2011-09-09: Oh noes! We had a double dose of technical difficulties during this webinar. The BigBlueButton server froze and required a couple of reboots to get it working. It worked fine for the remainder of the session. Thanks to those who stuck around, and apologies to everyone for the problem.

    On top of that, we had another recording failure. That is, we have a recording, but it has no audio. And as Remy demoed code rather than slides, there are no slides to share. We can post the recording if there’s interest, but it’s sadly lacking Remy’s narration. Double apologies for that.

    Please join us on on Friday, September 9th at 16:00 UTC (convert to your local time), for a webinar on the Geolocation API, with Remy Sharp. Remy is well-known in web development circles, and was one of the first folks interviewed for our People of HTML5 series.

    Geolocation is the topic for the September Dev Derby. Remy has indicated that this session will be light on slides and heavy on code, so get ready for some meaty stuff! Then whip up an awesome way to use geolocation and submit it to the Dev Derby.

    Once again, we’re using BigBlueButton, thanks to Blindside Networks. (The recording issues with last month’s webinar were independent of BBB.)

    Add this event to your calendar:

    We’d like to get a rough estimate of how many people will be attending. If you happen to use Plancast and you expect to attend the webinar, please join the event on Plancast.

    To join the webinar:

    1. Go to the BigBlueButton server for Mozilla.
    2. For Full Name, enter your name.
    3. Select “Mozilla Developer Network” in the Room list.
    4. For Password, enter MDNHacks.

    Note: Big Blue Button uses Flash, so make sure you have the latest version installed, especially if you are running Mac OS X Lion.

    We are planning to record the webinar and make it available for those who can’t attend to view later.

  6. The Making of the Time Out Firefox OS app

    A rash start into adventure

    So we told our client that yes, of course, we would do their Firefox OS app. We didn’t know much about FFOS at the time. But, hey, we had just completed refactoring their native iOS and Android apps. Web applications were our core business all along. So what was to be feared?

    More than we thought, it turned out. Some of the dragons along the way we fought and defeated ourselves. At times we feared that we wouldn’t be able to rescue the princess in time (i.e. before MWC 2013). But whenever we got really lost in detail forest, the brave knights from Mozilla came to our rescue. In the end, it all turned out well and the team lived happily ever after.

    But here’s the full story:

    Mission & challenge

    Just like their iOS and Android apps, Time Out‘s new Firefox OS app was supposed to allow browsing their rich content on bars, restaurants, things to do and more by category, area, proximity or keyword search, patient zero being Barcelona. We would need to show results as illustrated lists as well as visually on a map and have a decent detail view, complete with ratings, access details, phone button and social tools.

    But most importantly, and in addition to what the native apps did, this app was supposed to do all of that even when offline.

    Oh, and there needed to be a presentable, working prototype in four weeks time.

    Cross-platform reusability of the code as a mobile website or as the base of HTML5 apps on other mobile platforms was clearly prio 2 but still to be kept in mind.

    The princess was clearly in danger. So we arrested everyone on the floor that could possibly be of help and locked them into a room to get the basics sorted out. It quickly emerged that the main architectural challenges were that

    • we had a lot of things to store on the phone, including the app itself, a full street-level map of Barcelona, and Time Out’s information on every venue in town (text, images, position & meta info),
    • at least some of this would need to be loaded from within the app; once initially and synchronizable later,
    • the app would need to remain interactively usable during these potentially lengthy downloads, so they’d need to be asynchronous,
    • whenever the browser location changed, this would be interrupted

    In effect, all the different functionalities would have to live within one single HTML document.

    One document plus hash tags

    For dynamically rendering, changing and moving content around as required in a one-page-does-all scenario, JavaScript alone didn’t seem like a wise choice. We’d been warned that Firefox OS was going to roll out on a mix of devices including the very low cost class, so it was clear that fancy transitions of entire full-screen contents couldn’t be orchestrated through JS loops if they were to happen smoothly.

    On the plus side, there was no need for JS-based presentation mechanics. With Firefox OS not bringing any graveyard of half-dead legacy versions to cater to, we could (finally!) rely on HTML5 and CSS3 alone and without fallbacks. Even beyond FFOS, the quick update cycles in the mobile environment didn’t seem to block the path for taking a pure CSS3 approach further to more platforms later.

    That much being clear, which better place to look for best practice examples than Mozilla Hacks? After some digging, Thomas found Hacking Firefox OS in which Luca Greco describes the use of fragment identifiers (aka hashtags) appended to the URL to switch and transition content via CSS alone, which we happily adopted.

    Another valuable source of ideas was a list of GAIA building blocks on Mozilla’s website, which has since been replaced by the even more useful Building Firefox OS site.

    In effect, we ended up thinking in terms of screens. Each physically a <div>, whose visibility and transitions are governed by :target CSS selectors that draw on the browser location’s hashtag. Luckily, there’s also the onHashChange event that we could additionally listen to in order to handle the app-level aspects of such screen changes in JavaScript.

    Our main HTML and CSS structure hence looked like this:

    And a menu

    We modeled the drawer menu very similarily, just that it sits in a <nav> element on the same level as the <section> container holding all the screens. Its activation and deactivation works by catching the menu icon clicks, then actively changing the screen container’s data-state attribute from JS, which triggers the corresponding CSS3 slide-in / slide-out transition (of the screen container, revealing the menu beneath).

    This served as our “Hello, World!” test for CSS3-based UI performance on low-end devices, plus as a test case for combining presentation-level CSS3 automation with app-level explicit status handling. We took down a “yes” for both.

    UI

    By the time we had put together a dummy around these concepts, the first design mockups from Time Out came in so that we could start to implement the front end and think about connecting it to the data sources.

    For presentation, we tried hard to keep the HTML and CSS to the absolute minimum. Mozilla’s GAIA examples being a very valuable source of ideas once more.

    Again, targeting Firefox OS alone allowed us to break free of the backwards compatibility hell that we were still living in, desktop-wise. No one would ask us Will it display well in IE8? or worse things. We could finally use real <section>, <nav>, <header>, and <menu> tags instead of an army of different classes of <div>. What a relief!

    The clear, rectangular, flat and minimalistic design we got from Time Out also did its part to keep the UI HTML simple and clean. After we were done with creating and styling the UI for 15 screens, our HTML had only ~250 lines. We later improved that to 150 while extending the functionality, but that’s a different story.

    Speaking of styling, not everything that had looked good on desktop Firefox even in its responsive design view displayed equally well on actual mobile devices. Some things that we fought with and won:

    Scale: The app looked quite different when viewed on the reference device (a TurkCell branded ZTE device that Mozilla had sent us for testing) and on our brand new Nexus 4s:

    After a lot of experimenting, tearing some hair and looking around how others had addressed graceful, proportional scaling for a consistent look & feel across resolutions, we stumbled upon this magic incantation:

    <meta name="viewport" content="user-scalable=no, initial-scale=1,
    maximum-scale=1, width=device-width" />

    What it does, to quote an article at Opera, is to tell the browser that there is “No scaling needed, thank you very much. Just make the viewport as many pixels wide as the device screen width”. It also prevents accidental scaling while the map is zoomed. There is more information on the topic at MDN.

    Then there are things that necessarily get pixelated when scaled up to high resolutions, such as the API based venue images. Not a lot we could do about that. But we could at least make the icons and logo in the app’s chrome look nice in any resolution by transforming them to SVG.

    Another issue on mobile devices was that users have to touch the content in order to scroll it, so we wanted to prevent the automatic highlighting that comes with that:

    li, a, span, button, div
    {
        outline:none;
        -moz-tap-highlight-color: transparent;
        -moz-user-select: none;
        -moz-user-focus:ignore
    }

    We’ve since been warned that suppressing the default highlighting can be an issue in terms of accessibility, so you might wanted to consider this carefully.

    Connecting to the live data sources

    So now we had the app’s presentational base structure and the UI HTML / CSS in place. It all looked nice with dummy data, but it was still dead.

    Trouble with bringing it to life was that Time Out was in the middle of a big project to replace its legacy API with a modern Graffiti based service and thus had little bandwidth for catering to our project’s specific needs. The new scheme was still prototypical and quickly evolving, so we couldn’t build against it.

    The legacy construct already comprised a proxy that wrapped the raw API into something more suitable for consumption by their iOS and Android apps, but after close examination we found that we better re-re-wrap that on the fly in PHP for a couple of purposes:

    • Adding CORS support to avoid XSS issues, with the API and the app living in different subdomains of timeout.com,
    • stripping API output down to what the FFOS app really needed, which we could see would reduce bandwidth and increase speed by magnitude,
    • laying the foundation for harvesting of API based data for offline use, which we already knew we’d need to do later

    As an alternative to server-side CORS support, one could also think of using the SystemXHR API. It is a mighty and potentially dangerous tool however. We also wanted to avoid any needless dependency on FFOS-only APIs.

    So while the approach wasn’t exactly future proof, it helped us a lot to get to results quickly, because the endpoints that the app was calling were entirely of our own choice and making, so that we could adapt them as needed without time loss in communication.

    Populating content elements

    For all things dynamic and API-driven, we used the same approach at making it visible in the app:

    • Have a simple, minimalistic, empty, hidden, singleton HTML template,
    • clone that template (N-fold for repeated elements),
    • ID and fill the clone(s) with API based content.
    • For super simple elements, such as <li>s, save the cloning and whip up the HTML on the fly while filling.

    As an example, let’s consider the filters for finding venues. Cuisine is a suitable filter for restaurants, but certainly not for museums. Same is true for filter values. There are vegetarian restaurants in Barcelona, but certainly no vegetarian bars. So the filter names and lists of possible values need to be asked of the API after the venue type is selected.

    In the UI, the collapsible category filter for bars & pubs looks like this:

    The template for one filter is a direct child of the one and only

    <div id="templateContainer">

    which serves as our central template repository for everything cloned and filled at runtime and whose only interesting property is being invisible. Inside it, the template for search filters is:

    <div id="filterBoxTemplate">
      <span></span>
      <ul></ul>
    </div>

    So for each filter that we get for any given category, all we had to do was to clone, label, and then fill this template:

    $('#filterBoxTemplate').clone().attr('id', filterItem.id).appendTo(
    '#categoryResultScreen .filter-container');
    ...
    $("#" + filterItem.id).children('.filter-button').html(
    filterItem.name);

    As you certainly guessed, we then had to to call the API once again for each filter in order to learn about its possible values, which were then rendered into <li> elements within the filter‘s <ul> on the fly:

    $("#" + filterId).children('.filter_options').html(
    '<li><span>Loading ...</span></li>');
    
    apiClient.call(filterItem.api_method, function (filterOptions)
    {
      ...
      $.each(filterOptions, function(key, option)
      {
        var entry = $('<li filterId="' + option.id + '"><span>'
          + option.name + '</span></li>');
    
        if (selectedOptionId && selectedOptionId == filterOptionId)
        {
          entry.addClass('filter-selected');
        }
    
        $("#" + filterId).children('.filter_options').append(entry);
      });
    ...
    });

    DOM based caching

    To save bandwidth and increase responsiveness in on-line use, we took this simple approach a little further and consciously stored more application level information in the DOM than needed for the current display if that information was likely needed in the next step. This way, we’d have easy and quick local access to it without calling – and waiting for – the API again.

    The technical way we did so was a funny hack. Let’s look at the transition from the search result list to the venue detail view to illustrate:

    As for the filters above, the screen class for the detailView has an init() method that populates the DOM structure based on API input as encapsulated on the application level. The trick now is, while rendering the search result list, to register anonymous click handlers for each of its rows, which – JavaScript passing magic – contain a copy of, rather than a reference to, the venue objects used to render the rows themselves:

    renderItems: function (itemArray)
    {
      ...
    
      $.each(itemArray, function(key, itemData)
      {
        var item = screen.dom.resultRowTemplate.clone().attr('id',
          itemData.uid).addClass('venueinfo').click(function()
        {
          $('#mapScreen').hide();
          screen.showDetails(itemData);
        });
    
        $('.result-name', item).text(itemData.name);
        $('.result-type-label', item).text(itemData.section);
        $('.result-type', item).text(itemData.subSection);
    
        ...
    
        listContainer.append(item);
      });
    },
    
    ...
    
    showDetails: function (venue)
    {
      require(['screen/detailView'], function (detailView)
      {
        detailView.init(venue);
      });
    },

    In effect, there’s a copy of the data for rendering each venue’s detail view stored in the DOM. But neither in hidden elements nor in custom attributes of the node object, but rather conveniently in each of the anonymous pass-by-value-based click event handlers for the result list rows, with the added benefit that they don’t need to be explicitly read again but actively feed themselves into the venue details screen as soon a row receives a touch event.

    And dummy feeds

    Finishing the app before MWC 2013 was pretty much a race against time, both for us and for Time Out’s API folks, who had an entirely different and equally – if not more so – sportive thing to do. Therefore they had very limited time for adding to the (legacy) API that we were building against. For one data feed, this meant that we had to resort to including static JSON files into the app’s manifest and distribution; then use relative, self-referencing URLs as fake API endpoints. The illustrated list of top venues on the app’s main screen was driven this way.

    Not exactly nice, but much better than throwing static content into the HTML! Also, it kept the display code already fit for switching to the dynamic data source that eventually materialized later, and compatible with our offline data caching strategy.

    As the lack of live data on top venues then extended right to their teaser images, we made the latter physically part of the JSON dummy feed. In Base64 :) But even the low-end reference device did a graceful job of handling this huge load of ASCII garbage.

    State preservation

    We had a whopping 5M of local storage to spam, and different plans already (as well as much higher needs) for storing the map and application data for offline use. So what to do with this liberal and easily accessed storage location? We thought we could at least preserve the current application state here, so you’d find the app exactly as you left it when you returned to it.

    Map

    A city guide is the very showcase of an app that’s not only geo aware but geo centric. Maps fit for quick rendering and interaction in both online and offline use were naturally a paramount requirement.

    After looking around what was available, we decided to go with Leaflet, a free, easy to integrate, mobile friendly JavaScript library. It proved to be really flexible with respect to both behaviour and map sources.

    With its support for pinching, panning and graceful touch handling plus a clean and easy API, Leaflet made us arrive at a well-usable, decent-looking map with moderate effort and little pain:

    For a different project, we later rendered the OSM vector data for most of Europe into terabytes of PNG tiles in cloud storage using on-demand cloud power. Which we’d recommend as an approach if there’s a good reason not to rely on 3rd party hosted apps, as long as you don’t try this at home; Moving the tiles may well be slower and more costly than their generation.

    But as time was tight before the initial release of this app, we just – legally and cautiously(!) – scraped ready-to use OSM tiles off MapQuest.com.

    The packaging of the tiles for offline use was rather easy for Barcelona because about 1000 map tiles are sufficient to cover the whole city area up to street level (zoom level 16). So we could add each tile as a single line into the manifest.appache file. The resulting, fully automatic, browser-based download on first use was only 10M.

    This left us with a lot of lines like

    /mobile/maps/barcelona/15/16575/12234.png
    /mobile/maps/barcelona/15/16575/12235.png
    ...

    in the manifest and wishing for a $GENERATE clause as for DNS zone files.

    As convenient as it may seem to throw all your offline dependencies’ locations into a single file and just expect them to be available as a consequence, there are significant drawbacks to this approach. The article Application Cache is a Douchebag by Jake Archibald summarizes them and some help is given at Html5Rocks by Eric Bidleman.

    We found at the time that the degree of control over the current download state, and the process of resuming the app cache load in case that the initial time users spent in our app didn’t suffice for that to complete was rather tiresome.

    For Barcelona, we resorted to marking the cache state as dirty in Local Storage and clearing that flag only after we received the updateready event of the window.applicationCache object but in the later generalization to more cities, we moved the map away from the app cache altogether.

    Offline storage

    The first step towards offline-readiness was obviously to know if the device was online or offline, so we’d be able to switch the data source between live and local.

    This sounds easier than it was. Even with cross-platform considerations aside, neither the online state property (window.navigator.onLine), the events fired on the <body> element for state changes (“online” and “offline”, again on the <body>), nor the navigator.connection object that was supposed to have the on/offline state plus bandwidth and more, really turned out reliable enough.

    Standardization is still ongoing around all of the above, and some implementations are labeled as experimental for a good reason :)

    We ultimately ended up writing a NetworkStateService class that uses all of the above as hints, but ultimately and very pragmatically convinces itself with regular HEAD requests to a known live URL that no event went missing and the state is correct.

    That settled, we still needed to make the app work in offline mode. In terms of storage opportunities, we were looking at:

    Storage Capacity Updates Access Typical use
    App / app cache, i.e. everything listed in the file that the value of appcache_path in the app‘s webapp.manifest points to, and which is and therefore downloaded onto the device when the app is installed. <= 50M. On other platforms (e.g. iOS/Safari), user interaction required from 10M+. Recommendation from Moziila was to stay <2M. Hard. Requires user interaction / consent, and only wholesale update of entire app possible. By (relative) path HTML, JS, CSS, static assets such as UI icons
    LocalStorage 5M on UTF8-platforms such as FFOS, 2.5M in UTF16, e.g. on Chrome. Details here Anytime from app By name Key-value storage of app status, user input, or entire data of modest apps
    Device Storage (often SD card) Limited only by hardware Anytime from app (unless mounted as UDB drive when cionnected to desktop computer) By path, through Device Storage API Big things
    FileSystem API Bad idea
    Database Unlimited on FFOS. Mileage on other platforms varies Anytime from app Quick and by arbitrary properties Databases :)

    Some aspects of where to store the data for offline operation were decided upon easily, others not so much:

    • the app, i.e. the HTML, JS, CSS, and UI images would go into the app cache
    • state would be maintained in Local Storage
    • map tiles again in the app cache. Which was a rather dumb decision, as we learned later. Barcelona up to zoom level 16 was 10M, but later cities were different. London was >200M and even reduced to max. zoom 15 still worth 61M. So we moved that to Device Storage and added an actively managed download process for later releases.
    • The venue information, i.e. all the names, locations, images, reviews, details, showtimes etc. of the places that Time Out shows in Barcelona. Seeing that we needed lots of space, efficient and arbitrary access plus dynamic updates, this had to to go into the Database. But how?

    The state of affairs across the different mobile HTML5 platforms was confusing at best, with Firefox OS already supporting IndexedDB, but Safari and Chrome (considering earlier versions up to Android 2.x) still relying on a swamp of similar but different sqlite / WebSQL variations.

    So we cried for help and received it, as always when we had reached out to the Mozilla team. This time in the form of a pointer to pouchDB, a JS-based DB layer that at the same time wraps away the different native DB storage engines behind a CouchDB-like interface and adds super easy on-demand synchronization to a remote CouchDB-hosted master DB out there.

    Back last year it still was in pre-alpha state but very usable already. There were some drawbacks, such as the need for adding a shim for WebSql based platforms. Which in turn meant we couldn’t rely on storage being 8 bit clean, so that we had to base64 our binaries, most of all the venue images. Not exactly pouchDB’s fault, but still blowing up the size.

    Harvesting

    The DB platform being chosen, we next had to think how we’d harvest all the venue data from Time Out’s API into the DB. There were a couple of endpoints at our disposal. The most promising for this task was proximity search with no category or other restrictions applied, as we thought it would let us harvest a given city square by square.

    Trouble with distance metrics however being that they produce circles rather than squares. So step 1 of our thinking would miss venues in the corners of our theoretical grid

    while extending the radius to (half the) the grid’s diagonal, would produce redundant hits and necessitate deduplication.

    In the end, we simply searched by proximity to a city center location, paginating through the result indefinitely, so that we could be sure to to encounter every venue, and only once:

    Technically, we built the harvester in PHP as an extension to the CORS-enabled, result-reducing API proxy for live operation that was already in place. It fed the venue information in to the master CouchDB co-hosted there.

    Time left before MWC 2013 getting tight, we didn’t spend much time on a sophisticated data organization and just pushed the venue information into the DB as one table per category, one row per venue, indexed by location.

    This allowed us to support category based and area / proximity based (map and list) browsing. We developed an idea how offline keyword search might be made possible, but it never came to that. So the app simply removes the search icon when it goes offline, and puts it back when it has live connectivity again.

    Overall, the app now

    • supported live operation out of box,
    • checked its synchronization state to the remote master DB on startup,
    • asked, if needed, permission to make the big (initial or update) download,
    • supported all use cases but keyword search when offline.

    The involved components and their interactions are summarized in this diagram:

    Organizing vs. Optimizing the code

    For the development of the app, we maintained the code in a well-structured and extensive source tree, with e.g. each JavaScript class residing in a file of its own. Part of the source tree is shown below:

    This was, however, not ideal for deployment of the app, especially as a hosted Firefox OS app or mobile web site, where download would be the faster, the fewer and smaller files we had.

    Here, Require.js came to our rescue.

    It provides a very elegant way of smart and asynchronous requirement handling (AMD), but more importantly for our purpose, comes with an optimizer that minifies and combines the JS and CSS source into one file each:

    To enable asynchronous dependency management, modules and their requirements must be made known to the AMD API through declarations, essentially of a function that returns the constructor for the class you’re defining.

    Applied to the search result screen of our application, this looks like this:

    define
    (
      // new class being definied
      'screensSearchResultScreen',
    
      // its dependencies
      ['screens/abstractResultScreen', 'app/applicationController'],
    
      // its anonymous constructor
      function (AbstractResultScreen, ApplicationController)
      {
        var SearchResultScreen = $.extend(true, {}, AbstractResultScreen,
        {
          // properties and methods
          dom:
          {
            resultRowTemplate: $('#searchResultRowTemplate'),
            list: $('#search-result-screen-inner-list'),
            ...
          }
          ...
        }
        ...
    
        return SearchResultScreen;
      }
    );

    For executing the optimization step in the build & deployment process, we used Rhino, Mozilla’s Java-based JavaScript engine:

    java -classpath ./lib/js.jar:./lib/compiler.jar
      org.mozilla.javascript.tools.shell.Main ./lib/r.js -o /tmp/timeout-webapp/
      $1_config.js

    CSS bundling and minification is supported, too, and requires just another call with a different config.

    Outcome

    Four weeks had been a very tight timeline to start with, and we had completely underestimated the intricacies of taking HTML5 to a mobile and offline-enabled context, and wrapping up the result as a Marketplace-ready Firefox OS app.

    Debugging capabilities in Firefox OS, especially on the devices themselves, were still at an early stage (compared to clicking about:app-manager today). So the lights in our Cologne office remained lit until pretty late then.

    Having built the app with a clear separation between functionality and presentation also turned out a wise choice when a week before T0 new mock-ups for most of the front end came in :)

    But it was great and exciting fun, we learned a lot in the process, and ended up with some very useful shiny new tools in our box. Often based on pointers from the super helpful team at Mozilla.

    Truth be told, we had started into the project with mixed expectations as to how close to the native app experience we could get. We came back fully convinced and eager for more.

    In the end, we made the deadline and as a fellow hacker you can probably imagine our relief. The app finally even received its 70 seconds of fame, when Jay Sullivan shortly demoed it at Mozilla’s MWC 2013 press conference as a showcase for HTML5′s and Firefox OS’s offline readiness (Time Out piece at 7:50). We were so proud!

    If you want to play with it, you can find the app in the marketplace or go ahead try it online (no offline mode then).

    Since then, the Time Out Firefox OS app has continued to evolve, and we as a team have used the chance to continue to play with and build apps for FFOS. To some degree, the reusable part of this has become a framework in the meantime, but that’s a story for another day..

    We’d like to thank everyone who helped us along the way, especially Taylor Wescoatt, Sophie Lewis and Dave Cook from Time Out, Desigan Chinniah and Harald Kirschner from Mozilla, who were always there when we needed help, and of course Robert Nyman, who patiently coached us through writing this up.

  7. HTML5 and CSS3: Exploring Mobile Possibilities – presentation at London Ajax Mobile Event

    In the beginning of July, I was attending and giving a presentation at the London Ajax Mobile Event about possibilities offered by HTML5 and CSS3 when it comes to developing mobile web sites and applications.

    Short introduction of me

    Being my first post here at Mozilla Hacks, I thought I’d start by briefly introducing myself first: My name is Robert Nyman and I’ve recently joined Mozilla as a Technical Evangelist, talking about HTML5, the Open Web and how we can help web developers. I’ve been working since 1999 with Front End development for the web, and I regularly blog at http://robertnyman.com, tweet as @robertnyman and love to travel and meet people!

    The London Ajax Mobile Event

    The conference took place at the Vodafone headquarters in London, arranged by Sitepen CEO and Dojo co-founder Dylan Schiemann. It was packed to the brim with speakers, from early morning raging on into the night. Various talks were given on a number of topics – from mobile apps and implementations to a more experimenting approach and future visions.

    My presentation

    You can see the slides from my HTML5 and CSS3: Exploring Mobile Possibilities below or download the slides at SlideShare

    The aim of my talk was to give both an introduction and reiterate on some of the important options we have when developing web sites, especially when it comes to the mobile world. With the CSS3 field I covered CSS Media Queries and Flex Box and the options they give us in creating more flexible layouts and presentation alternatives. I also spoke about CSS Transitions and Animations and how they can assist in an easy manner to get nice effects, that are also hardware-accelerated on certain devices.

    When it comes to the HTML5 part, I’m excited by all the new HTML5 form elements and how they can improve both user experience and the input of data. As support grows for this in web browsers, I believe it will make things a lot easier for both developers and end users.

    I briefly touched on link protocols, such as tel: and sms:, to trigger mobile-specific actions when activating a link, and then various useful APIs such as Web Storage, Offline Applications, History API and Geolocation (not all necessary official HTML5 APIs, but usually used in conjunction with them).

    I ended the talk with touching on tool such as Steve Souders’ Mobile Perf bookmarklet and weinre, for remote debugging on mobile devices.

    You and mobile

    What I am interested in is if you are developing for a multitude of mobile and other devices, what you believe are the biggest obstacles as well as the most promising options. Any thoughts, please let me know!

  8. Interview with Giovanny Granada, winner of the Geolocation Dev Derby

    Giovanny Granada won the most recent Geolocation Dev Derby with GoGeoTweet, his wonderful web-based visualization of Twitter activity happening nearby. Recently, I had the chance to learn more about Giovanny: his work, his ambitions, and his thoughts on the future of web development.

    The interview

    How did you become interested in web development?

    I became interested in development because I saw that my motivation was to innovate, create new things, and experiment with different technologies toward the goal of creating new and complete tools.

    Can you tell us a little about how GoGeoTweet works?

    The application works by using the Geolocation API and the Twitter API to show Tweets published within 1km of you–a very useful tool if you’re at an event or a special place and would like to know what Twitter users are saying.

    What was your biggest challenge in developing GoGeoTweet?

    The biggest challenge was using the Geolocation API and the Twitter API show to show only Tweets within a 1km radius.

    What makes the web an exciting platform for you?

    The ability to create things using technologies of all types and obtain great results. Building free alternatives that are full of new experiences and that share knowledge is something you can only do on the web.

    What new web technologies are you most excited about?

    Right now I’m excited to explore and learn about WebGL, HTML5 and all of the Web APIs that are making the web better.

    If you could change one thing about the web, what would it be?

    I would change web standards to limit the way applications can be created (for example, prohibiting Flash), to discourage applications that do not use modern technologies. I am totally sure that the web would be better if that changed.

    Do you have any advice for other ambitious web developers?

    If you can imagine you can create! Let’s do it! There are no limits!

    Further reading

  9. Located: Winners of the September Dev Derby on Geolocation.

    With more people going mobile and taking the Web with them, we thought Geolocation was a great topic for the September Dev Derby.

    Web developers explored a number of ways to bring your physical location into the Web experience and we had 16 demos submitted for the Dev Derby.

    Voting was tough this month, but once the votes were counted, we had our winners circle:
    Winners of the September Dev Derby - Geolocation

    1st Place: Urban arteries by Jaume Sánchez aka spite
    2nd Place: I Need A… by David aka d-b-f
    3rd Place: Find Street Art by Aaron Has

    Runners-up:
    Geosocial
    Geoapp

    Thanks to everyone that participated in the September Dev Derby and congratulations to the winners! A special shout out to Jaume for taking 1st place after submitting amazing demos for the past three Dev Derbies. Check out spite’s MDN profile to see what he’s done in the past.

    NOTE: We recently updated our Dev Derby rules to allow developers to participate in multiple contests until they win 1st place. That means if you’ve submitted awesome demos and come up short in the past, you still have a chance to win that top spot in future Dev Derbies… so keep those demos coming!

    If you’re working on CSS Media Queries, you have a few more days to join the October Dev Derby.

    Or get ready for the joy of painting in the November Dev Derby with Canvas. We’re looking forward to seeing some creative demos next month. So hack on!

  10. Where on earth? This month's Developer Derby is all about geolocation.

    Another month, another Developer Derby. This month we want you to play with something that is not part of the HTML5 stack and we feel it doesn’t get the love it deserves from developers: the geolocation API. Firefox has supported this API for a long time and you can do some pretty cool things with it.

    So, what is the Geolocation API? In essence it allows you to detect where the user of your product is at the moment. The location data is found by different means: GPS location, mobile phone masts or wireless hub location. If you turn off wireless on your laptop and you have no 3G connectivity, the API will not be able to get any data.

    Using geolocation is incredibly simple. You ask the browser to tell you what the current location is with a method on the navigator.geolocation object:

    navigator.geolocation.getCurrentPosition(
      success,
      failure,
      { parameters }
    );

    Where success is the function that is called when the browser found a location, failure is the function called when there was an error and properties is an object that can contain a few parameters. The parameters are the Boolean enableHighAccuracy, the maximumAge of the location before the browser should ask for a new one and the timeout in milliseconds after which the browser should stop trying to find a location.

    Each function (success and failure) get a parameter with which to do your coding magic. The success function will get a location object when everything went well. This location object has the following properties: A timestamp telling you when the reading was done and a coordinates object with the following properties: accuracy, altitude, altitudeAccuracy, heading, latitude, longitude and speed.

    Some of these are dependent on having more than one reading as for example heading and speed are calculated from the distance in latitude and longitude from reading to reading.

    On Firefox you have an extra object called address which is the result of reverse geocoding the latitude and longitude. Reverse geocoding is normally done with an API but in the case of Firefox we have it baked in. When I currently do a call to getCurrentPosition I get the following result on Firefox:

    Timestamp: 1315378919289
    Coordinates:
        Accuracy: 18000
        Altitude: 0
        AltitudeAccuracy: 0
        Heading: NaN
        Latitude: 50.06465
        Longitude: 19.94498
        Speed: NaN
    
    Address:
        City: Kraków
        Country: Poland
        CountryCode: PL
        County: Kraków County
        Postal Code: null
        Premises: null
        Street: Lubicz
        Street Number: 1

    You can Run this test for yourself and see the code here on JSFiddle:

    The failure function gets an error object with a code property. This property can have three values: 1 is a permission denied error, 2 is a position unavailable error and 3 is a timeout.

    In addition to the getCurrentPosition method you also have a watchPosition method which keeps firing when a new location was found. The parameters are the same and when you keep reading (for example on a mobile device) then you will get values for the heading and the speed. You can stop watching the position change using the clearWatch method.

    Using watchPosition is very cool when you are on the go. Check the Geolocation demo page and turn the watch position on and off with the button on the bottom.

    In essence, this is what the map app on your mobile phone does.

    What can you do with this?

    Well, what you get is a latitude and longitude of your end user. This can be used with all kind of geo platforms like for example GeoNames to find places of interest around you.

    You could also allow people to set markers for their friends on a map, collaboratively paint with them, find photos, tweets or foursquare checkins around you – a lot is possible when you come from lat/lon.

    Resources

    We will also soon run a webinar on geolocation, stay tuned!

    Happy hacking!