In this post, I’ll be talking about how to create a basic note-taking app that syncs local and remote content if you are online and defaults to saving locally if offline.
Using Redis on the server-side
When adding records in Redis, we aren’t working with a relational database like in MySQL or PostgreSQL. We are working with a structure like IndexedDB where there are keys and values. So what do we need when we only have keys and values to work with for a notes app? We need unique ids to reference each note and a hash of the note metadata. The metadata in this example, consists of the new unique id, a creation timestamp and the text.
Below is a way of creating an id with Redis in Node and then saving the note’s metadata.
// Let's create a unique id for the new note.
client.incr('notes:counter',function(err, id){
...
// All note ids are referenced by the user's email and id.var keyName ='notes:'+ req.session.email+':'+ id;var timestamp = req.body.timestamp||Math.round(Date.now()/1000);// Add the new id to the user's list of note ids.
client.lpush('notes:'+ req.session.email, keyName);// Add the new note to a hash.
client.hmset(keyName,{
id: id,
timestamp: timestamp,
text: finalText
});
...
});
This gives us the following key pattern for all notes on the server-side:
notes:counter contains all unique ids starting at 1.
notes:<email> contains all the note ids that are owned by the user. This is a list that we reference when we want to loop through all the user’s notes to retrieve the metadata.
notes:<email>:<note id> contains the note metadata. The user’s email address is used as a way to reference this note to the correct owner. When a user deletes a note, we want to verify that it matches the same email that they are logged in with, so you don’t have someone deleting a note that they don’t own.
Adding IndexedDB on the client-side
Working with IndexedDB requires more code than localStorage. But because it is asynchronous, it makes it a better option for this app. The main reason for why it is a better option is two-fold:
You don’t want to wait around for all your notes to process before the page renders all elements. Imagine having thousands of notes and having to wait for all of them to loop through before anything on the page appears.
You can’t save note objects as objects – you have to convert them to strings first, which means you will have to convert them back to objects before they are rendered. So something like { id: 1, text: 'my note text', timestamp: 1367847727 } would have to be stringified in localStorage and then parsed back after the fact. Now imagine doing this for a lot of notes.
Both do not equate to an ideal experience for the user – but what if we want to have the ease of localStorage’s API with the asynchronous features of IndexedDB? We can use Gaia’s async_storage.js file to help merge the two worlds.
If we’re offline, we need to do two things similar to the server-side:
Save a unique id for the note and apply it in an array of ids. Since we can’t reference a server-side id created by Redis, we’ll use a timestamp.
Save a local version of the note metadata.
var data ={
content: rendered,
timestamp: id,
text: content
};
asyncStorage.setItem(LOCAL_IDS,this.localIds,function(){
asyncStorage.setItem(LOCAL_NOTE + id, data,function(){
...
});});
The structure of the IndexedDB keys are very similar to the Redis ones. The pattern is as follows:
All local ids are saved in a localNoteIds array
All local note objects are saved in note:local:<id>
All remote/synced ids are saved in a noteIds array
All remote/synced note objects are saved in note:<id>
Local notes use a timestamp for their unique id and this is converted to a valid server id once Redis saves the data
Once we’re online, we can upload the local notes, save the remote ones on the client-side and then delete the local ones.
Triggering note.js on the client-side
Whenever we refresh the page, we need to attempt a sync with the server. If we are offline, let’s flag that and only grab what we have locally.
/**
* Get all local and remote notes.
* If online, sync local and server notes; otherwise load whatever
* IndexedDB has.
*/
asyncStorage.getItem('noteIds',function(rNoteIds){
note.remoteIds= rNoteIds ||[];
asyncStorage.getItem('localNoteIds',function(noteIds){
note.localIds= noteIds ||[];
$.get('/notes',function(data){
note.syncLocal();
note.syncServer(data);}).fail(function(data){
note.offline=true;
note.load('localNoteIds','note:local:');
note.load('noteIds','note:');});});});
Almost done!
The code above provides the basics for a CRD notes app with support for local and remote syncing. But we’re not done yet.
On Safari, IndexedDB is not supported as they still use WebSQL. This means none of our IndexedDB code will work. To make this cross-browser compatible, we need to include a polyfill for browsers that only support WebSQL. Include this before the rest of the code and IndexedDB support should work.
This is a guest post from Parashuram Narasimhan on how to use IndexedDB today.
Using the polyfills mentioned in this article, web developers can start using IndexedDB APIs in their applications and support a wider range of browsers.
The IndexedDB API has matured into a stable specification with support by major browser vendors. However, the specification is still not supported in all browsers, making it harder to use in production. There are also some interesting differences in the implementations among browsers that support the specification. This article explores a couple of polyfills that could be leveraged to enable developers to use IndexedDB across different browsers.
Polyfill using WebSql
WebSql was one of the first specifications for data storage in the browsers and is supported in some browsers that do not have an implementation of IndexeddB yet. However, the specification is no longer in active maintenance. As mentioned in the document: “it was on the W3C Recommendation track but specification work has stopped. The specification reached an impasse: all interested implementors have used the same SQL backend (Sqlite), but we need multiple independent implementations to proceed along a standardisation path”.
This polyfill using WebSql utilizes the WebSQL implementations to expose the IndexedDB APIs. To use the polyfill, simply link or include the indexeddb.shim.js file.
The polyfill assigns window.indexedDB to be window.mozIndexedDB, window.webkitIndexedDB or window.msIndexedDB; if any of those implementations are available. If no implementation is available, the polyfill assigns a ‘window.shimIndexedDB’ to window.indexedDB. Web applications can thus start using window.indexedDB as the starting point for all database operations. Internally, the polyfill uses the WebSql tables to store object store data and borrows heavily from the IndexedDB implementation in Firefox. For example, it has separate tables and databases to maintain a record of the database versions, or the values for object store definitions (IDBObjectStoreParameters). More implementation details about the internals can be found in the blog post on it.
The polyfill has been tested to work with various IndexedDB libraries like PouchDB, LINQ2IndexedDB, JQuery-IndexedDB and DB.JS. Since it leverages WebSql, applications can write code according to the IndexedDB API on web browsers like Opera and Safari, and additionally on mobile browsers like Safari on iPad/iPhone, or mobile development platforms like Cordova that have WebSql enabled browsers embedded in them.
It is implemented in Javascript and is a work in progress. Note that it may not fully confirm to the specification and you discover issues, you could file a bug or send a pull request to the source repository.
Polyfill for setVersion
An earlier version of the IndexedDB Specification used the “setVersion” call to change the version of the IndexedDB Database. This was later revised to an easier to use “onupgradeneeded” callback. Though Internet Explorer and Firefox support the newer “onupgradeneeded” to initiate database version changes for creating or deleting objectStores and Indexes, Google Chrome (22.0.1194.0 canary) still uses the older setVersion.
All browsers will soon use the newer “onupgradeneded” method soon; till then, this simple polyfill for setVersion lets you use the IndexedDB API with the onupgradeneeded method.
The polyfill substitutes the indexeddb.open() call with an openReqShim() call. The openReqShim() call invokes setVersion() and then converts it to the “onupgradeneeded” callback if only “setVersion()” is supported. If the implementation supports the “onupgradeneeded”, openReqShim() is simply a transparent call to the indexedb.open method. Hence, indexeddb.open() calls should be replaced with the openReqShim() call to use this polyfill.
A question that I get asked a lot is why Firefox doesn’t support the FileSystem API. Usually, but not always, they are referring specifically to the FileSystem and FileWriter specifications which Google is implementing in Chrome, and which they have proposed for standardization in W3C.
The answer is somewhat complex, and depends greatly on what exact capabilities of the above two specifications the person is actually wanting to use. The specifications are quite big and feature full, so it’s no surprise that people are wanting to do very different things with it. This blog post is an attempt at giving my answer to this question and explain why we haven’t implemented the above two specifications. But note that this post represents my personal opinion, intended to spur more conversation on this topic.
As stated above, people asking for “FileSystem API support” in Firefox are actually often interested in solving many different problems. In my opinion most, but so far not all, of these problems have better solutions than the FileSystem API. So let me walk through them below.
Storing resources locally
Probably the most common thing that people want to do is to simply store a set of resources so that they are available without having to use the network. This is useful if you need quick access to the resources, or if you want to be able to access them even if the user is offline. Games are a very common type of application where this is needed. For example an enemy space ship might have a few associated images, as well as a couple of associated sounds, used when the enemy is moving around the screen and shooting. Today, people generally solve this by storing the images and sound files in a file system, and then store the file names of those files along with things like speed and firepower of the enemy.
However it seems a bit non-optimal to me to have to store some data separated from the rest. Especially when there is a solution which can store both structured data as well as file data. IndexedDB treats file data just like any other type of data. You can write a File or a Blob into IndexedDB just like you can store strings, numbers and JavaScript objects. This is specified by the IndexedDB spec and so far implemented in both the Firefox and IE implementations of IndexedDB. Using this, you can store all information that you need in one place, and a single query to IndexedDB can return all the data you need. So for example, if you were building a web based email client, you could store an object like:
{
subject: "Hi there",
body: "Hi Sven,\\nHow are you doing...",
attachments: [blob1, blob2, blob3]
}
Another advantage here is that there’s no need to make up file names for resources. Just store the File or Blob object. No name needed.
In Firefox’s IndexedDB implementation (and I believe IE’s too) the files are transparently stored outside of the actual database. This means that performance of storing a file in IndexedDB is just as good as storing the file in a filesystem. It does not bloat the database itself slowing down other operations, and reading from the file means that the implementation just reads from an OS file, so it’s just as fast as a filesystem.
Firefox IndexedDB implementation is even smart enough that if you store the same Blob multiple files in a IndexedDB database it just creates one copy of the file. Writing further references to the same Blob just adds to an internal reference counter. This is completely transparent to the web page, the only thing it will notice is faster writes and less resource use. However I’m not sure if IE does the same, so check there first before relying on it.
Access pictures and music folders
The second most common thing that people ask for related to a file system APIs is to be able to access things like the user’s picture or music libraries. This is something that the FileSystem API submitted to W3C doesn’t actually provide, though many people seems to think it does. To satisfy that use-case we have the DeviceStorage API. This API allows full file system capabilities for “user files”. I.e. files that aren’t specific to a website, but rather resources that are managed and owned by the user and that the user might want to access through several apps. Such as photos and music. The DeviceStorage API is basically a simple file system API mostly optimized for these types of files.
We’re still in the process of specifying and implementing this API. It’s available to test with in recent nightly builds, but so far isn’t enabled by default. The main problem with exposing this functionality to the web is security. You wouldn’t want just any website to read or modify your images. We could put up a prompt like we do with the GeoLocation API, given that this API potentially can delete all your pictures from the last 10 years, we probably want something more. This is something we are actively working on. But it’s definitely the case here that security is the hard part here, not implementing the low-level file operations.
Low-level file manipulation
A less common request is the ability to do low-level create, read, update and delete (CRUD) file operations. For example being able to write 10 bytes in the middle of a 10MB file. This is not something IndexedDB supports right now, it only allows adding and removing whole files. This is supported by the FileWriter specification draft. However I think this part of this API has some pretty fundamental problems. Specifically there are no locking capabilities, so there is no way to do multiple file operations and be sure that another tab didn’t modify or read the file in between those operations. There is also no way to do fsync which means that you can’t implement ACID type applications on top of FileWriter, such as a database.
We have instead created an API with the same goal, but which has capabilities for locking a file and doing multiple operations. This is done in a way to ensure that there is no risk that pages can forget to unlock a file, or that deadlocks can occur. The API also allows fsync operations which should enable doing things like databases on top of FileHandle. However most importantly, the API is done in such a way that you shouldn’t need to nest asynchronous callbacks as much as with FileWriter. In other words it should easier to use for authors. You can read more about FileHandle at
There is one more capability that exist in the FileSystem API not covered above. The specification introduces a new filesystem: URL scheme. When loading URLs from filesystem: it returns the contents of files in stored using the FileSystem API. This is a very cool feature for a couple of reasons. First of all these URLs are predictable. Once you’ve stored a file in the file system, you always know which URL can be used to load from it. And the URL will continue to work as long as the file is stored in the file system, even if the web page is reloaded. Second, relative URLs work with the filesystem: scheme. So you can create links from one resource stored in the filesystem to another resource stored in the filesystem.
Firefox does support the blob: URL scheme, which does allow loading data from a Blob anywhere where URLs can be used. However it doesn’t have the above mentioned capabilities. This is something that I’d really like to find a solution for. If we can’t find a better solution, implementing the Google specifications is definitely an option.
Conclusions
As always when talking about features to be added to the web platform it’s important to talk about use cases and capabilities, and not jump directly to a particular solution. Most of the use cases that the FileSystem API aims to solve can be solved in other ways. In my opinion many times in better ways.
This is why we haven’t prioritized implementing the FileSystem API, but instead focused on things like making our IndexedDB implementation awesome, and coming up with a good API for low-level file manipulation.
Focusing on IndexedDB has also meant that we very soon have a good API for basic file storage available in 3 browsers: IE10, Firefox and Chrome.
On a related note, we just fixed the last known spec compliance issues in our IndexedDB implementation, so Firefox 16 will ship with IndexedDB unprefixed!
As always, we’re very interested in getting feedback from other people, especially from web developers. Do you think that FileSystem API is something we should prioritize? If so, why?
TL;DR: we have to stop advocating localStorage as a great opportunity for storing data as it performs badly. Sadly enough the alternatives are not nearly as supported or simple to implement.
When it comes to web development you will always encounter things that sound too good to be true. Sometimes they are good, and all that stops us from using them is our notion of being conspicuous about *everything* as developers. In a lot of cases, however, they really are not as good as they seem but we only find out after using them for a while that we are actually “doing it wrong”.
One such case is local storage. There is a storage specification (falsely attributed to HTML5 in a lot of examples) with an incredibly simple API that was heralded as the cookie killer when it came out. All you have to do to store content on the user’s machine is to access the navigator.localStorage (or sessionStorage if you don’t need the data to be stored longer than the current browser session):
LocalStorage also has a lot of drawbacks that aren’t quite documented and certainly not covered as much in “HTML5 tutorials”. Especially performance oriented developers are very much against its use.
When we covered localStorage a few weeks ago using it to store images and files in localStorage it kicked off a massive thread of comments and an even longer internal mailing list thread about the evils of localStorage. The main issues are:
localStorage is synchronous in nature, meaning when it loads it can block the main document from rendering
localStorage does file I/O meaning it writes to your hard drive, which can take long depending on what your system does (indexing, virus scanning…)
On a developer machine these issues can look deceptively minor as the operating system cached these requests – for an end user on the web they could mean a few seconds of waiting during which the web site stalls
In order to appear snappy, web browsers load the data into memory on the first request – which could mean a lot of memory use if lots of tabs do it
localStorage is persistent. If you don’t use a service or never visit a web site again, the data is still loaded when you start the browser
This is covered in detail in a follow-up blog post by Taras Glek of the Mozilla performance team and also by Andrea Giammarchi of Nokia.
In essence this means that a lot of articles saying you can use localStorage for better performance are just wrong.
Alternatives
Of course, browsers always offered ways to store local data, some you probably never heard of as shown by evercookie (I think my fave when it comes to the “evil genius with no real-world use” factor is the force-cached PNG image to be read out in canvas). In the internal discussions there was a massive thrust towards advocating IndexedDB for your solutions instead of localStorage. We then published an article how to store images and files in IndexedDB and found a few issues – most actually related to ease-of-use and user interaction:
IndexedDB is a full-fledged DB that requires all the steps a SQL DB needs to read and write data – there is no simple key/value layer like localStorage available
IndexedDB asks the user for permission to store data which can spook them
The browser support is not at all the same as localStorage, right now IndexedDB is supported in IE10, Firefox and Chrome and there are differences in their implementations
Safari, Opera, iOS, Opera Mobile, Android Browser favour WebSQL instead (which is yet another standard that has been officially deprecated by the W3C)
As always when there are differences in implementation someone will come up with an abstraction layer to work around that. Parashuram Narasimhan does a great job with that – even providing a jQuery plugin. It feels wrong though that we as implementers have to use these. It is the HTML5 video debate of WebM vs. H264 all over again.
Now what?
There is no doubt that the real database solutions and their asynchronous nature are the better option in terms of performance. They are also more matured and don’t have the “shortcut hack” feeling of localStorage. On the other hand they are hard to use in comparison, we already have a lot of solutions out there using localStorage and asking the user to give us access to storing local files is unacceptable for some implementations in terms of UX.
The answer is that there is no simple solution for storing data on the end users’ machines and we should stop advocating localStorage as a performance boost. What we have to find is a solution that makes everybody happy and doesn’t break the current implementations. This might prove hard to work around. Here are some ideas:
Build a polyfill library that overrides the localStorage API and stores the content in IndexedDB/WebSQL instead? This is dirty and doesn’t work around the issue of the user being asked for permission
Implement localStorage in an asynchronous fashion in browsers – actively disregarding the spec? (this could set a dangerous precedent though)
Change the localStorage spec to store asynchronously instead of synchronously? We could also extend it to have a proper getStorageSpace interface and allow for native JSON support
Define a new standard that allows browser vendors to map the new API to the existing supported API that matches the best for the use case?
We need to fix this as it doesn’t make sense to store things locally and sacrifice performance at the same time. This is a great example of how new web standards give us much more power but also make us face issues we didn’t have to deal with before. With more access to the OS, we also have to tread more carefully.
The other day we wrote about how to Save images and files in localStorage, and it was about being pragmatic with what we have available today. There are, however, a number of performance implications with localStorage – something that we will cover on this blog later – and the desired future approach is utilizing IndexedDB. Here I’ll walk you through how to store images and files in IndexedDB and then present them through an ObjectURL.
Update 2011-12-20: The video recording of this webinar is now available:
IndexedDB is the emerging standard for structured client-side data storage. The IndexedDB standard is supported by current versions of Firefox and Chrome, and support for it is expected in Internet Explorer 10.
With this growing maturity and support, it’s time to start experimenting with what IndexedDB can do for Web applications. Working with IndexedDB requires a shift in mindset for many Web developers, as it is more similar to “NoSQL” systems like CouchDB or MongoDB than to traditional relational databases.
IndexedDB will be the theme for December’s MDN DevDerby. And to kick off the theme for the month, on December 1st, we’re offering an MDN Webinar on IndexedDB with Jonas Sicking, one of the editors of the IndexedDB draft standard and one of the implementors of IndexedDB in Gecko.
When
December 1st, at 9:00 a.m. US Pacific Time (17:00 UTC). Add this event to your Google calendar:
Where
Air Mozilla, with text chat on #airmozilla on irc.mozilla.org. (There’s an IRC widget on the Air Mozilla page if you need it.)
We’ll record this session for those who can’t attend (or can’t use Flash, which is currently used by Air Mozilla for live streaming video).
We’d like to get a rough estimate of how many people will be attending. If you happen to use Plancast, and you plan to attend the seminar, please join the event on Plancast.
Aurora 10 also implements incremental enhancements like IndexedDB setVersion API changes. Ongoing detailed attention to evolving specifications help to keep Firefox at the front of the Web revolution. (Read more about IndexedDB on MDN.)
DOM
We now fire a “load” event on stylesheet linking when the sheet load finishes or “error” if the load fails.
We turn the POSTDATA prompt into an information page (when navigating in session history).
We only forward event attributes on body/frameset to the window if we also forward the corresponding on* property.
We no longer allow more than one call to window.open() when we allow popups.
We fixed a bug where a success callback never fired when a position update is triggered after getCurrentPosition().
We removed replaceWholeText().
We fixed an error with createPattern(zero-size canvas).
We now handle putImageData(nonfinite) correctly.
We now throw INVALID_STATE_ERR when dispatching uninitialized events.
We’ve made Document.documentURI readonly.
We fixed document.importNode to comply with optional argument omitted.
Web workers
We now allow data URLs.
We implemented event.stopImmediatePropagation in workers.
We made XHR2 response/responseType work in Web Workers.
Graphics
We implement the WebGL OES_standard_derivatives extension.
We implement minimal-capabilities WebGL mode.
JavaScript
The function caller property no longer skips over eval frames.
We fixed E4X syntax so that it is not accepted in ES5 strict mode.
weakmap.set no longer returns itself instead of undefined.
We implemented the battery API.
Offline: IndexedDB enhancements
IndexedDB setVersion API changes
Added support for IDBObjectStore/IDBIndex.count
Various methods accept both keys and KeyRanges.
Added support for IDBCursor.advance.
Implemented deleteDatabase.
objectStoreNames are no longer updated on closed databases when another connection adds or removes object stores
IDBObjectStore.delete and IDBCursor.delete now return undefined.
No longer throws an error if there are unknown properties in the options objects to createObjectStore/createIndex.
We now the errorCode to “ABORT_ERR” for all pending requests when IDBTransaction.abort() is called.
Fixed the sort order for indexes.
Layout
We have updated the current rule for handling malformed media queries.
We now support the HTML5 <bdi> element and CSS property unicode-bidi: isolate.
The CSS3 implementation now supports unicode-bidi: plaintext.
Media
Implemented Document.mozFullScreenEnabled.
Enabled the DOM full-screen API on desktop Firefox by default.
This is a guest post from Ben Turner, one of the developers of IndexedDB for Firefox.
Mozilla is pleased to announce support for IndexedDB in the upcoming Firefox 4 Beta 9 and recent trunk nightlies. IndexedDB allows web apps to store large amounts of data on your local system (with your explicit permission, of course) for fast offline retrieval at a later time. We’re hoping that webmail, TV listings, and online purchase history will one day be as convenient to access offline as they are online.
Web developers who are interested in using this new feature are encouraged to read the IndexedDB primer on Mozilla Developer Network. The primer attempts to strip away some of the complexity in the specification, to discuss the most important concepts and provide simple examples. The truly hardcore are of course welcome to read the entire specification.
Keep in mind that IndexedDB is brand new and is not uniformly supported by all browsers, and please file bugs for any behavior that seems to contradict the specification!
HTML5 needs spokespeople to work. There are a lot of people out there who took on this role, and here at Mozilla we thought it is a good idea to introduce some of them to you with a series of interviews and short videos. The format is simple – we send the experts 10 questions to answer and then do a quick video interview to let them introduce themselves and ask for more detail on some of their answers.
Remy is one of those ubiquitous people of HTML5. Whenever something needed fixing, there is probably something on GitHub that Remy wrote that helps you. He is also very English and doesn’t mince his words much.
1) Reading “Introducing HTML5″ it seems to me that you were more of the API – focused person and Bruce the markup guy. Is that a fair assumption? What is your background and passion?
That’s spot on. Bruce asked me to join the project as the “JavaScript guy” – which is the slogan I wear under my clothes and frequently reveal in a superman ‘spinning around’ fashion (often at the surprise of clients).
My background has always been coding – even from a young age, my dad had me copying out listings from old spectrum magazines only to result in hours of typing and some random error that I could never debug.
As I got older I graduated to coding in C but those were the days the SDKs were 10Mb downloaded over a 14kb modem, and compile in to some really odd environment. Suffice to say I didn’t get very far.
Then along came JavaScript. A programming language that didn’t require any special development environment. I could write the code in Notepad on my dodgy Window 95 box, and every machine came with the runtime: the browser. Score!
From that point on the idea of instant gratification from the browser meant that I was converted – JavaScript was the way for me.
Since then I’ve worked on backend environments too (yep, I’m a Perl guy, sorry!), but always worked and played in the front end in some way or another. However, since started on my own in 2006, it’s allowed me to move focus almost entirely on the front end, and specialise in JavaScript. Basically, work-wise: I’m a pig in shit [Ed: for our non-native English readers, he means "happy")].
2) From a programmer’s point of view, what are the most exciting bits about the HTML5 standard? What would you say is something every aspiring developer should get their head around first?
For me, the most exciting aspects of HTML5 is the depth of the JavaScript APIs. It’s pretty tricky to explain to Joe Bloggs that actually this newly spec’ed version of HTML isn’t mostly HTML; it’s mostly JavaScript.
I couldn’t put my finger on one single part of the spec, only because it’s like saying which is your favourite part of CSS (the :target selector – okay, so I can, but that’s not the point!). What’s most exciting to me is that HTML5 is saying that the browser is the platform that we can deliver real applications – take this technology seriously.
If an aspiring developer wanted something quick and impressive, I’d say play around with the video API – by no means is this the best API, just an easy one.
If they really wanted to blow people away with something amazing using HTML5, I’d say learn JavaScript (I’m assuming they’re already happy with HTML and CSS). Get a book like JavaScript: The Good Parts and then get JavaScript Patterns and master the language. Maybe, just maybe, then go buy Introducing HTML5, it’s written by two /really/ good looking (naked) guys: http://flic.kr/p/8iyQTE and http://flic.kr/p/8iy6Z1[Ed: maybe NSFW, definitely disturbing].
3) In your book you wrote a nice step-by-step video player for HTML5 video. What do you think works well with the Video APIs and what are still problems that need solving?
The media API is dirt simple, so it means working with video and audio is a doddle. For me, most of it works really well (so long as you understand the loading process and the events).
Otherwise what’s really quite neat, is the fact I can capture the video frames and mess with them in a canvas element – there’s lots of fun that can be had there (see some of Paul Rouget’s demos for that!).
What sucks, and sucks hard, is the spec asks vendors, ie. browser makers, *not* to implement full screen mode. It uses security concerns as the reason (which I can understand), but Flash solved this long ago – so why not follow their lead on this particular problem? If native video won’t go full screen, it will never be a competitive alternative to Flash for video.
That all said, I do like that the folks behind WebKit went and ignored the spec, and implemented full screen. The specs are just guidelines, and personally, I think browsers should be adding this feature.
4) Let’s talk a bit about non-HTML5 standards, like Geolocation. I understand you did some work with that and found that some parts of the spec work well whilst others less so. Can you give us some insight?
On top of HTML5 specification there’s a bunch more specs that make the browser really, really exciting. If we focus on the browser being released today (IE9 included) there’s a massive amount that can be done that we couldn’t do 10 years ago.
There’s the “non-HTML5″ specs that actually were part of HTML5, but split out for good reason (so they can be better managed), like web storage, 2D canvas API and Web Sockets, but there’s also the /really/ “nothing-to-do-with-HTML5″ APIs (NTDWH5API!) like querySelector, XHR2 and the Device APIs. I’m super keen to try all of these out even if they’re not fully there in all the browsers.
Geolocation is a great example of cherry picking technology. Playing against the idea that the technology isn’t fully implemented. Something I find myself ranting on and on about when it comes to the question of whether a developer should use HTML5. Only 50% of Geolocation is implemented in the browsers supporting it, in that they don’t have altitude, heading or speed – all of which are part of the spec. Does that stop mainstream apps like Google Maps from using the API? (clue: no).
The guys writting the specs have done a pretty amazing job, and in particular there are few cases where the specs have been retrospectively written. XHR is one of these and now we’ve got a stable API being added in new browsers (i.e. IE6 sucks, yes, we all know that). Which leads us to drag and drop. The black sheep of the APIs. In theory a really powerful API that could make our applications rip, but the technical implementation is a shambles. PPK (Peter-Paul Koch) tore the spec a bit of a ‘new one’. It’s usable, but it’s confusing, and lacking.
Generally, I’ve found the “non-HTML5″ specs to be a bit of mixed bag. Some are well supported in new browsers, some not at all. SVG is an oldie and now really viable with the help of JavaScript libraries such as Raphaël.js or SVGWeb (a Flash based solution). All in all, there’s lots of options available in JavaScript API nowadays compared to back in the dark ages.
5) Let’s talk Canvas vs. SVG for a bit. Isn’t Canvas just having another pixel-based rectangle in the page much like Java Applets used to be? SVG, on the other hand is Vector based and thus would be a more natural tool to do something with open standards that we do in Flash now. When would you pick SVG instead of Canvas and vice versa?
Canvas, in a lot of ways is just like the Flash drawing APIs. It’s not accessible and a total black box. The thing is, in the West, there’s a lot of businesses, rightly or wrongly, that want their fancy animation thingy to work on iOS. Since Flash doesn’t work there, canvas is a superb solution.
However, you must, MUST, decide which technology to use. Don’t just use canvas because you saw a Mario Kart demo using it. Look at the pros and cons of each. SVG and the canvas API are not competitive technologies, they’re specially shaped hammers for specific jobs.
Brad Neuberg did a superb job of summarising the pros and cons of each, and I’m constantly referring people to it (here’s the video).
So it really boils down to:
Canvas: pixel manipulation, non-interactive and high animation
SVG: interactive, vector based
Choose wisely young padawan!
6) What about performance? Aren’t large Canvas solutions very slow, especially on mobile devices? Isn’t that a problem for gaming? What can be done to work around that?
Well…yes and no. I’m finishing a project that has a large canvas animation going on, and it’s not slow on mobile devices (not that it was designed for those). The reason it’s not slow is because of the way the canvas animates. It doesn’t need to be constantly updating at 60fps.
Performance depends on your application. Evaluate the environment, the technologies and make a good decision. I personally don’t think using a canvas for a really high performance game on a mobile is quite ready. I don’t think the devices have the ommph to get the job done – but there’s a hidden problem – the browser in the device isn’t quite up to it. Hardware acceleration is going to help, a lot, but today, right now, I don’t think we’ll see games like Angry Birds written in JavaScript.
That said… I’ve seriously considered how you could replicate a game like Canabalt using a mix of canvas, DIVs and CSS. I think it might be doable ::throws gauntlet::
I think our community could actually learn a lot from the Flash community. They’ve been through all of this already. Trying to make old versions of Flash from years back do things that were pushing the boundaries. People like Seb Lee-Delisle (@seb_ly / http://seb.ly) are doing an amazing job of teaching both the Flash and JavaScript community.
7) A feature that used to be HTML5 and is now an own spec is LocalStorage and its derivatives Session Storage or the full-fledged WebSQL and IndexedDB. Another thing is offline storage. There seems to be a huge discussion in developer circles about what to use when and if NoSQL solutions client side are the future or not. What are your thoughts? When can you use what and what are the drawbacks?
Personally I love seeing server-less applications. Looking at the storage solutions I often find it difficult to see why you wouldn’t use WebStorage every time.
In a way it acts like (in my limited experience of) NoSQL, in that you lookup a key and get a result.
Equally, I think SQL in the browser is over the top. Like you’re trying to use the storage method *you* understand and forcing it into the browser. Seems like too much work for too little win.
Offline Apps, API-wise, ie. the application cache is /really/ sexy. Like sexy with chocolate on top sexy. The idea that our applications can run without the web, or switch when it detects it’s offline is really powerful. The only problem is that the events are screwed. The event to say your app is now offline requires the user to intervene via the browser menu, telling the browser to “work in offline mode”. A total failure of experience. What’s worse is that, as far as I know, there’s no plan to make offline event fire properly :-(
That all said, cookies are definitely dead for me. I’ve yet to find a real solution for cookies since I found the Web Storage API – and there’s a good decent number of polyfills for Web Storage – so there’s really no fear of using the API.
8) Changing the track a bit, you’ve built the HTML5shiv to make HTML5elements styleable in IE. This idea sparked quite a lot of other solutions to make IE6 work with the new technologies (or actually simulate them). Where does this end? Do you think it is worth while to write much more code just to have full IE6 support?
There’s two things here:
Supporting IE6 (clue: don’t)
Polyfills
IE6, seriously, and for the umpteenth time, look at your users. Seriously. I know the project manager is going to say they don’t know what the figures are, in that case: find out! Then, once you’ve got the usage stats in hand, you know your audience and you know what technology they support.
If they’re mostly IE6 users, then adding native video with spinning and dancing canvas effect isn’t going to work – not even with shims and polyfills. IE6 is an old dog that just isn’t up to doing the mileage he used to be able to do back in his prime. But enough on this subject – the old ‘do I, or don’t I developer for IE6′ is long in the tooth.
Polyfills – that’s a different matter. They’re not there to support IE6, they’re there to bring browsers up to your expectations as a developer. However, I’d ask that you carefully consider them before pulling them in. The point of these scripts is they plug missing APIs in those older browsers. “Old browsers” doesn’t particularly mean IE6. For example, the Web Sockets API has a polyfill by way of Flash. If native Web Sockets aren’t there, Flash fills the gap, but the API is exposed in exactly the same manner, meaning that you don’t have to fork your code.
I don’t think people should be pulling in scripts just for the hell of it. You should consider what you’re trying to achieve and decide whether X technology is the right fit. If it is, and you know (or expect) your users have browsers that don’t support X technology – should you plug it with JavaScript or perhaps should you consider a different technology?
This exact same argument rings true for when someone adds jQuery just to add or remove a class from an element. It’s simply not worth it – but clearly that particular developer didn’t really understand what they needed to do. So is education the solution? I should hope so.
9) Where would you send people if they want to learn about HTML5? What are tutorials that taught you a lot? Where should interested people hang out?
I tend to also direct people to my http://html5demos.com simply to encourage viewing source, and hacking away.
Regarding what tutorials taught me – if I’m totally honest, the place I’ve learnt the most from is actually HTML5Doctor.com. There’s some pretty good JavaScript / API tutorials coming from the chaps at http://bocoup.com. Otherwise, I actually spend a lot of time just snooping through the specifications, looking for bits that I’ve not seen before and generally poking them with a stick.
10) You have announced that you are concentrating on building a framework to make Websockets easy to work with. How is that getting on and what do you see Websockets being used for in the future? In other words, why the fascination?
Concentrating is a strong word ;-) but it is true, I’ve started working on a product that abstracts Web Sockets to a service. Not the API alone, since it’s so damn simple, but the server setup: creating sessions, user control flow, waiting for users and more.
The service is called Förbind. Swedish for “connect”, ie. to connect your users. It’s still early days, but I hope to release alpha access to forbind.net this month.
I used to work in finance web sites and real-time was the golden egg: to get that data as soon as it was published. So now that it’s available in a native form in the browser, I’m all over it!
What’s more, I love the idea of anonymous users. I created a bunch of demos where the user can contribute to something without ever really revealing themselves, and when the users come, you start to see how creative people are without really trying. Sure, you get a lot of cocks being drawn, but you also see some impressive ideas – my business 404 page for example allows people to leave a drawing, one of the most impressive is a Super Mario in all his glory. Anonymous users really interest me because as grey as things can seem sometimes, a stranger can easily inspire you.
Do you know anyone I should interview for “People of HTML5″? Tell me on Twitter: @codepo8
