Beyond HTML5: Database APIs and the Road to IndexedDB

IndexedDB is an evolving web standard for the storage of significant amounts of structured data in the browser and for high performance searches on this data using indexes. Mozilla has submitted substantial technical feedback on the specification, and we plan to implement it in Firefox 4. We spoke to prominent web developers about evolving an elegant structured storage API for the web. While versions of Safari, Chrome, and Opera support a technology called Web SQL Database, which uses SQL statements as string arguments passed to a JavaScript API, we think developer aesthetics are an important consideration, and that this is a particularly inelegant solution for client-side web applications. We brought developer feedback to the editor of the IndexedDB specification, and also spoke with Microsoft, who agree with us that IndexedDB is a good option for the web. With additional implementations from the Chrome team in the offing, we think it is worth explaining our design choices, and why we think IndexedDB is a better solution for the web than Web SQL Database.

Web applications can already take advantage of localStorage and sessionStorage in IE 8+, Safari 4+, Chrome 4+, Opera 10.5+ and Firefox 2+ to store key-value pairs with a simple JavaScript API. The Web Storage standard (encompassing localStorage and sessionStorage), now widely implemented, is useful for storing smaller amounts of data, but less useful for storing larger amounts of structured data. While many server-side databases use SQL to programmatically store structured data and to meaningfully query it, on the client-side, the use of SQL in a JavaScript API has been contentious.

SQL? Which SQL?

Many web developers certainly are familiar with SQL, since many developers touch just as much server-side code (e.g. PHP and database operations) as client-side code (e.g. JavaScript, CSS, and markup). However, despite the ubiquity that SQL enjoys, there isn’t a single normative SQL standard that defines the technology. In particular, SQLite supports most of SQL-92, with some notable omissions, and is what the WebDatabase API is based on. But SQLite itself isn’t a specification — it’s a release-ready technology! And the best definition of what constitutes the supported subset of SQL that SQLite uses is the SQLite manual. In order to really get Web SQL Database right, we’d have to first start with defining a meaningful subset of SQL for web applications. Why define a whole other language, when more elegant solutions exist within JavaScript itself?

The Benefits and Pitfalls of SQLite

We think SQLite is an extremely useful technology for applications, and make it available for Firefox extensions and trusted code. We don’t think it is the right basis for an API exposed to general web content, not least of all because there isn’t a credible, widely accepted standard that subsets SQL in a useful way. Additionally, we don’t want changes to SQLite to affect the web later, and don’t think harnessing major browser releases (and a web standard) to SQLite is prudent. IndexedDB does not have this problem; even though our underlying implementation of IndexedDB may be based on SQLite, we keep developers insulated from changes to SQLite by exposing an API that isn’t based on SQLite’s supported syntax.

Aesthetics and Web Developers

Last year, we held a summit at the Mozilla campus to discuss storage on the web. We invited web developers to speak to us about a desirable structured storage API on the web. Many did express resigned acceptance of a SQLite-based API, since they had already experimented with releases of Web SQL Database in some browsers, and claimed that something in circulation was better than a collection of ideas. Yet, all voiced enthusiasm for better design choices, and how a simpler model would make life easier for them. We watched as developers whiteboarded a simple BTree API that addressed their application storage needs, and this galvanized us to consider other options. We were resolved that using strings representing SQL commands lacked the elegance of a “web native” JavaScript API, and started looking at alternatives. Along with Microsoft, we sent feedback about the IndexedDB proposal and actively became involved in the standardization effort.

In another article, we compare IndexedDB with Web SQL Database, and note that the former provides much syntactic simplicity over the latter. IndexedDB leaves room for a third-party JavaScript library to straddle the underlying primitives with a BTree API, and we look forward to seeing initiatives like BrowserCouch built on top of IndexedDB. Intrepid web developers can even build a SQL API on top of IndexedDB. We’d particularly welcome an implementation of the Web SQL Database API on top of IndexedDB, since we think that this is technically feasible. Starting with a SQL-based API for use with browser primitives wasn’t the right first step, but certainly there’s room for SQL-based APIs on top of IndexedDB.

We want to continue the discussion with web developers about storage on the web, since that helps us structure our thoughts about product features and future web standards. We look forward to seeing the next generation of web applications with support for high performance searches on indexed data, and to seeing web applications work even more robustly in “airplane mode.”


About Arun Ranganathan

More articles by Arun Ranganathan…


  1. Mats

    Hi – a swede is calling.

    I have been into webdev for some years and recently stumbled over Web SQL Database. And it make it possible for me to write apps that I have been dreaming of for mobile devices. As a fan of FireFox I was really dissapointed that there was no support for this in my Firefox 4.

    So, now I am FORCED to use Chrome in my developing.
    Please, re-think and start support Web SQL as soon as possible.

    Cheers from Mats

    May 14th, 2011 at 13:39

  2. Christian R Simms

    I don’t have much to contribute, just that I’ve used sqlite for several years in a server-side app, and it’s really solid. Solid enough that if all the browser vendors adopted its use, it would be a good thing. Dr. Hipp did a great job developing it. And no client-side layering on top of IndexedDB will be as fast, plus will take at least a year to get solid.

    May 17th, 2011 at 12:35

  3. david karapetyan

    None of what you said is holding up in the current implementation. The stuff that is documented in MDC is anything but elegant compared to SQL. All I see is a more imperative and clunky implementation of a small subset of SQL with none of the elegance of the declarative SQL syntax. Plus, who was the genius that thought of separating requests and transactions?

    June 25th, 2011 at 00:54

  4. Guido Leenders

    I am happy to see that parties are working together to find a standard solution across all browser for midsized storage. That finally allows html apps to work when offline. Not all regions in the world currently have 100% coverage.

    I can imagine that the current status of the SQL standard is insufficient and semantics should further be cleared. But as far as I can recall from my university days, SQL is based upon solid semantics from set theory.

    The alternative (Indexed DB API) is very low level and I strongly encourage our own developers to work on specifications that map directly into code instead of many low level operations and with of course too many forgotten exception handlers. Sure developers can be trained, but with training needs comes a lower bang for the development buck.

    I would hoera it when you Mozilla would reconsider to add Web SQL to Firefox and parties would continue working on the Web SQL standard.

    July 10th, 2011 at 04:52

    1. Bob H

      I have to agree, this seems to be an attempt to make data storage in web applications as difficult as possible. I just read the specification, I am surprised someone didn’t just go a step further and ask web developers to code data functions within ADA+ASM.

      Web SQL seems like a far better solution and the arguments presented here by Mozilla really don’t stand up when you think about this from a web development perspective. Perhaps from the perspective of someone who uses C++ every day IndexedDB looks good, but most web developers aren’t doing that and this is overly complex.

      I agree with the previous posters, step away from this effort.

      August 16th, 2011 at 19:32

  5. […] 最后我们要介绍的就是IndexedDB了,相比其它两个规范,目前只有Firefox实现了IndexedDB(顺便提一下,Mozilla表示它们永远不会去实现Web SQL Database),不过Google已经表示正在考虑在Chrome中加入IndexDB支持。 […]

    August 20th, 2011 at 22:18

  6. […] 最后我们要介绍的就是IndexedDB了,相比其他两个规范,目前只有Firefox实现了IndexedDB(顺便提一下,Mozilla表示它们永远不会去实现Web SQL Database),不过Google已经表示正在考虑在Chrome中加入IndexDB支持。 […]

    August 24th, 2011 at 02:25

  7. klkl

    Good decisoin!

    I appreciate that you stand for IndexedDB, even though most developers unfamiliar with web standards process are unable to understand depth of the issue.

    August 26th, 2011 at 16:14

  8. Alek Paunov


    Complex applications (like evolving IDEs as instance) need a real DB (sqlite or other), not just key-value API without query language.

    It is obvious which solution is better:
    * it is easy for everyone to emulate IndexedDB using WebSQL (with exactly same performance)
    * it is impossible for anyone to emulate WebSQL on top of IndexedDB – (as “Aesthetics” argument absolutely irresponsible suggests)

    Current official Moziilla position on this decision (which is very, very influential for the shape of the Web in the next few years) is just a horrible mistake, which need to be corrected.

    September 5th, 2011 at 14:19

  9. Leigh Harrison

    This really is an unfortunate decision.

    Writing complex software for off-line use finally appeared possible with the Web SQL DB. Having carefully considered IndexedDB it doesn’t even start to be an viable alternative, both in terms of its feature set and its interface. Writing a second abstraction layer on top of IndexedDB to emulate the underlying SQLite interface is crazy, and the performance hit would be unacceptable.

    It will be sad not to include Firefox among the target browsers for future projects. I sincerely hope Mozilla will rethink this decision.

    September 20th, 2011 at 01:23

  10. Greg

    I would encourage folks to contact the Mozilla advisory board and/or individuals within Mozilla (even Brendan Eich) to complain about the decision to exclude WebSQL; from looking at these posts at least, they seem to be out-of-sync with the development community.

    September 21st, 2011 at 11:25

  11. ed

    thanks for not giving developers what they need. You could give us web sql while you think of and develop better solutions. Standards are nice but we have work to do.

    October 24th, 2011 at 04:13

  12. Gary Schaecher

    Sounds like a last ditch effort by two companies late to the party who want to spoil the hard work and productivity of all who making a goog living with WebSQL and SQLite! Booooooo

    October 30th, 2011 at 19:07

  13. Alon Raskin

    Definitely a horrible decision. We have apps written for Android, BB and iOS using WebSQL. We are now faced with the decision of support Windows Phone and it turns out that like Mozilla they too have gone down the IndexDB. What do we do now? Do we create another version of our app just to support Windows Phone?

    We are even toying with writing a JS layer which will take our WebSQL calls and convert them to IndexDB Calls! The effect of this is to shield our developers from IndexDB!! I dont see what other choice we have…

    November 8th, 2011 at 12:44

    1. Leigh Harrison

      The abstraction layer is a sensible solution.

      I’m adopting a slightly different abstraction approach, as my apps do background synchronisation to a MySQL DB on the server anyway. If the user is running a compatible browser we use WebSQL, if the user’s running Firefox or Internet Exploder we use a RESTful service to the MySQL DB on the server. I looked at an IndexDB abstraction, but for the complex databases I’m using it was slower than getting and setting via the web.

      To make it clear where the speed issue lies we display a message advising the user that they should switch to Chrome or Safari for superior performance.

      I don’t like this solution much, and I’m hoping Firefox will soon see sense and support both IndexDB and WebSQL. If they do, Microsoft will eventually get with the programme too, although on past performance that may not be until IE13 or thereabouts.

      November 8th, 2011 at 13:37

  14. Eric Kizaki

    This is why M$ does not support SQLite:

    They have SQL Server Mobile.

    November 25th, 2011 at 14:08

  15. Eric Kizaki

    Here is a more updated version of LINQ to SQL for MS.

    So saying that MS does not support Web db is a lame argument because MS is doing their own proprietary thing. There was no mention of IndexedDB. If I was developing a MS app I would use this stuff not IndexedDB.

    November 25th, 2011 at 14:23

  16. Marcelo Cantos

    You are talking about mobile apps. This discussion is about the web.

    November 26th, 2011 at 02:40

  17. Ramin

    Marcelo, this is the same thing.
    every mobile device has a browser, and devs can use HTML, CSS, Javascript and something like phonegap to develop software for not one mobile os but for all of them, just changing the CSS to match the native OS look. Or you can go and use something like JQueryMobile.
    So you can use web technology to develop standalone local software for all kinds of mobile OSs that works offline (so they have no access to a web server with a database). Still you want to use a database for your software, and of course it is nice to be able to use the same SQL queries that you would send to a sql serverto send them to your local sqlite DB.
    as a dev you do not want to learn a new thing like indexed db that you can use no where else you want to use WebSQL, to use your SQL knowledge, or you want to use LINQ what you use everyday when you develop with C#.

    November 26th, 2011 at 12:08

    1. Marcelo Cantos

      @Ramin: If you and/or Eric are trying to make a case that Microsoft is being hypocritical, I won’t dispute that. Microsoft is a very large company, and to some extent, all large companies struggle to publicly convey a coherent view of the world. You wouldn’t try to argue that Microsoft doesn’t think tablets are the way to go just because the Office org has always had it in for the technology and refused to make Office usable on tablet devices.

      November 26th, 2011 at 20:04

      1. Ramin

        No, every company can and should do what is best for it. And actually I am very impressed by what MS offers with LINQ. I am just saying that accessing DB on mobiles is really important, and that I think that no dev wants to learn about indexed db.
        WebSQL is a nice thing and should be supported by mozilla and by MS.
        Like most other devs I really cannot understand the argument against WebSQL.

        November 27th, 2011 at 10:13

  18. Marcelo Cantos

    I guess we’re in violent agreement then. :-)

    November 27th, 2011 at 15:27

  19. Waseem

    Disappointing to say the least. It would be nice to use a real relational DB engine on the client side, or at least *have the option*. Mozilla completely screwed the pooch on this one, and I am disgusted by this decision. WebSQL was implemented almost completely across the board, and Mozilla’s petulance has set the web 1-3 years back in terms of offline and local development. It is for reasons like this (see also Firefox’s html5 audio/video supported formats) that I no longer recommend Firefox to other users, but Chrome.

    March 22nd, 2012 at 07:17

  20. […] 最后我们要介绍的就是IndexedDB了,相比其他两个规范,目前只有Firefox实现了IndexedDB(顺便提一下,Mozilla表示它们永远不会去实现Web SQL Database),不过Google已经表示正在考虑在Chrome中加入IndexDB支持。 […]

    April 1st, 2012 at 05:10

  21. […] […]

    May 20th, 2012 at 07:16

  22. Freddy May

    I agree with the vast majority of the commenters here. Sure, Mozilla are right to add support for IndexedDB – having NoSQL support is a great thing. But to put the frighteners on the development community by deprecating a hugely practical RDBMS that works beautifully in desktop and especially mobile applications is a bloody disgrace.

    July 16th, 2012 at 04:16

  23. Eric

    “In another article, we compare IndexedDB with Web SQL Database, and note that the former provides much syntactic simplicity over the latter”

    Uh? In what world? The article demonstrates the very opposite of that statement!

    August 14th, 2012 at 05:08

  24. John Willis

    Yet another asinine screwup by Mozilla… I really hope that Chrome completely destroys FF’s market share so that developers like me won’t have to pander to Firefox, which is to web browsers in this decade what IE was to web browsers in the previous one.

    September 21st, 2012 at 12:32

  25. Rene Schickbauer

    I’m the developer of the Maplat Webserver (and Framework). I’m currently in the process of designing Client-Side storing/caching of data for a large project.

    After evaluating the documentation, IndexedDB has no chance whatsoever of getting used in my projects. It does not fullfill requirement, it’s neither readable, fast nor remotely ACID compliant. IT#s API is just a junk of “when i grow up, someday i’ll be a database”.

    WebSQL on the other hand fits the bill nicely for most parts. This is what i’m probably going to use.

    So, the plan is as follows: Either completly drop support for browsers that don’t support WebSQL (Firefox is currently at less than 1% of “market share” on my business systems anyway) or don’t cache/store data on the client side for those browsers (which would make them much slower). Either way, i’m probably going to have to present affected users with a link to the latest Chrome build and suggest they “upgrade”.

    This isn’t an ideal solution. But in my opinion, IndexedDB is just too broken and convoluted for any serious use…

    October 1st, 2012 at 12:59

  26. Nick Vaidyanathan

    This is probably a correct and important architectural decision. The mature software engineering community has long acknowledged that it is better to Program to an Interface, Not An Implementation.

    Most of the web developers responding to this with disgust are likely to be “coders”, not engineers or programmers. The common “web developer”, sadly, can’t architect their way out of a paper bag and needs 4 Drupal/Wordpress modules to display JSON content on a page. Don’t worry about the harshness of their criticism, it’s simply the crying of children being told “No, don’t stick your fork in the electric socket.” (“But it looks really fun! >:(“)

    Passing around hard coded SQL queries as strings is wrong. The last thing we need on the web is a slew of client side SQL injection attacks because dumbo.IcopypastaJqueryPlugin().dev didn’t sanitize his inputs, potentially knocking out tables/harvesting user personally identifiable information.

    Additionally, the idea of the web being built around open standards as opposed to “whatever does the job right now” is the reason we’re all using different programs that implement HTTP 1.x and TCP/IP, and not MSIE (or, poignantly for Mozilla, Netscape Navigator). Philosophically, this is an excellent decision.

    But the commenters do raise a good point. Come on, guys. It’s 2012. Hibernate has been around for over a decade. GORM, LINQ to SQL..there are a lot of implementations of ORM/Abstract Data Mapper layers out there.

    If you want to roll your own NoSQL hash map based data store, you should be providing an entity based model with nice iterators and convenience magic methods like GORM, not some low level cursor transaction thing like we’re programming with ADO.NET 2.0.

    November 8th, 2012 at 17:23

    1. Waseem S

      Please, the argument that there are too many bad developers to allow a standard like WebSQL is not relevant at all. We already have AJAX, client side scripting and IndexedDB – if a developer is out to cause havoc, havoc will be caused. A SQL statement pointing to client side data is just as “dangerous” as a JavaScript command that modifies the local data store.

      It’s almost 2013; web apps have come a long way and matured. To treat web developers with kiddie gloves only hinders progress.

      December 3rd, 2012 at 21:14

    2. ArchE

      Nick, it’s so much architectural discussion that for trees you don’t see the forest.

      Something like this:
      scholar: “we grow a red sweet apples dear Master. Everyone likes it.”
      master: “dear scholar, that is bad. The apples must be blue”
      scholar: “but blue apples are toxic”
      master: “nevermind dear scholar, no one will eat them then. Finally we will have blue apples.”

      December 4th, 2012 at 13:36

  27. ArchE

    Generally I strongly disagree. By this decision you limit the freedom of choice.

    Philosophically BAD!

    1. SQL relational engine is philosophocally on the same level as NoSQL. While NoSQL uses JavaScript to access objects, RDBMS uses SQL.
    Difference? No…there’s no real ORM in NoSQL.
    3. Aesthetics – code unification? Okay, not really the argue, but I understand
    4. SQLLite changes? common, this is only supplemental argument, you’re always enable to hold stable builds

    You should take a care of developer, instead you say how developer would work. Thi is not the good way, as I said, you limit the freedom of choice.

    Moreover, just now, you kicked off FF from applications which are using WebSQL and are available for mobile devices also. Because, Chrome, Safari and IE supports WebSQL.

    November 13th, 2012 at 23:57

    1. Robert Nyman

      When it comes to Internet Explorer, it has never supported WebSQL. Current support is listed at caniuse.

      Additionally, the WebSQL specification is deprecated.

      November 14th, 2012 at 01:30

      1. ArchE

        Deprecated, right…according to W3C:
        The specification reached an impasse: all interested implementors have used the same SQL backend (Sqlite), but we need multiple independent implementations to proceed along a standardisation path.

        From this POV I really don not understand. Multiple implementations?
        Many projects already use WebSQL or persistence layers.
        Why not to support such an easy implementation like SQLLite?

        It seems to me there is strong dislike of SQLLite :)

        November 14th, 2012 at 02:56

  28. ArchE

    Other arguments why SQL Lite is not philosophically bad idea can be found here

    November 14th, 2012 at 01:07

  29. Richard Collins

    I got really excited about having a decent database available for use in an offline web app I am starting to develop. Then I discovered this crazy decision by Mozilla to not support WebSQL. I am now faced with the decision of increasing development time significantly by using IndexedDB, or creating a Chrome/Safari only app.

    Whilst I understand the reasoning, it seems quite narrow. Let’s back a good technology. Sqlite is public domain. In the unlikely event that incompatibilities are introduced in the future the community will fork it.

    December 3rd, 2012 at 12:59

    1. ArchE

      Thumbs up.
      It seems there’s strong disappointment with this descicion in the community.

      While I’m in the phase I already have WebSQL application my decision is to not support FF.

      I strongly believe guys from FF will change the decision. Hope it will not be too late.

      December 3rd, 2012 at 23:48

  30. Waseem S

    Firefox (Mozilla) with decisions like these come across as puritanical sticklers who can’t be bothered to listen to other people because they know what’s best for everyone. Aside from sabotaging client side data storage, Firefox is the only mainstream browser that can’t play files encoded in the ubiquitous MP3 format via the HTML5 tag. Any attempts to create a cross platform media solution has to include re-encoding all media to ogg or some other format, which doubles encoding time, media storage requirements and costs. Instead of making the web more accessible to all, they are putting hurdles, especially to smaller organizations.
    Their argument that MP3 is a proprietary format that shouldn’t be utilized is laughable when you see their GIF support. .

    December 3rd, 2012 at 21:09

  31. Sebastian Baltes

    Mozilla is so wrong about this.

    We are building a medium sized html5 app based on websql and phonegap, and it’s wonderfull to have sql in javascript! We have it in all mobile plattforms (in win8 via a sqlite plugin). Even in a medium sized app the relational model has a lot of advantages: a powerfull query language, transactions, speed and solidness. And we have some large sql queries – doing that by hand with IndexDB would be a nightmare.

    More than ten years ago, Object-Oriented DBs were the hype. What survived? SQL. Then NoSQL was the next thing… it’s nice, and they have their niches, but still the majority uses relational databases. Guess what will be dominant in 10 years?

    Your IndexDB may be good for the easiest use cases, but for this we allready have localStorage. The complex use cases are not covered by your immature design-baby.

    And why the heck must you guys at mozilla design your own peristent standard at all? Please explain this to me: Why didn’t you took MongoDB for example? Because it’s not a standard??? Smells like some kind of hybris, kind of ‘oh, we are sooo smart, and we have all this money and don’t know how to spend it’ attitude. You guess what? Follow this road, and in ten years firefox will be irrelevant, but we will still use SQL in the browser.

    January 17th, 2013 at 10:06

  32. Sergei

    I’d agree that structured indexed storage is more useful as a basis for Web platform. And it’s better to have Web SQL Storage atop of primitive structured storage. I’d agree with Mozilla’s arguments about depending on SQLite’s sysntax is a bad thing. But. IndexedDB’s API is terrible. Just look at example 4 in
    It’s simply not a way to go.

    February 5th, 2013 at 07:19

  33. Vikash Madhow

    Anyone who says that IndexedDB is better than WebSQL does not know what he/she is talking about.

    Except for the very simple use-cases, IndexedDB is NEVER going to enable the web-browser to become a platform where we have a useful offline mode for web apps when connectivity is down. I recommend that we move collectively to Chrome and recommend to all to do same. I, for one, need something like WebSQL for the type of apps I’m building and therefore firefox is off the supported-browsers list.

    March 8th, 2013 at 23:14

Comments are closed for this article.