Beyond HTML5: Database APIs and the Road to IndexedDB

IndexedDB is an evolving web standard for the storage of significant amounts of structured data in the browser and for high performance searches on this data using indexes. Mozilla has submitted substantial technical feedback on the specification, and we plan to implement it in Firefox 4. We spoke to prominent web developers about evolving an elegant structured storage API for the web. While versions of Safari, Chrome, and Opera support a technology called Web SQL Database, which uses SQL statements as string arguments passed to a JavaScript API, we think developer aesthetics are an important consideration, and that this is a particularly inelegant solution for client-side web applications. We brought developer feedback to the editor of the IndexedDB specification, and also spoke with Microsoft, who agree with us that IndexedDB is a good option for the web. With additional implementations from the Chrome team in the offing, we think it is worth explaining our design choices, and why we think IndexedDB is a better solution for the web than Web SQL Database.

Web applications can already take advantage of localStorage and sessionStorage in IE 8+, Safari 4+, Chrome 4+, Opera 10.5+ and Firefox 2+ to store key-value pairs with a simple JavaScript API. The Web Storage standard (encompassing localStorage and sessionStorage), now widely implemented, is useful for storing smaller amounts of data, but less useful for storing larger amounts of structured data. While many server-side databases use SQL to programmatically store structured data and to meaningfully query it, on the client-side, the use of SQL in a JavaScript API has been contentious.

SQL? Which SQL?

Many web developers certainly are familiar with SQL, since many developers touch just as much server-side code (e.g. PHP and database operations) as client-side code (e.g. JavaScript, CSS, and markup). However, despite the ubiquity that SQL enjoys, there isn’t a single normative SQL standard that defines the technology. In particular, SQLite supports most of SQL-92, with some notable omissions, and is what the WebDatabase API is based on. But SQLite itself isn’t a specification — it’s a release-ready technology! And the best definition of what constitutes the supported subset of SQL that SQLite uses is the SQLite manual. In order to really get Web SQL Database right, we’d have to first start with defining a meaningful subset of SQL for web applications. Why define a whole other language, when more elegant solutions exist within JavaScript itself?

The Benefits and Pitfalls of SQLite

We think SQLite is an extremely useful technology for applications, and make it available for Firefox extensions and trusted code. We don’t think it is the right basis for an API exposed to general web content, not least of all because there isn’t a credible, widely accepted standard that subsets SQL in a useful way. Additionally, we don’t want changes to SQLite to affect the web later, and don’t think harnessing major browser releases (and a web standard) to SQLite is prudent. IndexedDB does not have this problem; even though our underlying implementation of IndexedDB may be based on SQLite, we keep developers insulated from changes to SQLite by exposing an API that isn’t based on SQLite’s supported syntax.

Aesthetics and Web Developers

Last year, we held a summit at the Mozilla campus to discuss storage on the web. We invited web developers to speak to us about a desirable structured storage API on the web. Many did express resigned acceptance of a SQLite-based API, since they had already experimented with releases of Web SQL Database in some browsers, and claimed that something in circulation was better than a collection of ideas. Yet, all voiced enthusiasm for better design choices, and how a simpler model would make life easier for them. We watched as developers whiteboarded a simple BTree API that addressed their application storage needs, and this galvanized us to consider other options. We were resolved that using strings representing SQL commands lacked the elegance of a “web native” JavaScript API, and started looking at alternatives. Along with Microsoft, we sent feedback about the IndexedDB proposal and actively became involved in the standardization effort.

In another article, we compare IndexedDB with Web SQL Database, and note that the former provides much syntactic simplicity over the latter. IndexedDB leaves room for a third-party JavaScript library to straddle the underlying primitives with a BTree API, and we look forward to seeing initiatives like BrowserCouch built on top of IndexedDB. Intrepid web developers can even build a SQL API on top of IndexedDB. We’d particularly welcome an implementation of the Web SQL Database API on top of IndexedDB, since we think that this is technically feasible. Starting with a SQL-based API for use with browser primitives wasn’t the right first step, but certainly there’s room for SQL-based APIs on top of IndexedDB.

We want to continue the discussion with web developers about storage on the web, since that helps us structure our thoughts about product features and future web standards. We look forward to seeing the next generation of web applications with support for high performance searches on indexed data, and to seeing web applications work even more robustly in “airplane mode.”

Links

About Arun Ranganathan

More articles by Arun Ranganathan…


117 comments

  1. […] Older Article […]

    June 1st, 2010 at 11:44

  2. anon

    FYI: Your link http://hacks.mozilla.org/2010/05/comparing-indexeddb-and-webdatabase should be http://hacks.mozilla.org/2010/06/comparing-indexeddb-and-webdatabase/

    June 1st, 2010 at 12:34

    1. Shawn Wilsher

      Thanks, I’ve gone and fixed that.

      June 1st, 2010 at 13:25

  3. Matthew

    Some interesting comments at http://news.ycombinator.com/item?id=1395818

    June 1st, 2010 at 15:25

  4. Steve Antal

    I can’t seem to understand why does Mozilla shoot itself in the foot all the time.

    A framework that encapsulates SQL, sorry , this is just so unimpressive. Why not give me SQL and let me write a framework for myself? Am I(as in web developer) too incompetent?

    SQLite has a documentation, if it’s not good enough get together with Microsoft, Google and Opera and write one. Is it so hard?

    Having an implementation that everyone would agree to use is superior to writing yet another standard.

    I really hate it when organizations tend to know what’s better for developers. Sarcasm aside, I think the best thing to do if you really want this, is to have a JavaScript wrapper that wraps around this IndexedDB(you said that it’s easy to write one) and gives me some SQL that works everywhere.

    If you really think you have a superior solution just give it to us, but please give us our crappy solution too just in case it’s the other way around. Let us choose what is best for us!

    June 1st, 2010 at 16:05

    1. Arun Ranganathan

      Steve,

      Thanks for your comments. Right now, neither Microsoft nor Mozilla support the Web SQL Database API. In order to have really given you SQL, we’d have had to have *first* standardized a subset of SQL for the web. Simply saying “do what SQLite does” isn’t really a good standard. Also, that makes all existing implementations tied to SQLite, which we really wanted to avoid. All the existing implementations use SQLite. We didn’t want a scenario where version changes between releases of SQLite introduced instability; we’ve definitely seen this with our mozStorage API exposed to extensions, and we didn’t want to expose that to web content.

      We actually did talk to developers about this. IndexedDB allows for a wide possibility of use cases, but I completely agree that JavaScript libraries that either introduce a btree-style model *or even* an SQL model would be useful. In fact, we predict that libraries will do for structured storage exactly what they’ve done for the rest of the web — create syntactic simplifications and better abstractions.

      Are you currently using Web SQL Database in any of your projects, which is probably Safari, Chrome, and Opera only right now? Do you think the code isn’t portable to IndexedDB? I’d love to learn more if you can give me a pointer.

      Thanks again for your comments.

      — A*

      June 1st, 2010 at 17:44

      1. Marcelo Cantos

        I’m sure just about any SQL syntax will be portable to IndexedDB, just as it would also be portable to x86 assembly language. But that’s hardly the point. The last example in the comparison page is a perfect example of why no one with a shred of common sense is going to use IndexedDB.

        Please tell me that the solution to this ludicrously verbose and cryptic API (I’ve never seen a more inscrutable looping construct.) isn’t just to await an SQL wrapper that isn’t SQLite.

        August 30th, 2010 at 04:46

      2. Fazal Majid

        The argument that we can’t document and standardize on SQLite’s dialect of SQL is ludicrous. All modern OSes use a TCP/IP stack based on BSD 4.2, and there are subtle nuances between versions, e.g. TCP SACK. Should we ditch TCP/IP and use SNA just because there is not an independent implementation of the stack?

        September 1st, 2010 at 10:47

  5. Steve Antal

    Actually I did an application to test drive Google Gears, and it was awesome(the fact that I could write such a web app), with very little effort I could create offline data and sync it with a MySQL database. In a nutshell: it worked, It made me feel that this feature needs to be in ALL browsers as soon as possible.

    On the other hand IndexedDB API looks a lot like something I did server side. Yes, I actually did implement an IndexedDB-like API on top of MySQL with PHP, and here are the problems:

    – in the end it had no real advantage over using SQL
    – other developers did not understand it, and learning new stuff is not something some people desire
    – I ended up with writing more code that I would normally write with SQL
    – it made some simple operations hard and made complex operations nearly impossible

    My code that I did should be portable because it doesn’t use complex hierarchies. The problem however is that on the server side I use SQL, and on the client side I would use IndexedDB….which makes life harder than it already is. I know that there will be some differences anyway between server side and client side db queries but still it helps a lot that they are nearly the same.

    I really think that what we need is a minimal subset of SQLite, that should always work, a client side database should not have all the “cool” features of server side databases.

    Yes, I agree on that, like nearly everything else: storage will suck, but a JS based library will come and save the day. I am already seeing this happen with YUI Storage.

    June 1st, 2010 at 23:12

  6. Tim Lind

    I’m looking forward to IndexedDB, but it looks like the IndexedDB API introduces significant inflexibility, regarding the tying of index traversals to the row data.

    I’m happy that the indexing is dealt with by the implementation, because that is obviously troublesome to implement, but I always wonder, why do we only have the full effect exposed, when you’ve done the work to get the BTree itself into the browser, why not expose a BTree to us directly, so that we can use that for other purposes that perhaps don’t need a whole row of data and several indexes?

    It’s the same as with the localStorage API. We are only given one storage container, with a count and iteration, when you’ve done all the work, and to allow us to create a new container would be trivial and actually quite important. It’s not something we can implement over the API. So you’ve done the work, and then you still don’t provide us with the fundamental building blocks which allows us much more flexibility. It seems bizarre.

    June 2nd, 2010 at 00:45

  7. Tim Lind

    Actually there isn’t really any inflexibility, that was maybe from a dated draft I looked at, your current implementation looks good, and should make for good building blocks.

    But I still think that more focus generally needs to go toward providing the building blocks not just the end product, especially how the localStorage API has failed to do so.

    Well done for some great work on this spec.

    June 2nd, 2010 at 01:37

  8. fpiat

    Why make it simple when you can make complicated.
    After 15 years of web developement I’m really tired about all these new kind of stuff. We must knew HTML, CSS, JavaScript, PHP, SQL and now we have another piece of … for managing data. When I look at the spec I can’t imagine why people loose their time for making something so … insignifiant.
    Every web developer know SQL. Why another API?
    Why don’t face real problems like security?

    June 2nd, 2010 at 04:01

  9. Mark Davies

    I have to agree with some of the other guys here. I have developed a web app which uses Web SQL for Safari and Chrome browsers. I found this page by a google search to see if or when you would be implementing Web SQL. Web SQL works and is easy to use. There are some good wrapper libraries available. Can we not just standardize and make developers happy? I have no doubt I will be able to *port* my code. But I would prefer to have one set of code that works everywhere.

    June 4th, 2010 at 14:17

  10. Brona

    I can’t help myself, but I find all this reasoning selfserving
    1/ SQL? Which SQL?
    Yes, SQLite does not support full SQL92, but which database does and look all webdevelopers writing server code seems to be ok with that, having no problem with that… And sorry, creating and documenting new API which is much less known to developers instead of revriting existing SQLite documentation to more suitable one doesn’t seem to be benefit for anybody.
    2/ The Benefits and Pitfalls of SQLite
    Again, developers (not only web developers) deal with the same issue every day: new version is not compatible with the old one. SQLite is quite fast and powerful already. There’s no need to upgrade it in case of new version beeing incompatible
    3/ Aesthetics and Web Developers
    Seems to me like there is not developer in Mozilla who would write webpage with DB access on server side… We are mixing technologies everyday… HTML, CSS, ECMAS, PHP, SQL…. everyday rutine….
    The beauty of SQL is the fact, that it’s extremely powerfull, very easy to learn and tweekable (one can play with queries).
    It’s not nice of you, that you create somewhat object oriented interface, because that is library’s, programmer’s job to do it. So that the solution fit ones needs… And your solution? As far as I know, multiple keys are not supported, doing compound connections across 3 structures (easily done with simple join in Sql) would mean to spend a day playing with dozens of lines of code filled with callbacks, traversing with cursors, insane comparing and actually inventing SQL all over again

    But it’s you choice, Chrome, Safari and Opera have made their own… And webdevelopers will choose, and this will not help you…

    June 7th, 2010 at 04:52

  11. Brona

    My point is, don’t invent wheel, we already have one…
    I wonder what will happen in case WedDatabse will be in standard… Would you ignore it, because you know better?

    June 7th, 2010 at 04:56

    1. Shawn Wilsher

      It is very likely that it won’t become a standard as the editor’s draft of the spec currently states, so there isn’t much point in speculating here.

      June 7th, 2010 at 10:34

      1. Sam Orchard

        Isn’t what these comments are for? :P

        April 29th, 2011 at 14:08

  12. Fazal Majid

    Replacing an expressive syntax that separates logical and physical data models like SQL with a half-baked hierarchical model where the app has to handle everything itself is not one step backwards, but ten.

    June 7th, 2010 at 22:46

  13. Tim Lind

    I think most of the commenters are misunderstanding the crucial aspects of the IndexedDB spec, so Iet’s make a few things clear:

    – All SQL databases are built ONTOP of an IndexedDB-like API, which is mimmicked after the old ISAM API (that has long lied under many popular databases you use).

    – The IndexedDB spec is now bringing that *lower* layer into the browser, allowing *more* options to be available, instead of less.

    – SQL on the browser *will* be available, it will be created by a user loaded framework library, so that you can choose your SQL dialect and the function API to the SQL library.

    – Alternative data storage systems, that can be easier to use and more efficient, can *also* be provided on top of the IndexedDB API, so we are not *just* tied to SQL, that is the point of this spec. You will still have SQL functionality.

    June 8th, 2010 at 02:13

    1. Steve Antal

      Why implement a DEAD API?
      Why do a low level API?
      In that case why not just give fread() and fwrite() and let the developers write the whole database engine in JavaScript? Yeah I know it wouldn’t make sense, but it’s basically the same case with IndexedDB, it’s too low level!

      This whole thing reminds me of innerHTML, it was not in the standard, the standards group apparently thought you should just write a HTML parser in JavaScript and use the DOM methods which suck.
      Why did it make sense to not being able to use the browser’s HTML parser???? It’s a lot faster than anything you could ever write in JavaScript!

      Same is trough in this case, SQLite is a fast C engine, it has an SQL parser which is blazing fast, then why on earth should we duplicate that in JavaScript? Why should we have 2 SQL parsers in a browser(1 in SQLite and 1 in JavaScript).

      I can write an IndexedDB API on top of SQL in very few lines of code, then what is the point of crippling SQLite then?

      Show me the user loaded framework! I don’t want to wait eons for it to happen I want to use it right now!

      Sorry guys, but I really feel that this is a huge step backwards.

      June 9th, 2010 at 03:35

    2. ged

      I agree fully with TIM.

      1. SQL is well known but its a syntax on top of a indexer in the first place.
      So its far better to have an indexer at the heart of html5 to allow greater combination of SQL and NOSQL implementations on top.

      2. I feel allot of developers are biased towards SQL because its well known to them. I agree that its easy. But your all missing the point.
      The SQL you use on the serevr wil not be the same as what you use on the client !!!
      It is nice to be able to use the same data storage API on the server and the client. Everyone would agree with this OF COURSE.
      So this allow ALL developers access to the ability to be DRY in their architectural stacks.
      DRY = DO NOT REPAT YOURSELF.

      With the Index DB there will be wrappers that match MS SQL ; Oracle SQL, etc etc. Do you will get DRY.

      Conclusion.
      Its very important that the html5 Data storage API an engine is low level and wrappable.
      Jquery , Dojp and other wrappers have been the saviour of web development because they cover up the gaps between browsers versions.
      SQL wrappers onto of the IndexDB will do the same.

      To NOT put this level of flexibility into something as important as the web is architectural suicide…
      Mozilla is trying to avoid the HACK that is the current SQL API from W3C.

      October 21st, 2010 at 12:28

      1. blago

        Finally, I get it. Oracle, SUN, Microsoft, MySQL, Sybase, SQLite, etc. were all stupid to write all that stuff in C/C++/Assembly when they could’ve written some kickass jQuery plugins.

        Makes perfect sense: as far as I know an SQL engine has a query parser and a few for loops. We can have ALL that in 50 lines of JS. Even if it were 10,000 who cares about download times anymore.

        Moreover, I recently read somewhere that Nitro is 2 times faster than C++ in the worst case. The 1 order of magnitude penalty per interpretation layer is sooo 90s. And it’s not like data access is frequently used.

        I have a special request for all browser vendors. In the spirit of DRY (it means DO NOT REPAT YOURSELF), can we have row file access so that we can have the flexibility to write more highly optimized and custom indexes. When you think about it IndexDB could be implemented on top of files (I think).

        Conclusion: Go take a computer science class.

        October 22nd, 2010 at 17:18

  14. Brona

    2 Tim: it’s much easier to write object oriented interface on top of SQL than write SQL on top of object/callback oriented API. Second of all databases are already optimized for speed and performance, and that is one of the most difficult task to do when creating tools managing a lot of related data. SQL is much easier when dealing with relations that this nonsense and building really fast SQL interface upon IndexedDB would be reinventing the wheel (so the choice is either reinventing the wheel or learning something that seems less efficient with no apparent benefits). And the last paragraph is like a joke: alternative data storage that can be easier to use and more efficient can also be provided on the top of SQL engine, so we are not just tied to IndexDB API. This is the point of current WebDatabase Spec. You can still have IndexedDB funcionality…. seems to me more coherent. Every programmer has written object layer upon SQL with the benefits of easy but powerful query language. I wonder how many programmers will be able to create databases containing related objects that will be similarly efficient upon IndexedDB.
    But again, the choice will be upon webdevelopers assuming WebKit and Presto supporting WebDatabase.

    June 8th, 2010 at 04:00

  15. jonathan chetwynd

    this article does not mention XML, nor native xml databases in particular.

    could you comment on how xml data is to be incorporated in these proposals?

    June 9th, 2010 at 03:46

  16. InfiniteRand

    People in glass houses shouldn’t throw stones. How many browsers implement the complete DOM API? If a browser vendor argues that its the focus of its developers on applications integrated with the desktop environment requires a whole different DOM, well, perhaps web developers can re-implement the current DOM API on top of things

    June 10th, 2010 at 18:00

  17. Greg Quinn

    I have to agree with Steve Antal about Mozilla continually shooting itself in the foot. I’m currently writing website applications where I need to be able to efficiently store megabytes of data on the client side, and Mozilla, Chome and Safari have all come through like champs. SQLite works. It’s as simple as that; raising straw men about whether they will alter the API at some time in the future – so what? Using Microsoft’s acceptance of IndexDB spec is (to misquote Oscar Wilde), frankly the last bastion of the scoundrel. Honestly, it’s gotten to the stage where my impression is that you guys live in some kind of ivory tower. First Shaver talking on about H264, and now SQLite, which should have been implemented by Firefox a year back. I really don’t appreciate this client side API fragmentation. Carry on like this, and you guys will put yourself out of business.

    June 13th, 2010 at 22:17

  18. Brett Zamir

    Per jonathan chetwynd’s note, I just left a comment in favor of XML databases at http://hacks.mozilla.org/2010/06/comparing-indexeddb-and-webdatabase/comment-page-1/#comment-95595

    June 17th, 2010 at 20:10

    1. jonathan chetwynd

      havent yet had a chance to try out xquery support in browsers,

      but xslt support is excellent:
      http://www.peepo.com/games/Shusai-GoSeigen-19340119.xml
      runs from laptop so….

      a simple game player

      June 23rd, 2010 at 01:32

  19. Federico

    If you won’t provide web sql storage, you’ll make two great damages:

    The first:

    Firefox will loose a serious part of users: we’re developing a web application using web sql storage (it’s simply super) and we will soon recommend chrome to all of our users. I’m sure we’re not the unique company developing over this really good standard:other companies/developers are doing the same.

    Web sql storage solves in a great way the problem of offline usage of complex applications. Do you have a better standard? Ok, you can build it… *too*. The w3c specs about web sql storage are well defined: in the while, it shouldn’t be a bad idea to develop them.

    The second: you’re making developers life more difficult. The lacking of this feature in firefox is simply boring: we will have to force our users to another browser, making our apps more difficult to distribute and to be used.

    I don’t like google. Please, continue to give us the chance to rely oncemore on the mozilla foundation.

    June 20th, 2010 at 17:25

  20. Rastaman

    I agree with the previous devs, it’s really not fine to impose ourselves a new tech while SQL do perfectly fine the job. It will be painful to do complex queries ( such as calculates bills, taxes…) in an object oriented way while we can do them in one line of SQL. You loose me as a dev on this, and you really didn’t listen to all the devs which criticize this decision. If it easy to implement a native SQL interpreter in front of IndexDB (which run as fast as Web SQL in Webkit) when do it, support the Web SQL spec with your own implementation on top of indexed-DB and satisfy our needs: you have been supported by the community at one time isn’t it ? Or is it your sponsors (Oracle…) which don’t want to have SQL (not shipped by them) on the client side ?

    June 26th, 2010 at 02:59

  21. Nevio

    I need to say few things as well.

    As I saw in earlier posts, I understand that you all want good and standardized feature prior implementation but unfortunately, for examle:

    I am working on social media service full of HTML5 (WebSockets, WebSql, localStorage, etc…). I simply cannot nor I want to write bunch of server side or java code in order to get some kind of scalability. Wait… Server side and Scalability + Real time platform == sucks! (sorry for bad words) So you get my point…

    It’s sad that now I will need to force Firefox users to install google gears what will be a terrible solution because I have already ban complete Internet Explorer users from platform. (maybe IE9 later)

    As Federico said I will repeat again. Sometimes it’s better to be a follower than to do the right job. Developers who re using Web SQL Database are fully aware of damages it can bring to project but still you know what…

    Going to hawai, eating in fine restoran, etc… MasterCard can buy!
    Web SQL Database cant! It’s priceless! LoL!

    June 28th, 2010 at 16:33

  22. Blago

    “Why define a whole other language, when more elegant solutions exist within JavaScript itself”

    This sentence makes no sense. JavaScript has no existing solutions for querying structured data. In practice we are going to have to learn a whole new way to design our schemas and query the data. The existing solution is called “SQL” and it would be a huge relief if we could share some code/schema between the client and the server for once. I am not scared of having to change a keyword or two.

    If you absolutely have to invent something new, get John Resig to design a jQuery-like selector/filter API – that is elegant and yes… we are familiar with.

    Speed is another big concern. I would never elect voluntarily to run a JS callback for every row in the DB just so I can filter out everything but the few object I need. This is insane. We are in the era of mobile devices. Anyone sane will let the C code (with the help of indexes) handle that.

    As for the elegancy of code… Just look at the last SELECT example here: http://hacks.mozilla.org/2010/06/comparing-indexeddb-and-webdatabase/

    June 29th, 2010 at 11:14

  23. Guido Tapia

    Does anyone know of any abstraction js library that will allow indexdb / websql transparency? Perhaps even a localStorage fallback for IE?

    July 10th, 2010 at 22:12

    1. Sam Dutton

      Good question. Did you find anything?

      persistence.js does something similar, but different.

      March 26th, 2011 at 04:30

      1. JJ

        There’s a Javascript library called lawnchair – http://westcoastlogic.com/lawnchair/ – which seems to be an abstraction layer between your data requests and your data. It claims to support WebSQL and IndexedDB.

        August 12th, 2011 at 21:02

  24. Komrade Killjoy

    user storage is passe

    stuff everything in a “free” cloud owned by a megalithic datamining entity

    with handy US Federal data storage laws they “get to” store information on subject- err um citizens even longer

    win win

    July 28th, 2010 at 06:24

  25. pkario

    Really, none of the argument for not implementing Web SQL Database makes sense.
    * SQLLite is already there,
    * most developers don’t want to go back to dbase III+ (event old ones like me)
    * of course you know better what is good for us but…

    If begging is what you need:
    PLEASE…
    implement Web SQL Database…

    July 31st, 2010 at 04:02

    1. Greg Quinn

      It’s like Mozilla seems intent in becoming superfluous; after having worked on a client side persistence project with SQLite it feels like a game-changer for web-based applications, it’s lightening fast in action, it’s fast to implement and quick to debug. Fortunately, there are other cross-platform browsers that support SQLite and Mozilla is no longer the only game in town as a multi-platform browser choice. “SQL? Which SQL?” is a heading that still make me both angry and sad.

      July 31st, 2010 at 13:56

  26. sri

    You can find an indexed db playground at http://tinyurl.com/ff-idxdb. It has examples for the APIs. Wrote it since I could not find documentation elsewhere. Hope it helps.

    August 4th, 2010 at 00:57

  27. meh

    Will this switch solve the HUGE LAG and memory slurping of having a very very large bookmarks collection?

    will these ease a transition to NESTED TAGS?

    August 5th, 2010 at 03:41

  28. Marcelo Cantos

    I just followed the link to the page by the Chrome team. My favourite bit was this:

    Q: Why this over WebSQLDatabase?
    A: Microsoft and Mozilla have made it very clear they will not implement SQL in the browser. If you want to argue this is silly, talk to them, not me.

    This pretty nicely distils the vibe I’m getting just about everywhere I look. No one outside Firefox or Microsoft seems to agree with this direction. I suspect Microsoft just wants to tie the web to T-SQL, and IndexedDB, lacking even the rudiments of a query language, will give them the perfect excuse.

    On a related note, it strikes me as a bit disingenuous to mention Microsoft and the Chromium team one after the other, without giving any indication that their positions are quite different. This leaves the impression that the Chromium team are just as happily on board as Microsoft, which is clearly not the case.

    August 31st, 2010 at 06:09

  29. Jason

    Add me to the disappointed list. I appreciate all the work the people do on Mozilla for basically nothing but (scarce) credit. That said, I think they’re making the wrong choice. Other browsers are giving us lemonade while Mozilla is offering us lemons, water and sugar cane and telling us about how wonderful it will be to be able to make precisely the kind of lemon flavored beverage we want. What I forsee is that, eventually, someone will hack together a javascript library to get a compatibility layer for SQL out of Indexed DB for Mozilla and IE. It will be slower than the native implementations for Safari, Chrome and Opera. Eventually, the speed lag will become enough of an embarrassment that Mozilla will play catch-up and finally just implement Web SQL. IE probably never will, because they have a pathological need to be useless.

    As other posters have already addressed, the reasoning against Web SQL is fairly paper-thin and easily answered. It smacks of reverse-justification of a decision that has already been made. As someone who has stuck with Firefox as their primary browser since day 1, this makes me sad.

    September 1st, 2010 at 18:30

    1. greg

      To be fair about Microsoft’s reluctance to implement a SQLite persistence layer; it seems to me that they would be highly unlikely to support distribution of a SQL engine other than their own, and I’m sure that there are valid commercial reasons that they use internally to justify this. I wish they would instead implement SQL Server Express as part of MSIE as an alternative, but I assume that they consider this too much work to implement and/or that it competes with other product pathways. That’s their right as a company to do what they believe is in the best interests for the company. For this reason, using Microsoft as some kind of crutch for the nonsensical decision of Mozilla not to implement an SQLite persistence layer is simply absurd. I really expected more from you guys.

      September 2nd, 2010 at 10:29

      1. Christopher Blizzard

        Greg, I already replied to this on another post where you said more or less the same thing with the same tone. I’m sorry that you disagree with our decision on this.

        September 9th, 2010 at 10:11

  30. Blago

    Greg, I’m sorry for the decision you made because pretty much everybody on this thread disagrees with you.

    September 9th, 2010 at 10:39

    1. Steve Antal

      Blago, actually most people including me, agrees with him. I am highly disappointed in Mozilla, because it seems that Firefox is starting to become the new IE6.

      September 9th, 2010 at 10:42

      1. Christopher Blizzard

        Hyperbole, indeed!

        September 9th, 2010 at 14:25

        1. Brett Zamir

          Of course, the Mozilla team is very hard at work adding useful technologies like audio and video, and innovating on many fronts, and its mostly other browsers that have catching up to do as far as developer conveniences (E4X, fixed object iteration order, etc.)–notwithstanding the likes of my long-standing gripe about no external DTD support or withdrawal of enablePrivilege support. I think those of us in favor of SQLite can also chill a little bit in recognition that no doubt some SQLite layer will eventually come, even if it requires adding a script tag whenever we want to use it.

          But I agree with the sentiment about speed at any costs (and I’d add to that “dumb down the interface at any cost”). It’s not just about what COULD be done with technologies (esp. with a perception that the likes of Google and Apple are dominating the direction of HTML and who have more resources to take advantage of them) but how easy and robust it is for regular developers to do so, and do so in the immediate future.

          Maybe those at Mozilla who are pushing for this can actually implement at least some proof of concept SQL layer (SQLReallyLite) if it can be done for the data-centric crowd accustomed to such a model, or at least make some syntax which can handle joining of data, etc.

          In psychology, they say for any behavior you wish to remove, you need to provide a substitution, or otherwise there will be a vacuum. Mozilla should expect to run into some resistance whenever it removes or even plans to remove a feature especially if it is does not provide a better alternative (and not just better in some ways), though we should appreciate timing is not always under your control either.

          For my very humble part, I’ve started work on http://code.google.com/p/jsxqueryparser/ to try to eventually bring XQuery support to JavaScript (though a jQuery plugin which allowed querying multiple documents would be fairly trivial to implement today with IndexedDB (though not safe to expose fully to the user) ; I just like the syntax of XQuery and its lack of full exposure to JavaScript which a jQuery plugin would need to allow unless it used a JS parser to subset allowable syntax).

          Also, XQuery (a spec which was co-edited by those who created SQL itself), besides allowing relational-style joins (see http://en.wikibooks.org/wiki/XQuery/XQuery_from_SQL#Table_Joins ), has a syntax which is even easier to follow (and more powerful) than SQL, and works with hierarchical (document-centric data) as well as relational data (and can work directly on something as familiar and transparent as XHTML tables).

          Personally I think XQuery or a modified jQuery would be great candidates for replacing SQL and allowing the web to start off fresh (map-reduce, to my sense, not so much). So, if the browser will not support XQuery, then I hope we can bring XQuery to the browser. Volunteers welcome (brettz9 – yahoo).

          We do appreciate all your hard and innovate work on Mozilla! That’s why we get passionate about it…

          September 9th, 2010 at 19:58

          1. Marcelo Cantos

            A SQLite layer?? SQLite isn’t an API, or a language, or a standard, for that matter; it’s a rock-solid, proven-in-the-field, high-performance, 600:1-testing-code-to-production-code database engine that comes as one .h file and one .c file (ignoring the auxiliary files: shell.c, sqlite3.def and sqlite3ext.h). It is a piece of pure awesomeness that you couldn’t possibly hope to match with a few weeks (or, frankly, many months) of effort on a little side-project. In fact, I doubt it is even possible to match SQLite’s power via a layer on top of IndexedDB.

            What people are really bothered about is not that you won’t be able to do SQL, but that a truly fantastic piece of software that has delivered astounding results till now is being unceremoniously dumped in favour of a rewrite-the-world strategy that won’t deliver anything close to it in terms of reliability, performance and overall maturity for at least another couple of years. And frankly, two years is wildly optimistic, since IndexedDB won’t let Microsoft or Oracle drop their own “light” engines in either. Everyone will have to go back to the drawing board and start building their SQL-on-IndexedDB engines from scratch. And most of what makes the existing engines worthwhile (stability and decades of optimisations) won’t make it across to these new implementations.

            September 12th, 2010 at 03:20

          2. Brett Zamir

            I understand the concern about implementation possibilities and choices and I’m not taking a position about that; my point is simply that many of the complaints here ARE about the API and not having something in place for that is I think a big cause for frustration by developers who at least want some familiar (or at least similar) way to continue their old approach to getting data.

            As a fan of XML databases, I am also concerned about implementation capacity, as I wonder whether IndexedDB could in principle support such a database with high performance as well.

            September 12th, 2010 at 04:48

      2. Blago

        This was a typo. My comment was directed at Chris. It is more than obvious that few (if anybody) people in this thread are in favor of IndexedDB.

        September 9th, 2010 at 18:09

  31. Blago

    I too think that Mozilla people are out of touch and in denial. BTW 5 years ago I donated money for the NYT ad so I feel real pain when I say that.

    Today it looks like Mozilla couldn’t care less about developers. Think about it. FireBug was awesome back when Joe Hewitt was in charge but turned into a giant BUG since. In the meantime Webkit created WebInspector that is part of and maintained with the browser and works like a champ after a bit of used to.

    September 9th, 2010 at 18:24

  32. Meni

    I have just read this article, not the comments yet, so excuse me for rushing in to comment, if I make a fool of myself it’s only because i feel strongly about this situation. Isn’t this SVG vs VML all over again? From what I have read, it was egos and pride back then, and we all lost because of that. Had we have a agreed upon vector standard back then the web better today.

    What i’m arguing for is get the guys from Google, Apple, MSFT, Mozilla and Opera in a room, NO ONE leaves until we have white smoke. MOVE THE WEB FWD FAST. Ration the pizza while locked there :-)

    Sheesh, had JavaScript was to be created by a you guys, we would still be waiting for it today ;-) If there is a lesson to be learned i think it’s: choose something already, who cares if it’s less then perfect, no one is going to jail you for that.

    Good day (BTW, thks for FF, can’t live without it)

    September 19th, 2010 at 01:13

  33. Jaka Jancar

    I love it how you manage to say this like it’s a bad thing:

    “But SQLite itself isn’t a specification — it’s a release-ready technology!”

    September 27th, 2010 at 13:56

  34. Terry

    A huge +1 for web sql db.

    Terry

    October 12th, 2010 at 07:50

  35. Martijn Otto

    I think everything has been said before, so I won’t go into a long diatribe here. I just want to let everybody know that the lack of SQL in Firefox is really a very, very big letdown.

    November 5th, 2010 at 15:01

  36. dennis

    Ridiculous. This just gives me one more reason to ditch Firefox.

    November 22nd, 2010 at 11:15

  37. axmcomp

    As the vast majority of posters have already said, the justification is just -post rationalisation of a crappy decision taken for purely political reasons.

    The fact remains what developers want is an sql interface, not a a crappy btree interface. I don’t want to have to re-invent sql just to get the work done.

    Having said that in my experience this affects mainly mobile developers as we are developing complex javascript based webapps. As the majority of browsers are webkit based, they will continue to use websql so this decision is a niggle at the back of the head but won’t affect anything medium term (assuming they probably take 5 years to agree and implement it).

    I think this is going to be another case where Mozilla et al will eventually tuck their tail between their legs and implement what everyone else is using because otherwise firefox will become even less popular than it is today.

    When you look at the two technologies dispassionatly one is ten thousand miles ahead than the other.

    December 12th, 2010 at 18:02

  38. Ramin

    I am a Web Developer, and want to write APPs for smartphones in HTML/CSS/Javascript and SQL.
    I have allready some PHP Server Scripts that make use of SQL and communicate with my client side HTML/Javascript (Ajax). Now I would like to write offline Apps that use a local database instead of contacting a web server all the time.
    With WebSQL it is simple, to port my online apps to offline apps.

    Indexed DB may be nice, but I have to change a lot in my code.
    Could Mozilla not provide both WebSQL and IndexedDB, and give the Developers the choice?

    December 23rd, 2010 at 07:37

  39. Li Zhou

    I can not open the indexedDB database with Firefox 4 beta 8.
    var request = window.moz_indexedDB(“candy”, “candy store”);

    December 28th, 2010 at 09:46

  40. Juan Antonio

    It’s easy. No websql , no firefox

    January 10th, 2011 at 00:36

  41. vicm

    IndexedDB is Mozilla only product. A developer does not have to think about it much: not supported by all major browsers means I am not using it. PERIOD!!!
    It is usually IE that pulls such tricks. Now, Mozilla wants to join in.

    If you think that IndexedDB is such a great idea, implement it, and, also, implement WebSQL at the same time. If developers like IndexedDB, all browsers will implement it sooner or later.

    February 8th, 2011 at 17:46

  42. wHiTeHaT

    It is funny that i find this discussion, as being a kind of noob in this case.
    But let me tell you this.
    You can say what you want, indexedDB and SQL(ite), will go be the standard in ordinary websites.Of course browser depending.
    For the users who use a browser supporting SQL go become in trouble, that is for sure.
    The users who runs on a microsoft/mozilla browser can profit from both.
    It is a market strategics,and it is a clever one.
    Let the work be done for you instead of do all yourself.
    The browser and Os war is coming to an end.
    The final goal is the end user, and all go depend on the hardware.
    Wouldn’t it be nice to have a cpu per browser. ;)
    So who decide at the end what soft you run?
    The hardware supplier.
    And they let you think you buy the best.
    LOL it is so funny to read this discussion,who do the dishes?

    February 12th, 2011 at 17:25

  43. Remy97

    I agree with using a NoSQL approach. For many solutions, a relational model is overkill. I agree with data being structured in the form of JSON, but simply optimized with indexing capabilities. I also like the idea of MapReduce and scaling considerations for the future.

    February 17th, 2011 at 10:17

    1. Marcelo Cantos

      If you think that SQL is overkill, then you obviously haven’t seen the comparative code samples at http://hacks.mozilla.org/2010/06/comparing-indexeddb-and-webdatabase/.

      February 17th, 2011 at 20:34

  44. Sam Danielson

    Sqlite’s type system is a perfect match for JavaScript, e.g. 1 == ‘1’. The fact that it has a type affinity system, is fast, and was developed for an independent purpose is amazing. Even if IndexDB surpasses all expectations, it will still be a loss to not include some portion of sqlite. It’s a great piece of software. If IndexDB must be included, why can’t we have both? Embrace, extend, …

    March 4th, 2011 at 13:26

    1. Sam Danielson

      Never mind. 1 ‘1’ in sqlite. It’s surprising how long an old assumption can remain unchecked.

      March 14th, 2011 at 16:23

  45. Kevin Layman

    insulting comment deleted

    March 21st, 2011 at 10:10

    1. Paul Rouget

      Being insulting is not helping here. I will delete any other comments you’ll post. You’re probably way smarter than Mozilla + Microsoft + Google, but it doesn’t allow you to be insulting.

      March 22nd, 2011 at 07:15

      1. Marcelo Cantos

        I have reread my copies of the deleted posts. They were certainly forcefully presented, but also hit the nail on the head on several counts. Sure, words like “idiots” and “morons” were better left on the bench, but, given the (IMO) colossally bad decision that has been made, I can understand Kevin’s vehemence. And, based on the comment trail above, it is obvious that he’s not alone in this sentiment; from top to bottom, his posts had me thinking, “Yes, my thoughts exactly!” and were thus an important contribution to this debate.

        Unless a post is offensive in nature (i.e., threatening, lewd, etc.), there is no justification for removing it. You could have castigated him for the inappropriate tone and left it at that. And if you thought his arguments were wrong or baseless, then the appropriate response would have been a rebuttal.

        The removal of Kevin’s posts strikes me as nothing short of circling the wagons by censorship. Unilaterally vetoing his future posts is a disgrace.

        Show the community that you are genuinely interested in hearing our voice. Reinstate Kevin’s original posts.

        March 22nd, 2011 at 14:48

  46. Marcos Lima

    I know that my comment is coming too late, but I’m interested on indexedDB concerns…

    I just would like to load data to the local client storage like JSON arrays, strutured objects, simple arrays of any data and after all query like SQL (using simple relationships), like Object-oriented DB. Also, retrieve all the data like objects again would be pretty nice…

    I guess you are making this IndexedDB too much difficult coz’ you’re aiming a cutting-edge implementation…

    Make it as easy as possible man! =)

    Thanks,

    Marcos Lima
    System Analyst/Web Developer
    Brazil

    March 24th, 2011 at 09:51

  47. Marcos Lima

    I would like to replace the strategy method that query the data, or set myself the indexed properties, and also crete in-memory views, I think that would be useful, create handlers to the storage modifications, like events to the buttons actually do.

    March 24th, 2011 at 10:00

  48. MarkIt

    Again, HTML5 standards are not standards. The cake is a lie.
    For every new feature we have to wait until a jquery thing abstract all the interactions with the browser, erasing any advantage browsers developers think to gain with their own solution.
    At this point I just hope Adobe include their standard sqlite in the flash plugin… oh wait, the Apple thing.

    March 26th, 2011 at 08:09

  49. John Z

    Kevin Layman’s comments were right on the mark. Delete the parts you found offensive and respond to his technical points.

    Censorship sucks!

    March 28th, 2011 at 16:38

  50. Lucas

    As I don’t want to repeat what others have said like a hundred or thousand times in this discussion I can only say:

    * No SQLite implementation in JavaScript on top of IndexedDB will be ever nearly as fast as the WebSQL solution

    * Have you been to lazy to re-write the SQLite documentation or is it a little baby thing to do what all the grown-up’s refuse? Like the Internet Explorer in past times …

    * I think you have made the wrong decision and I hope for you (really!) that you soon implement the WebSQL standard or it will cost you a lot of active developers and users. Don’t know if it’s that what you want?

    April 19th, 2011 at 12:03

Comments are closed for this article.