Revitalizing Caching

Apparently, there are only two hard problems in computer science: cache invalidation and the naming of things (or so Phil Karlton’s dictum goes). Earlier this month, we invited representatives of Twitter, Facebook, SproutCore, Palm’s webOS, Microsoft’s “Office On The Web”, Yahoo, and Google to talk to us about the former problem (amongst other things), though we also learned something about the latter.

Caching is an important issue to get right on the web, not least of all because of the proliferation of web applications on mobile devices. The goals of our caching summit were to identify use cases that would help us move forward with caching and with HTTP request efficiency. How desirable was rolling up our sleeves to look at HTTP/1.1 Pipelining in Firefox, for instance? What else was needed at the HTTP layer? And was the vaunted HTML5 AppCache, implemented in Firefox 3.5 onwards, actually useful to developers? What else needed to be exposed to web applications, either within content or via additional headers?

Developer feedback is invaluable, and is increasingly the basis of how we want to evolve the next integral pieces of the web platform. Web developers are one of our primary constituencies; going forward, we want them to help us prioritize what we should implement, and what we need to focus on with respect to web standards. We chose our attendees wisely; if any group of people could talk about web applications at scale, the current performance of the cache, and their wishlist for future browser caching behavior on the web platform, it was this group of people. And the feedback they gave us was copious and useful — our work is cut-out for us. Notably, we’ve got a few actions we’re going to follow-up on:

  • Increase the default size of our disk and memory caches. Firefox’s disk cache is currently set at 50MB, a small-ish number given the amount of disk space available on hardware currently (and although this limit can be increased using advanced preferences, few users actually change the default). This is low-hanging fruit for us to fix. An interesting open question is whether we should determine disk cache size heuristically, in lieu of choosing a new upper bound. Steve Souders, who attended our caching summit, blogs about cache sizes, as well as premature cache evictions.

  • Conduct a “Mozilla Test-Pilot” project to get more data about how exactly we cache resources currently. This is part of a larger question about updating our caching algorithm. Like other browsers, we use an optimization of the Least Recently Used (LRU) caching algorithm, called LRU-SP. Data that we would want to gather includes determining what the hit rate, mean, variance and distribution of cached resources are. What’s the average lifetime? How about different modes where our LRU-SP replacement policy doesn’t work well for certain apps, where big resources (such as an essential script file) may get eliminated before smaller ones (such as map tiles)? We’ll also have to research the role that anti-virus software plays in routinely flushing out the cache, leading to further occurrences of untimely eviction of relevant resources.

  • Explore prioritization of resources in the cache based on MIME type. For instance, allowing for JavaScript (text/javascript) to always get higher priority in terms of what gets pruned by our LRU-SP algorithm. A good follow-up for this would be to get Chrome, IE, Apple, and Opera to discuss this with us, and then document what we come up with as a MIME-type based “naive” prioritization. We also want to allow developers to set resource priority on their own, perhaps through a header. This is likely to be an ongoing discussion.

  • Really figure out what hampers the adoption of HTTP/1.1 Pipelining on the web, including data from proxies and how they purge. While Pipelining is enabled in Mobile Firefox builds (such at that running on the Nokia N900 and on Android devices) by default, we have it turned off in desktop Firefox builds. We do this for a variety of reasons, not least of all the risk of performance slow downs if one of the pipelined requests slows down the others. It’s clear from our discussion that many who attended our caching summit think pipelining will help their applications perform much better. The situation on the web now with respect to pipelining is a kind of hostage’s dilemma: of the main desktop browsers, nobody has turned on pipelining, for fear of a scenario that slows down performance (leading to that particular browser being accused of “being slow”). The developers who visited us threw down the proverbial gauntlet; at a minimum we’ve got to figure out what hamstrings the use of pipelining on the web, and determine what we can actually do to remove those obstacles.

  • Figure out how to evolve the HTML5 AppCache, which frankly hasn’t seen the adoption we initially expected. While we tend to view parts of HTML5 such as Cache Manifests and window.applicationCache as yet another performance tool (to ensure web applications rapidly load upon subsequent accesses), it is different than the general browser cache. What’s in a name, and why is naming something amongst the hardest problems in computer science? The use of the word “cache” to describe the parts of HTML5 that deal with offline web applications has confused some developers. What we’re calling the HTML5 AppCache was primarily conceived of to enable offline web application use, but many applications (such as those built with SproutCore) treat it as an auxiliary cache, to ensure that applications have a rapid start-up time and generally perform well. Why, we were asked, should we have two things: a general purpose browser cache, and something else, uniquely for offline usage? On the one hand, the HTML5 AppCache allows web apps to act like native apps (enabling “rapid-launch icons” on some mobile handsets), perhaps even eventually integrating with native application launchers. And on the other hand, the HTML5 AppCache’s separateness from the general cache may mean that we coin different APIs to work with the general cache. In general, simultaneously evolving multiple APIs with “cache” in the name may be confusing. But that’s why naming is amongst the hard problems, and that’s why we have to architect the next iteration mindful of the potential for both redundancy as well as confusion.

  • We’ve got a tracking bug in place with a bold moniker: “Improve HTTP Cache.” You’ll see the gamut of changes we’d like to introduce here, including benchmarking our cache against Chromium’s (and perhaps just using Chromium’s cache code, if we need to).

    Caching is important, but difficult. It would be fair to describe most of the near-term evolution of the web that way, whether that is the introduction of an indexable database capability, streaming video, or new layout models within CSS. These evolutions won’t necessarily happen within a standards body or on a listserv, but rather through rapid prototyping and meaningful feedback. That’s why we have to talk to web developers to help us do the right thing, and that’s why we’ll keep organizing meet-ups such as the recent caching summit.

    About Arun Ranganathan

    More articles by Arun Ranganathan…


  1. Anonymous

    And caching just boils down to the proper naming of things. :)

    Personally, I’d love to see a cross-site content-addressable caching mechanism. “Load jquery-1.4.2.min.js from my server, which has hash $DECENT_HASH, but if you already have the file with that hash then use it even if it didn’t come from my server.”

    That would reduce the temptation to load third-party scripts for performance and caching reasons, which would improve security.

    May 3rd, 2010 at 13:16

  2. Ted Mielczarek

    FYI, the last time we tried enabling pipelining, for SSL sites only ( ), we still ran into a problem where sites were using reverse proxies that didn’t understand pipelining, with the net result being that we broke people’s bank websites ( ). AFAIK fear of performance has nothing to do with it, and “breaking the web” because of broken intermediate proxies has everything to do with it.

    May 3rd, 2010 at 13:19

  3. Arun Ranganathan

    @Anonymous, I agree that redundant third-party script caching should be a solvable problem, and we’ve talked about similar solutions before. I also sometimes wonder whether we shouldn’t make life easier by shipping with common libraries. @Ted — you’re right that “fear of performance” isn’t the *only* reason, but we’re still at the hostage’s dilemma stage, and I think we need to evangelize downstream proxies. This could be a case of compiling data about what stymies pipelining, and knocking on doors, just like we did with webmasters back in the days the DOM changed.

    May 3rd, 2010 at 14:19

  4. Alkarex

    Opera for desktop (checked on version 10.53) has pipelining enabled by default.

    May 3rd, 2010 at 15:03

  5. discoleo

    “Load jquery-1.4.2.min.js from…” and “third-party script caching” – I agree fully with these comments. A standardised set of third libraries (like the C/C++ libraries). Should probably allow for versioning of such libraries, and sites should define a minimum version they request in order to run.

    Some sites are also using complex js to render the page, so my question: would it be feasible to cache a compiled version of the js/page? This will offer a significant speed-up, because the browser does not need to run all the js and render the whole page again.

    May 3rd, 2010 at 15:47

  6. Yusuf Goolamabbas

    There are a lot of old Squid installations out there and Squid basically talks HTTP/1.0 to clients. That’s going to put a damper to pipelining

    You need to work with Proxy Cache developers such as Squid and Yahoo Traffic Server to ensure that they work properly with pipelining and then encourage sysadmins to upgrade to a supported proxy cache

    May 3rd, 2010 at 15:58

  7. Ehsan Akhgari

    I think one perhaps simple improvement that we can make to our cache eviction algorithm is using the Places concept of frecency, so that we first try to evict entries that the user has least frequently and least commonly visited.

    May 3rd, 2010 at 19:21

  8. sysKin

    Can I suggest a dot point “go through all the cache bugs and find low-hanging fruits”. For example, Firefox cache is completely wiped out every time you download a large-ish file. See bug 531700.

    May 3rd, 2010 at 20:57

  9. Ehsan Akhgari

    One simple measure that we can use for cache entry eviction is using the Places notion of frecency, i.e., evict the least recently and least commonly visited resources first.

    May 3rd, 2010 at 22:11

  10. Alkarex

    Opera for desktop (checked on version 10.53) has pipelining enabled by default.
    (New message because my comment from yesterday seems filtered)

    May 4th, 2010 at 11:50

  11. Dan

    I’d like to see a good attempt to improve the cache before adopting Chrome’s. I’m not a NIH person in general, just that I think Mozilla has the capabilities to produce something equally as good or better, and if you take too many pieces of other products then competition will be reduced (see Apple’s JIT assembler, Chrome’s IPC code).

    May 4th, 2010 at 12:37

  12. Yoav Weiss

    IMO, the 2 main reasons that prevented pipelining adoption are:
    * There is no way for a server to notify a browser that it supports pipelining. The specification suggests a “trial and error” approach that results in performance problems when the server does not support pipelining, as well as compatibility problems when the servers fail to handle multiple requests.
    * There is no way to send responses in a different order from the one in which the requests arrived. That may lead to performance problems if a response to early request is delayed from some reason. Other responses on that pipe are then blocked and cannot be returned to the user.

    We can gracefully address these 2 issues using:
    * An “X-pipeline-support: yes” response header that would indicate it supports pipelining
    * An “X-pipeline-id: ” request and response headers that will enable the server to deliver responses as they come, and will prevent possible performance penalty.

    In a way, both these issues were addressed by Google’s SPDY protocol proposal, but I think there’s a place to revive pipelining (with some extensions to make it more usable) in a way that will keep current protocol stack to provide almost similar benefits without the cost of “breaking the web”.

    May 5th, 2010 at 00:22

  13. Kim Sullivan

    Increase the size of the disk cache – don’t forget the effects a disk cache can have on SSD drives. Already, there are many pages that show how to either disable the disk cache completely, or how to move it to memory or a ram-disk.

    I have no idea what the actual effect of a 50MB or 100MB disk cache on the wear of an SSD drive is, but even a 50MB disk cache leads to the perception that Firefox reduces the lifetime of a SSD disk. A larger cache could make this problem better (relatively less evictions and reloading, less repeated writes, longer lifetime) or worse (generally a larger amount of data). Maybe do some tests with regards to the amount of data that is written to the disk cache, and then some documentation and marketing effort to explain it to users? Or special handling of SSD drives?

    May 5th, 2010 at 00:53

  14. Arun Ranganathan

    @Kim: can you point to data which shows the Firefox “reduces the lifetime” of a SSD disk through the disk cache? Do you mean constant writes/evictions literally reduce the lifetime of the disk? I’m also not sure we can make an elegant exception for SSD disks, but more data here will help.

    May 7th, 2010 at 14:14

  15. Kim Sullivan

    @Arun – like i said in my post, I have no idea about the actual effect firefox’s cache has on SSD drives, just that writes to SSD disks are percieved to be “evil”.

    It’s just that SSD drives really do have a limited amount of data that can be written to them (you can’t just overwrite some old data, you have to wipe a whole frame before you can write to it again, and theres only so much wipes you can do before the electrochemical reaction stops working, or something). For example, Kingston estimates that on a 64GB SSD disk, you can write 22GB of data to it every day for 5 years (,0101-237939-0-2-3-0-jpg-.html). Now, I don’t think normal users are going to download 22GB of web every day, so having the disk cache on a SSD drive is pretty safe, and is not going to reduce the lifetime of a disk in any significant way.

    But still there is a perception that expensive SSD drives (and as as of today, this means all of them) should not be used for things like disk caches – basically they should be just a mostly read only boot drive, with all regular data moved elswhere (such as disk caches, profiles or even the windows page file).

    As a result, there are many pages on the internet that show you how to disable the cache, just google “ssd disable firefox cache” (for example – “Since SSD’s have limited write-cycles, Firefox’s behavior of writing cache files to disk is not good”).

    I’m not saying that this is good advice or bad advice- I simply don’t know. I have no idea how much data firefox writes to its file cache every day, or how much data it writes to the SQLite database and how daily firefox usage will reduce the lifetime of a SSD disk (esp. since the drive vendors seem to be rather vague about the actual lifetime).

    So I think it would be a good idea to run a TestPilot study that would collect information about the amount of data a typical user writes to a hard disk on a daily basis, and afterwards educate users on the effects of having the disk cache on an SSD disk.

    Maybe a larger disk cache will make this actually better, because file won’t have to be redownloaded as often.

    May 7th, 2010 at 14:53

  16. Baboo

    AFAIK, is increasing the cache size moot as long as the 8192 objects bug isn’t fixed…

    May 9th, 2010 at 10:00

  17. Baboo


    May 9th, 2010 at 10:28

  18. Brett Zamir

    Is there an appropriate forum for contributing to the topic (at a high level)? I have some ideas which may be beyond your scope, but which I think would still fit in nicely with a caching discussion.

    Among other ideas, I think it would be great to allow a special protocol link and optionally targeting a file package with a manifest (or a manifest pointing to packages) which could trigger not only the decompression and caching of resources, but also specify update conditions (for the manifest itself as well as its constituent components) and optional registering of the package as protocol/urn or content handlers (or namespace handlers if such a thing could be created) so that a shared cache becomes workable and robust.

    It would not only work to cache frequently used files on the web like jQuery, but it could allow the likes of the scholarly TEI XML language, now applying for its own content type, to point to default XSL stylesheets which could offer themselves for use against any TEI files encountered on the web, thus avoiding the need for re-downloading a large set of stylesheets and thereby making such very semantically rich XML documents practical on the web (and without complicating things nor dumbing it down by converting it to XHTML server-side).

    There are other advantages to allowing a single standard hierarchical (preferably XML) file packaging manifest format (given that the web is mostly limited to specifying individual pages with a few accompanying resources) such as single HTTP request access of bundled resources, hierarchical sitemaps, XLink-style linkbases for use by the browser or search engines in suggesting links from/to sites, Gopher 2.0 navigation from link set to link set, and my favorite, transparent client-side mash-ups (and mash-overs, mash-unders, and mash-downs: see ) which could be assemblable via JavaScript, XSL, or XQuery.

    The Library of Congress has an XML standard, METS (see ) which could serve as a very good beginning for such a manifest, but it would need some further standardization to work across browsers and meet web needs.

    As I recall, the HTML5 editor suggested an IETF proposal would be suitable for such a proposal, but I’m afraid I do not have the capacity at this point to write such a specification on my own. Might anyone be interested to collaborate or point me to an appropriate venue for raising this? Maybe Drumbeat?

    May 12th, 2010 at 20:01

  19. Jimmy

    @Yoav Weiss By advertising that the HTTP server supports HTTP/1.1 the server is declaring that it supports pipelining, as this is part of the HTTP/1.1 standard, no?

    May 17th, 2010 at 15:37

    1. Christopher Blizzard

      Unfortunately, because of the fact that there are a lot of intermediaries out there that don’t get pipelining right advertising 1.1 support isn’t enough.

      May 17th, 2010 at 20:12

  20. Yoav Weiss

    @Christopher Blizzard – You are right, and my earlier proposition does not cover proxies. So I’d like to suggest some enhancements.
    We need a mechanism that will help detect intermediary proxies, and will enable them to declare their support for pipelining. If such a proxy is present and it does not declare pipelining support, it is probably an older proxy and we should avoid pipelining.

    Proxy detection can be achieved using:
    * “X-pipeline-hash” request header field that will hash all the request headers and their values (besides the X-pipeline-hash header itself)
    * An equivalent “X-pipeline-hash” response header field that will hash response headers and values

    These will enable both browser and server to know if the request or response were modified along the way. If so, then we have an intermediary proxy.

    A mechanism for proxies to declare pipelining support can be:
    * “X-pipeline-proxy: no” request header field will be sent by default by the browser
    * Pipeline aware proxies will modify that header field to “X-pipeline-proxy: yes”
    * The server will mirror the received header field to the response

    That way, the client can start sending pipelined requests after the initial response was received if no proxy was detected or if a pipelining supporting proxy was detected.

    There can be scenarios where this mechanism fails, but then the current “save sent requests in case of failure” fall-back mechanism can be used.

    Obviously, this suggestion is something I just came up with and there’s always room for improvement, but IMO it can provide a good enough way to start pushing pipelining and make the web just a little bit better.

    I hope I’m making myself clear…


    May 18th, 2010 at 02:51

  21. […] 원저자: Arun Ranganathan – 원문으로 가기 […]

    June 5th, 2010 at 02:58

  22. What cache replacement strategies do various popular web browsers use?…

    According to [1] most current browsers use a cache repacement strategy called LRU-FP introduced by Cheng and Kambayashi back in 2000 [2]. Cache replacement in the context of web caching is a lot harder than in classical systems (i.e. CPU caching hierar…

    June 16th, 2012 at 11:44

Comments are closed for this article.