This is why servers are so frequently sending content to 'empty caches'. Another survey from 2007 reveals that 40-60% of unique users came to yahoo.com with an 'empty cache' at least once per day (meaning the yahoo components were not in the caches of regular users). The survey also revealed that 20% of page views were done with an 'empty cache'. Sounders says that these numbers are in line with the figures he's heard about from other major websites.
That's a pretty significant number of users with an empty cache even if their assets are optimized for maximum caching. The 'empty cache' user's browser will bypass the disk cache and request all of the components to load the page (more HTTP requests means worse performance), while the full cache user will have most of the components for a site already in the cache. However, the browsers with a 90% full disk cache still don't have much of a user's frequently used cached components because there is not sufficient space in the browser's cache and the eviction algorithms are not as smart as they could be.
- Internet Explorer: 8-50 MB
- Firefox: 50 MB
- Safari: no max size setting (???)
- Chrome: < 80 MB (varies depending on available disk space)
- Opera: 20 MB
Considering that most computers have at least 100GB of disk space, it wouldn't be unreasonable, Sounders says, to allocate 5GB for the browser so that performance could increase.
Chrome cache load
However, a cache will eventually fill up for people that never empty their cache, so cache eviction must also get smarter and have greater control over what users can clear. Sounders says that most eviction algorithms are LRU-based, which means that the least recently-used resources are deleted. This algorithm doesn't take into account the growing knowledge about web performance bottlenecks. Scripts, for example, are more costly to download because they block other downloads and rendering. A better algorithm would evict most images before evicting scripts.
For now, web developers need to keep the empty cache user experience in mind. Users should review their cache settings and jack up the defaults. You can also post them in Sounders' Browser Disk Cache Survey form to help his research. Sounders expects most of the major browsers to improve eviction algorithms and disk cache sizes in their next releases. If you can't wait that long, you can also check out Polipo, a small and fast caching web proxy.