Offline Files in HTML5: The FileSystem API

I’ve been experimenting with the FileSystem API in Chrome over the last couple of weeks and thought I’d share my musings with a little demo of a file syncing system that I am thinking of using in an application where the user can download a copy of the file, work on it offline and then sync it when connected again. In this post I just want to show the pull side of the sync where the file is downloaded and replicated in the local file system (within the browser) and I’ll follow up with a second post showing the silent sync with the server (push and pull based on last edit timestamp) once I have that part working.

On the server I just have a flat file structure within a root folder and a few files from which I build the list in the UI. Currently each file entry has a sync button to manually just pull the file down from the server into the local file system. Now, this file system is not your operating systems file system but instead a sandboxed environment within the browser that is only accessible to the application that requested and created it. To be able to store files persistently you need to initially ask the user to give up this space and allow you to create the local file system for the application.

window.webkitStorageInfo.requestQuota(
    window.PERSISTENT
  , 5*1024*1024
  , function(gb) {
      window.webkitRequestFileSystem(
          window.PERSISTENT
        , gb
        , fileSync.init
        , fileSync.err
      );
    }
  , fileSync.err
);

Here I request 5MB of storage space to put the files into and once the user accepts the file system is requested with the initialisation function of the fileSync object given as the success callback. This will only ask the user once for the allocated space and on subsequent requests will just request the file system and run the init function. On initialisation the fileSync object sets up some event handling and checks if the application is on or offline. If it is online it requests the latest file list from the server and enables the sync buttons. If offline it just reads the local directory structure and runs down the list of files marking the ones that are already synced locally in the UI leaving the sync buttons hidden and disabled.

// set up event handlers and file system
api.init = function (fs) {
 
  dir = document.getElementById('dir-tree');
  dir.addEventListener('click', api.fileAction, false);
 
  root = fs.root;
 
  // off/online detection
  w.addEventListener("offline", api.toggleOnlineState, false);
  w.addEventListener("online", api.refreshFiles, false);
 
  if (w.navigator.onLine) {
 
    api.refreshFiles();
    return;
 
  }
 
  api.toggleOnlineState();
  api.syncStatus();
 
};
 
// get the latest list of files from the server
api.refreshFiles = function () {
 
  var xhr = new XMLHttpRequest;
 
  xhr.open('get', 'file-list.php', true);
  xhr.onerror = api.err;
  xhr.onload = function () {
 
    dir.innerHTML = this.response;
    api.syncStatus();
 
  }
 
  xhr.send();
 
};
 
// marks synced files in the dir tree
api.syncStatus = function () {
 
  var dr = root.createReader();
  dr.readEntries(api.updateStatus, api.err);
  api.toggleOnlineState();
 
};
 
// show / hide sync buttons when off or online
api.toggleOnlineState = function () {
 
  var i = 0
    , d = 'none'
    , sy = dir.querySelectorAll(".sync");
 
  if (w.navigator.onLine) {
    d = 'inline-block';
  }
 
  for (i = 0; i < sy.length; ++i) {
    sy.item(i).style.display = d;
  }
 
};
 
// mark synced items in the tree
api.updateStatus = function (listing) {
 
  var i = 0, entry;
 
  for (; i < listing.length; ++i) {
 
    entry = listing.item(i);
    api.flagSynced(entry);
 
  }
 
};

There is a bit going on here and granted it can all do with some optimising but anyhow… within the init function I set up a reference to the root of the file system which is a DirectoryEntry object. I add event listeners to the on and offline events to show and hide the sync buttons as appropriate and check if the application is on or offline to do the relevant initialisation. If it is offline it goes straight to setting up the tree by reading the files in using a DirectoryReader to list the contents of the root directory. I loop through the returned EntryArray and flag the files as synced by adding a class to the list item where they reside in the UI. If you want to put the pieces of what is going on here you can check the full code on Github.

OK, that is great but how did I get the files into the local file system in the first place…

Ajax file download

You may have read some of my other posts on the Level 2 spec of XMLHttpRequest which has added some very nice new features. One of these features is the addition of the responseType. With this we can set the response type of a request to be a Blob or ArrayBuffer so that we can deal with binary data directly, cool eh. With this I request the file and write the response data straight into a file on the local file system.

// pull file down into local
api.pull = function (url, name) {
 
  var xhr = new XMLHttpRequest;
 
  // request the file
  xhr.open('get', url, true);
  xhr.responseType = 'arraybuffer'; // give us an array buffer back please
  xhr.onload = function () {
 
    var res = this.response; // ArrayBuffer!
 
    // get the local file or create it if it doesn't exists
    root.getFile(name, {create: true}, function (fe) {
 
      // get a handle to write to the file
      fe.createWriter(function(writer) {
 
        // create a blob builder to append the data to
        var bb = new w.WebKitBlobBuilder;
 
        writer.onwriteend = function () {
          api.flagSynced(fe) // mark as synced in the UI
        }
        writer.onerror = api.err;
 
        // append the data and write to the file
        bb.append(res);
        writer.write(bb.getBlob());
 
      });
 
    }, api.err);
 
  }
 
  xhr.send(); // send the request
 
};

This is really awesome, hopefully the comments in the code explain well enough what is going on. The (WebKit)BlobBuilder object provides methods to create a blob and append data to it which is great if we want to deal with chunked data or slice a file up and stream it back up to the server, more to come on that in another post!

Once we have our local files we can open them by pointing a window to the files URL in local storage. Guess what, there’s a method for that too – toURL

api.open = function (name) {
 
  // get the file and open it
  root.getFile(name, {}, function (fe) {
 
    w.location = fe.toURL();
 
  }, api.err);
 
};

Taking the application offline

Setting this application to run offline requires that we create a manifest file for the application cache and link to it in the HTML tag of the index page.

<html manifest="sync.appcache">

The cache file had me running in circles a bit as once the files are cached the browser always uses the cache and doesn’t even attempt to get the files from the network when connected. This is understandable but trying to download files when online resulted in 404s as it tried to get them from the cache. I ended up adding the file-list.php page to the NETWORK section of my manifest file so that I could get the fresh files from the server when working online. The other files are just js and css so great cache those ’til the cows come home. Below is what I ended up with in my manifest file.

CACHE MANIFEST
 
NETWORK:
file-list.php
/fs
 
CACHE:
index.php
css/style.css
css/images/sync.png
js/sync.js

I’m amazed how simple all this stuff is and how powerful these new APIs are becoming, with a little help from Eric Bidelman’s article on HTML5 Rocks I wrote this thing in a couple of hours last night.

Source: http://www.profilepicture.co.uk/tutorials/html5-filesystem-api/