Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

WebAssembly: Caching to HTML5 IndexedDB

DZone's Guide to

WebAssembly: Caching to HTML5 IndexedDB

This article is a continuation of a series exploring how we can build and work with WebAssembly modules using Emscripten.

· Web Dev Zone ·
Free Resource

Deploying code to production can be filled with uncertainty. Reduce the risks, and deploy earlier and more often. Download this free guide to learn more. Brought to you in partnership with Rollbar.

This article is a continuation of a series exploring how we can build and work with WebAssembly modules using Emscripten. The previous articles are not required reading to understand what we're going to cover today but, if you're curious, you can find them here:

Today we're going to continue using a bare-bones WebAssembly module (no Emscripten built-in helper methods) just to keep things as clear as possible as we dig into an important topic.

One of the main reasons why we would want to use a WebAssembly module in the first place is for the performance improvements that it brings but we haven't yet made use of one key performance item.

Typically, when you go to a webpage that has a JavaScript file, the browser will cache it so that it doesn't have to download it again. The next time you go to that same webpage, if the file is in the browser's cache, it will load that rather than pull it from the server, which saves time.

If you've been watching your network traffic, while working with the WebAssembly examples so far, you may have noticed that the wasm file is requested every time your page is loaded which isn't desired if the module hasn't changed.

HTML5 IndexedDB

WebAssembly modules were created with the ability to cache the compiled module in mind but the trick is that the caching is something that needs to be done explicitly by us.

In JavaScript, modules are cached to IndexedDB.

Up until this point, with all of my WebAssembly testing, I've simply been using the file system and double-clicking on the HTML file to test things in the browser.

For security reasons, however, browsers will not allow websites to access IndexedDB from a local file which means we need to set up a server of some sort. I'm a developer on Windows so I'm going to use IIS.

IIS has a whitelist of file extensions that it will allow a website to provide and .wasm was not in my list which resulted in a 404 error when the page tried to fetch the wasm file.

Adding .wasm to the list in IIS is fairly simple:

  • Open up Internet Information Services (IIS) Manager (found in Control Panel, Administrative Tools).
  • You can set this at the root, Default Web Site, or at the individual Application level.
  • Double click on the MIME Types link.
  • Click the Add... link on the Actions pane to the right.
  • Enter wasm for the file name extension.
  • Enter application/wasm for the MIME type.
  • Click OK.

Caching

So far, we've been working with the module's instance but the result object from the WebAssembly.instantiate call also returns a module object which is the compiled module that we can cache in an IndexedDB database:

// Request the wasm file from the server and compile it...
fetch(sWasmURI).then(response => 
  response.arrayBuffer()
).then(bytes =>
  WebAssembly.instantiate(bytes, g_importObject)
).then(results => { 
  // We've been working with the .instance object so far
  objModuleInstance = results.instance;

  // The results object also holds a .module object which is what we can cache:
  // results.module 
}); 

When we retrieve the compiled module from the cache, we will need to pass it to WebAssembly.instantiate but there are a couple of differences compared to when we download the file.

  • The first difference is that we don't need to do a fetch or set up an arrayBuffer.

  • The second difference is that the return object from the instantiate object is the instance itself.

WebAssembly.instantiate(objModule, g_importObject).then(instance => 
  g_objModuleInstance = instance 
); 

The following is some example code that shows how you can work with IndexedDB to cache and load WebAssembly modules:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
  </head>
  <body>
    <input type="button" value="Test" onclick="OnClickTest();" />

    <script src="IndexedDB.js"></script>
    <script type="text/javascript">
      var g_importObject = { 
        'env': { 
          'memoryBase': 0,
          'tableBase': 0,
          'memory': new WebAssembly.Memory({ initial: 256 }),
          'table': new WebAssembly.Table({ initial: 0, element: 'anyfunc' })
        }
      };

      // The WebAssembly module instance that we'll be working with
      var g_objModuleInstance = null;

      // If we need to change the structure of the database, we can increment
      // the DB_VERSION value to trigger the onupgradeneeded event when
      // opening the database
      var DB_VERSION = 1;
      var DB_NAME = "WasmCache";
      var DB_OBJSTORE_MODULES = "Modules";

      // We've set things up in such a way so that each wasm file can have a
      // version and we only clear the items from the cache if the version
      // doesn't match
      var g_sTestWasmURI = "test.wasm";
      var g_sTestWasmVersion = "1.0.0";

      // Check to see if the module is cached and, if so, use that. 
      // Otherwise, download the module and cache it.
      GetCompiledModuleFromIndexedDB(g_sTestWasmURI, g_sTestWasmVersion);


      function GetCompiledModuleFromIndexedDB(sWasmURI, sWasmVersion) {

        // If we successfully opened the database then...
        OpenDB(DB_NAME, DB_VERSION, HandleUpgradeDB).then(dbConnection => {

          // If we successfully obtained the requested record then...
          GetRecordFromObjectStore(dbConnection, DB_OBJSTORE_MODULES, sWasmURI).then(objRecord => {

            // If the version stored for this module doesn't match the
            // version we need then the module cached is out of date...
            if (objRecord.WasmVersion !== sWasmVersion) {

              // Have the record deleted and then fetch the proper file
              DeleteRecordFromObjectStore(dbConnection, DB_OBJSTORE_MODULES, sWasmURI).then(result => { 
                LoadWebAssemblyFromFile(dbConnection, sWasmURI, sWasmVersion);
              });

            } else { // The cached record is the version we need...

              // Have the module instantiated.
              //
              // NOTE: Unlike when we pass in the bytes to instantiate in
              // the LoadWebAssemblyFromFile method below, we don't have a 
              // separate 'instance' and 'modules' object returned in this
              // case since we started out with the module object. We're
              // only passed back the instance in this case.
              WebAssembly.instantiate(objRecord.WasmModule, g_importObject).then(instance =>
                g_objModuleInstance = instance // Hold onto the module's instance so that we can reuse it
              );

            }

          }, sErrorMsg => { // Error in GetRecordFromObjectStore...

            // We weren't able to pull the module from cache (most likely
            // because it doesn't exist yet - hasn't been cached yet). Log
            // the error and then fetch the file.
            console.log(sErrorMsg);
            LoadWebAssemblyFromFile(dbConnection, sWasmURI, sWasmVersion);

          });

        }, sErrorMsg => { // Error in OpenDB...

          // Log the error and then fetch the file (won't be able to cache 
          // it in this case because we don't have a database connection to
          // work with)
          console.log(sErrorMsg);
          LoadWebAssemblyFromFile(null, sWasmURI, sWasmVersion);

        });

      }

      // Called by indexeddb if the database was just created or if the
      // database version was changed
      function HandleUpgradeDB(evt) {
        // Create the object store which will hold 3 properties:
        // • WasmURI - (primary key) e.g. 'test.wasm'
        // • WasmVersion - e.g. '1.0.1'
        // • WasmModule - the compiled module
        var dbConnection = evt.target.result;
        dbConnection.createObjectStore(DB_OBJSTORE_MODULES, { keyPath: "WasmURI" }); 
      }

      function LoadWebAssemblyFromFile(dbConnection, sWasmURI, sWasmVersion) { 
        // Request the wasm file from the server and compile it...
        fetch(sWasmURI).then(response => 
          response.arrayBuffer()
        ).then(bytes =>
          WebAssembly.instantiate(bytes, g_importObject)
        ).then(results => {
          // Hold onto the module's instance so that we can reuse it
          g_objModuleInstance = results.instance;

          // Only do the following if we have a database connection object 
          // (this method will be passed a null if we failed to load the
          // module from cache due to an error when trying to open the
          // database)
          if (dbConnection !== null) { 
            // WARNING: Not all browsers that support WebAssembly also
            // support the ability to store the module in IndexedDB (seems
            // to work fine in Edge 16 and in Firefox but it doesn't work
            // for me in Chrome 63)
            try { 
              // Create the object we're about to store
              var objRecord = { "WasmURI": sWasmURI, "WasmVersion": sWasmVersion, "WasmModule": results.module };

              // Cache the compiled module so that we don't have to pull the
              // file from the server again unless we change the module's
              // version number.
              SaveRecordToObjectStore(dbConnection, DB_OBJSTORE_MODULES, objRecord); 
            }
            catch (ex) { 
              console.log(`Unable to save the WebAssembly module to IndexedDB: ${ex.message}`); 
            }
          }
        });
      }

      function OnClickTest() {
        // Call the module's add method and display the results
        var iResult = g_objModuleInstance.exports._add(1, 2);
        alert(iResult.toString()); 
      }
    </script> 
  </body>
</html>

The following is the content of our IndexDB.js file:

// Helper methods to work with an IndexedDB database
//
// Note: IndexedDB methods are asynchronous. To make things a bit easier to
// work with for the calling code, I've added Promises.

function OpenDB(sDatabaseName, sDatabaseVersion, fncUpgradeDB) { 
  return new Promise(function (fncResolve, fncReject) {

    // Make a request for the database to be opened
    var dbRequest = indexedDB.open(sDatabaseName, sDatabaseVersion);

    dbRequest.onerror = function (evt) { fncReject(`Error in OpenDB: ${evt.target.error}`); }

    // Pass the database connection object to the resolve method of the
    // promise
    dbRequest.onsuccess = function (evt) { fncResolve(evt.target.result); }

    // This event handler will only be called if we're creating the database
    // for the first time or if we're upgrading the database to a new version
    // (this will be triggered before the onsuccess event handler above if it
    // does get called). Let the calling code handle upgrading the database
    // if needed to keep this file as generic as possible.
    dbRequest.onupgradeneeded = fncUpgradeDB;

  });
}

// Helper method to simplify the code some
function GetObjectStore(dbConnection, sObjectStoreName, sTransactionMode) { 
  // Create a transation and, from the transaction, get the object store
  // object
  return dbConnection.transaction([sObjectStoreName], sTransactionMode).objectStore(sObjectStoreName); 
}

function GetRecordFromObjectStore(dbConnection, sObjectStoreName, sRecordID) {
  return new Promise(function (fncResolve, fncReject) {

    // Request the record specified
    var dbGetRequest = GetObjectStore(dbConnection, sObjectStoreName, "readonly").get(sRecordID);
    dbGetRequest.onerror = function (evt) { fncReject(`Error in GetRecordFromObjectStore: ${evt.target.error}`); }
    dbGetRequest.onsuccess = function (evt) {
      // If we have a record then...(we have to check because there won't be
      // a record if the database was just created)
      var objRecord = evt.target.result;
      if (objRecord) { fncResolve(objRecord); }
      else { fncReject(`The record '${sRecordID}' was not found in the object store '${sObjectStoreName}'`); }
    }

  });
}

function DeleteRecordFromObjectStore(dbConnection, sObjectStoreName, sRecordID) { 
  return new Promise(function (fncResolve, fncReject) {

    // Request the delete of the record specified
    var dbDeleteRequest = GetObjectStore(dbConnection, sObjectStoreName, "readwrite").delete(sRecordID);
    dbDeleteRequest.onerror = function (evt) { fncReject(`Error in DeleteRecordFromObjectStore: ${evt.target.error}`); }
    dbDeleteRequest.onsuccess = function (evt) { fncResolve(); }

  });
}

function SaveRecordToObjectStore(dbConnection, sObjectStoreName, objRecord) {
  // Request the put of our record (if it doesn't already exist, it gets
  // added. otherwise, it gets updated)
  var dbPutRequest = GetObjectStore(dbConnection, sObjectStoreName, "readwrite").put(objRecord);
  dbPutRequest.onerror = function (evt) { console.log(`Error in SaveToIndexedDB: ${evt.target.error}`); }
  dbPutRequest.onsuccess = function (evt) { console.log(`Successfully stored the record`); }
} 

The following is the C code and command line needed to turn the C code into a WebAssembly module for today's article:

int add(int x, int y) { return x + y; } 

  emcc test.c -s WASM=1 -s SIDE_MODULE=1 -O1 -o test.wasm 

Summary

Even though we tried to keep the code as clean as possible, this article was a bit more involved because of everything that is involved when working with IndexedDB databases.

Because this article was focused around WebAssembly caching, we only dug into IndexedDB as deep as was needed.

Fortunately, I had the privilege of writing a DZone Refcard on HTML5 IndexedDB a little while ago that you're more than welcome to check out if you would like more information on the technology.


Edit made on Monday, January 1st, 2018: Changed the MIME Type from application/octet-stream to application/wasm after discovering the WebAssembly meeting notes from October 23rd, 2017 proposing the MIME Type be submitted to IANA to make it official.

Deploying code to production can be filled with uncertainty. Reduce the risks, and deploy earlier and more often. Download this free guide to learn more. Brought to you in partnership with Rollbar.

Topics:
web assembly ,indexeddb ,javascript ,cache ,web dev

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}