Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

WebAssembly: Caching When Using Emscripten

DZone's Guide to

WebAssembly: Caching When Using Emscripten

Welcome back! In today's article, we're going to look at caching a WebAssembly module to an HTML5 IndexedDB database, using Emscripten and its built-in framework.

· Web Dev Zone ·
Free Resource

Bugsnag monitors application stability, so you can make data-driven decisions on whether you should be building new features, or fixing bugs. Learn more.

Update: September 1, 2018

Originally, the specification for WebAssembly called for explicit caching of a compiled WebAssembly module to HTML5 IndexedDB.

Firefox and Edge added support for serialization to IndexedDB but, after some discussion between the WebAssembly Community Group, browser makers, and others, it has been decided that it will be best for browsers to implicitly cache WebAssembly modules instead.

As a result of this decision, Firefox 63, which will ship on October 23rd, 2018 will no longer allow WebAssembly modules to be cached to IndexedDB (https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/63)

The content of this article should not be used and any explicit caching already in place in your code should be removed.


In our first article about WebAssembly, An Introduction to WebAssembly, we allowed Emscripten to handle all of the plumbing for us (including the HTML) so that we could get a feel for what was possible with WebAssembly modules by jumping right into some code.

In some of the articles that followed, we examined some of the technologies related to WebAssembly and used a bare-bones module (no Emscripten plumbing) to do so.

In this article, we're going to circle back to caching a WebAssembly module to an HTML5 IndexedDB database, but, this time, we're going to enable some of Emscripten's plumbing to show how you can implement caching within Emscripten's framework.

Note: We're not going to dig into some of the details of using Emscripten or working with an IndexedDB database because we've talked about them in previous articles. We will have links to our other WebAssembly articles at the end of this article if you would like to explore the other topics more.

Building the WebAssembly Module and JavaScript Plumbing Code

The following is some C code that we will compile into a .wasm file for today's article:

#include <stdio.h>
#include "../emscripten/emscripten.h"

int main()
{ 
  printf("Hello World from C\n");
  return 0; 
}

void EMSCRIPTEN_KEEPALIVE TestFunction(int iVal)
{ 
  printf("TestFunction called...value passed in was: %i\n", iVal); 
} 

The following command line is similar to the one we used in our Introduction to WebAssembly article but with one change. Here, we're specifying the output file to have a .js extension rather than .html. This tells Emscripten to only generate the JavaScript plumbing code and not the HTML file.

emcc test.c -s WASM=1 -o test.js -s NO_EXIT_RUNTIME=1

Controlling the Module's Instantiation

Emscripten allows you to override methods of the Module object in order to control execution (e.g. you can implement Module.print so that you could display stdout messages to an alert or a div rather than the default which writes the messages to the console).

Documentation of the Module methods that can be overridden can be found here.

In order to control the load of the WebAssembly module, the method we're interested in is instantiateWasm.

The instantiateWasm method accepts two parameters:

  • The first parameter is the import object that we need to pass as the second parameter to WebAssembly.instantiate.
  • The second parameter is a call back function that we need to use to pass the WebAssembly module instance back to Emscripten once we have it.

The following is an example of implementing the instantiateWasm method:

var Module = {

  // Override instantiateWasm
  'instantiateWasm': function (importObject, fncReceiveInstance) { 
    // We've set things up in such a way so that each wasm file can have a version and we
    // only clear the items from the cache if the version doesn't match
    var sWasmURI = "test.wasm";
    var sWasmVersion = "1.0.0";

    // Load the wasm file from cache if it's there. If not, fetch it and cache it.
    GetCompiledModuleFromIndexedDB(sWasmURI, sWasmVersion, importObject, fncReceiveInstance);

    // Pass back an empty dictionary object if the instantiation is performed
    // asynchronously which it is in this case
    return {}; 
  }

}; 

The final piece of the puzzle to get everything working is to include the JavaScript file that was generated by Emscripten. The trick to this, however, is that the JavaScript file needs to be included after our code that implements the instantiateWasm method.

The following is our example HTML page:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
  </head>
  <body>
    <input type="text" id="txtValue" />
    <input type="button" value="Test" onclick="OnClickTest();" />

    <script type="text/javascript"> 
      // If we need to change the structure of the database, we can increment the
      // DB_VERSION value to trigger the onupgradeneeded event when opening the 
      // database
      var DB_VERSION = 1;
      var DB_NAME = "WasmCache";
      var DB_OBJSTORE_MODULES = "Modules";

      var Module = {

        // Override instantiateWasm
        'instantiateWasm': function (importObject, fncReceiveInstance) { 
          // We've set things up in such a way so that each wasm file can have a
          // version and we only clear the items from the cache if the version
          // doesn't match
          var sWasmURI = "test.wasm";
          var sWasmVersion = "1.0.0";

          // Load the wasm file from cache if it's there. If not, fetch it and
          // cache it.
          GetCompiledModuleFromIndexedDB(sWasmURI, sWasmVersion, importObject, fncReceiveInstance);

          // Pass back an empty dictionary object if the instantiation is
          // performed asynchronously which it is in this case
          return {};
        }

      };

      function GetCompiledModuleFromIndexedDB(sWasmURI, sWasmVersion, importObject, fncReceiveInstance) {

        // If we successfully opened the database then...
        OpenDB(DB_NAME, DB_VERSION, HandleUpgradeDB).then(dbConnection => {

          // If we successfully obtained the requested record then...
          GetRecordFromObjectStore(dbConnection, DB_OBJSTORE_MODULES, sWasmURI).then(objRecord => {

            // If the version stored for this module doesn't match the version we
            // need then the module cached is out of date...
            if (objRecord.WasmVersion !== sWasmVersion) {

              // Have the record deleted and then fetch the proper file
              DeleteRecordFromObjectStore(dbConnection, DB_OBJSTORE_MODULES, sWasmURI).then(result => { 
                LoadWebAssemblyFromFile(dbConnection, sWasmURI, sWasmVersion, importObject, fncReceiveInstance);
              });

            }
            else { // The cached record is the version we need...

              // Have the module instantiated.
              //
              // NOTE: Unlike when we pass in the bytes to instantiate in the 
              // LoadWebAssemblyFromFile method below, we don't have a separate 
              // 'instance' and 'modules' object returned in this case since we 
              // started out with the module object. We're only passed back the 
              // instance in this case.
              WebAssembly.instantiate(objRecord.WasmModule, importObject).then(instance => 
                // Pass the instance back to the emscripten receiveInstance method
                fncReceiveInstance(instance)
              );

            }

          }, sErrorMsg => { // Error in GetRecordFromObjectStore...

            // We weren't able to pull the module from cache (most likely because 
            // it doesn't exist yet - hasn't been cached yet). Log the error and 
            // then fetch the file.
            console.log(sErrorMsg);
            LoadWebAssemblyFromFile(dbConnection, sWasmURI, sWasmVersion, importObject, fncReceiveInstance);

          });

        }, sErrorMsg => { // Error in OpenDB...

          // Log the error and then fetch the file (won't be able to cache it in
          // this case because we don't have a database connection to work with)
          console.log(sErrorMsg);
          LoadWebAssemblyFromFile(null, sWasmURI, sWasmVersion, importObject, fncReceiveInstance);

        });

      }

      // Called by indexeddb if the database was just created or if the database
      // version was changed
      function HandleUpgradeDB(evt) { 
        // Create the object store which will hold 3 properties:
        // • WasmURI - (primary key) e.g. 'test.wasm'
        // • WasmVersion - e.g. '1.0.1'
        // • WasmModule - the compiled module
        var dbConnection = evt.target.result;
        dbConnection.createObjectStore(DB_OBJSTORE_MODULES, { keyPath: "WasmURI" }); 
      }

      function LoadWebAssemblyFromFile(dbConnection, sWasmURI, sWasmVersion, importObject, fncReceiveInstance) { 
        // Request the wasm file from the server and compile it...
        fetch(sWasmURI).then(response => 
          response.arrayBuffer()
        ).then(bytes =>
          WebAssembly.instantiate(bytes, importObject) 
        ).then(results => { 
          // Pass the instance back to the emscripten receiveInstance method
          fncReceiveInstance(results.instance);


          // Only do the following if we have a database connection object (this
          // method will be passed a null if we failed to load the module from
          // cache due to an error when trying to open the database)
          if (dbConnection !== null) { 
            // WARNING: Not all browsers that support WebAssembly also support the
            // ability to store the module in IndexedDB (seems to work fine in Edge
            // 16 and in Firefox but it doesn't work for me in Chrome 63)
            try { 
              // Create the object we're about to store
              var objRecord = { "WasmURI": sWasmURI, "WasmVersion": sWasmVersion, "WasmModule": results.module };

              // Cache the compiled module so that we don't have to pull the file
              // from the server again unless we change the module's version
              // number.
              SaveRecordToObjectStore(dbConnection, DB_OBJSTORE_MODULES, objRecord); 
            }
            catch (ex) { 
              console.log(`Unable to save the WebAssembly module to IndexedDB: ${ex.message}`);
            } 
          }
        });
      }

      function OnClickTest() { 
        Module.ccall('TestFunction', // name of C function 
                     null, //return type
                     ['number'],//argument types
                     [ parseInt(document.getElementById("txtValue").value,10) ]);
      }
    </script>
    <script src="IndexedDB.js"></script>
    <script src="test.js"></script><!-- Emscripten plumbing. Very important that this comes *after* our JavaScript that creates the Module object and sets the 'instantiateWasm' method. -->
  </body> 
</html> 

The following is the content of our IndexDB.js file:

// Helper methods to work with an IndexedDB database
//
// Note: IndexedDB methods are asynchronous. To make things a bit easier to
// work with for the calling code, I've added Promises.
function OpenDB(sDatabaseName, sDatabaseVersion, fncUpgradeDB) { 
  return new Promise(function (fncResolve, fncReject) {

    // Make a request for the database to be opened
    var dbRequest = indexedDB.open(sDatabaseName, sDatabaseVersion);
    dbRequest.onerror = function (evt) { fncReject(`Error in OpenDB: ${evt.target.error}`); }

    // Pass the database connection object to the resolve method of the
    // promise
    dbRequest.onsuccess = function (evt) { fncResolve(evt.target.result); }

    // This event handler will only be called if we're creating the database
    // for the first time or if we're upgrading the database to a new
    // version (this will be triggered before the onsuccess event handler
    // above if it does get called). Let the calling code handle upgrading
    // the database if needed to keep this file as generic as possible.
    dbRequest.onupgradeneeded = fncUpgradeDB;

  });
}

// Helper method to simplify the code some
function GetObjectStore(dbConnection, sObjectStoreName, sTransactionMode) { 
  // Create a transation and, from the transaction, get the object store
  // object
  return dbConnection.transaction([sObjectStoreName], sTransactionMode).objectStore(sObjectStoreName);
}

function GetRecordFromObjectStore(dbConnection, sObjectStoreName, sRecordID) {
  return new Promise(function (fncResolve, fncReject) {

    // Request the record specified
    var dbGetRequest = GetObjectStore(dbConnection, sObjectStoreName, "readonly").get(sRecordID);
    dbGetRequest.onerror = function (evt) { fncReject(`Error in GetRecordFromObjectStore: ${evt.target.error}`); }
    dbGetRequest.onsuccess = function (evt) { 
      // If we have a record then...(we have to check because there won't be
      // a record if the database was just created)
      var objRecord = evt.target.result;
      if (objRecord) { fncResolve(objRecord); }
      else { fncReject(`The record '${sRecordID}' was not found in the object store '${sObjectStoreName}'`); }
    }

  });
}

function DeleteRecordFromObjectStore(dbConnection, sObjectStoreName, sRecordID) { 
  return new Promise(function (fncResolve, fncReject) {

    // Request the delete of the record specified
    var dbDeleteRequest = GetObjectStore(dbConnection, sObjectStoreName, "readwrite").delete(sRecordID);
    dbDeleteRequest.onerror = function (evt) { fncReject(`Error in DeleteRecordFromObjectStore: ${evt.target.error}`); }
    dbDeleteRequest.onsuccess = function (evt) { fncResolve(); }

  }); 
}

function SaveRecordToObjectStore(dbConnection, sObjectStoreName, objRecord) {
  // Request the put of our record (if it doesn't already exist, it gets 
  // added. otherwise, it gets updated)
  var dbPutRequest = GetObjectStore(dbConnection, sObjectStoreName, "readwrite").put(objRecord);
  dbPutRequest.onerror = function (evt) { console.log(`Error in SaveToIndexedDB: ${evt.target.error}`); }
  dbPutRequest.onsuccess = function (evt) { console.log(`Successfully stored the record`); }
} 

Since we have already discussed some of Emscripten's basics, like the EMSCRIPTEN_KEEPALIVE attribute in the C code above, as well as caching WebAssembly modules to HTML5 IndexedDB databases in previous articles, we didn't explain those details today.

If you'd like to know more about WebAssembly, the following are a few articles that might be of interest:

Monitor application stability with Bugsnag to decide if your engineering team should be building new features on your roadmap or fixing bugs to stabilize your application.Try it free.

Topics:
web assembly ,emscripten ,wasm ,javascript ,web dev

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}