We are all well aware of the building blocks of the modern website/web app:
- HTML defines the structure and textual content of the page
- CSS states the page look and feel.
- Other static resources like images, fonts, videos, etc. enrich overall user experience.
These, however, are also the integral parts of browser extensions, hybrid mobile applications (e.g. Cordova-based ones), as well as Electron-based desktop applications (or alternatives to the Electron framework).
A browser extension can be authored using web technologies too. Chrome extensions, for example, are just simple web apps that run in a little page that appears when the user clicks the icon. The only difference is that an extension requires a manifest file to define the structure, permissions, icons and other metadata. (Check out Aidan Breen’s blog post on building Chrome extensions.)
Challenges Cross-Platform Tools Bring to Tracking User Behavior
Session replay tools "record" applications by listening for changes in the DOM (see how we do it at SessionStack by using MutationObserver) as well as for all user events like mouse clicks, mouse moves, page scrolls, window resizing, and so on. All static resources referenced from the page that’s been recorded need to be accessed by the third party tool so that it’s able to offer a pixel-perfect replay of the session. The collected data is used by the tool to recreate every single user activity on the page in the form of a video recording.
As you can tell, a common trait across the above development platforms and frameworks is the way static resources such as imagery, fonts, multimedia, etc. get served in a closed environment, where there’s an intermediary between the browser and the application, be it a WebView or a Manifest file. In essence, all static files are embedded in the application and loaded as files that live on a local server which makes it impossible for 3rd party tools to access them from outside the network.
This is what you’d be hoping to see on user session replay.
That’s how a session replay would look if static files have not been loaded (and recorded).
And that’s precisely why most session replay tools hit a wall trying to record and replay user activities on mobile hybrid and desktop applications, or capture browser extension usage.
A solution for this problem would be to serve the resources referenced from the app on a remote server which can be accessed externally. Also, the session recording tool should provide a way to change the original resource addresses to point to the remotely accessible server. For example, the recording tool might have the ability to map the original URLs to different ones based on some rules defined by the user.
Another option would be if the tool allows changing the server origin of the app. This method would also require that all of the URLs in the app are relative and the resources on the remote server keep the same structure as in the app. The latter method is the one we employ to make sure SessionStack is able to record and replay user sessions in hybrid mobile and desktop apps, as well as browser add-ons:
Specify a new location for your app’s static resources to allow SessionStack to reproduce user behavior in your hybrid mobile and desktop apps.