Top Shelf vs. Watered-Down RUM

Last week, we asked our readers to sign a petition requesting Apple to add the Navigation Timing specification on Safari browsers. This week, we are detailing the two different methods for Real User Measurement and why using Navigation Timing is crucial for accurate metrics.

Top Shelf vs. Watered-down RUM

Real User Measurement (RUM) is a method of capturing performance metrics from actual people browsing a webpage via JavaScript. Other information such as engagement, bandwidth and demographic stats can be captured as well to correlate performance to engagement and sales.

Currently, there are two methods of tracking performance: The preferred method is to rely on Navigation Timing API, and the less effective method is to rely on timestamps during key browser actions.

Navigation Timing

The ideal approach for gathering timing data is to utilize the Navigation Timing API. Browsers supporting the Navigation Timing API provide start- and end-times of various events that occur during the loading and rendering of the webpage:

Using these timings in various calculations, the Navigation Timing API provides the values of the following metrics: DNS, Connect (TCP Handshake), SSL (not all browsers, it was left optional in the spec), Wait (first byte), Load (first byte to last byte), Response, DOM Interactive, DOM Loaded, Content Load and Document Complete.

The metrics not only provide better understanding of the page performance, but also provide insight into what impacted user experience. For instance, one could relate high bounce rates to slow page load due to long processing on server-side. With the addition of new Resource Timing Specification (currently in the works and supported by IE 10, IE on Windows Mobile 8, and Chrome 28), we will be able to gather more insight on the impact of various vendors referenced on the webpage (CDNs, ads, widgets, tracking pixels).

Unfortunately, Safari does not yet support the Navigation Timing API so we have to rely on an older technique based on heuristics.

Heuristic (or RUM on Browsers Without Navigation Timing)

This method relies on capturing a timestamp of when the page starts rendering, a timestamp when page fires the “onload” event, and storing a timestamp on a cookie when the browser unloads the page (when the user leaves the page).

Here is a diagram of this approach and the metrics captured by it:

This method’s major shortcoming is that the first time the user accesses a webpage of the site, there is no way to measure how long it takes to make the HTTP request to the server. This means that, for bounce visits (visits with one page view), we cannot determine if the server was too slow and that’s why the user left.

The second problem is the timer for the “Entry” point starts when the browser starts rendering the webpage and requires placing the timer code as the first thing in the section of the HTML.

Needless to say, this method is spotty and does not deliver insight into what the page performance was and what impacted it.

Summary

While the heuristic approach to Real User Measurements provides useful data, it is very limited compared to the data provided by the Navigation Timing API. It is difficult to get any insight as to what specifically might be impacting performance. Using the Navigation Timing spec is the only way to get accurate metrics from end users.

Please sign the petition and ask Apple to include Navigation Timing!