Mobile UX, the Real 2015 Internet Trend

In her annual report for KPCB back in May, Mary Meeker cited a number of trends, statistics and metrics that show the increasing importance of online activity such as the growth in internet users to 39% of the population globally, the growth of mobile phone users to 73% penetration of the population globally (40% of that being on smartphones), the growth in mobile data traffic up 69% in the past year, the time adult users in the US spend online per day (mobile up 51%), and the implications of all of those various statistics for many businesses.

But in all of her 196 slides, the main thing she failed utterly to report on is how all of those metrics and business are affected by the performance of those online business and apps.

The problem is that it’s easy to take performance for granted, but if you look at all of the business metrics that Meeker cites, whether its messaging/notifications, user generated content (Pinterest Pins, Snapchat/Facebook Video uploads, live gaming streams, audio remixes etc.), just in-time products and services, and every other category of commerce, the amount and velocity of activity is directly affected by the performance of the sites and apps. In fact, performance is so critical to these statistics that Facebook explicitly cited it as the reason for re-architecting their mobile applications from hybrid HTML5 to native iOS and Android and the resulting improvements in their user engagement metrics they achieved due to the better performance.

But if there is one industry where the connection between performance and business results should be abundantly obvious, that has to be in retail. If your website is slow to load, especially on mobile devices, then customers will do less business with you or they may even abandon you completely.

This summer, AppDynamics teamed up with Internet Retailer to measure the homepage mobile performance at 3G and 4G network speeds of the top 500 retailers globally based on their revenue. IR provided AppDynamics with the URLs to measure, and we used our Browser Synthetic Monitoring product (now in beta) to measure the performance of the websites and record the results using our unique Speed Index and Visually Complete metrics in addition to a host of other values including the resources loaded per page, the first render times, and the complete size of the page (or page weight) in MegaBytes.

The results of these measurements are published as part of Internet Retailer’s 2016 Mobile 500reference guide that was published this month both online and in print. In addition to industry trends and analysis, the database version of this guide provides a summary of key data about each of the 500 retailers, including financial, operational, and corporate summaries, and a snapshot of mobile site performance as measured by AppDynamics. The websites were measured at 30-minute intervals over the course of a week from locations in North America, Europe, and Asia with the Chrome browser and mobile network profiles.

Result Trends

What the measurements show is that there is a very broad spread in retail website performance. As you might expect, the best websites were visually complete in the 2-3 second range on 3G network profiles and 1-2 seconds on on 4G network profiles. At the extreme opposite end of the spectrum were sites that had average visually complete times on the order of 30, 40, and even close to 50 seconds, at both 3G and 4G network profile speeds. Needless to say, the likelihood that customers will be sticking around that long for a page to load starts to drops off dramatically. Studies by companies such as Google have shown that introducing delays of as little as 500 milliseconds into search results can start to have a seriously negative effect on revenue.

Strategies for Improving Website Performance

Among the better performing websites, there are three strategies that website operators can take to try to improve their site’s performance.

The first approach is to try to keep the size of the web page in general as small as possible. In general the data suggest that each megabyte of page size can cost you about six seconds of visually complete time at 3G network profile speeds. The best websites are able to do a bit better than that, but, the less well optimized sites can be way slower. For example, one athletic clothing retailer’s homepage had a page weight of 3.2 MB, yet a visually complete time of more than 34 seconds, or 10 seconds of visually complete time for each MB of page content.

Of course, keeping the page size small is particularly challenging for the retail sector because the natural tendency is to want to fill the page with lots of interesting and compelling product images or a fancy UI/UX experience to appeal to the consumer. But companies need to be very disciplined about understanding the performance impact of their graphics and UXD choices or they can quickly bloat the size of their pages, slowing them down, turning customers off due to the poor performance and thereby wasting all of the time, money, and effort they put into trying to attract customers to their site in the first place.

The second approach is to try to minimize the number the number of resources loaded per page view. The data shows that each resource loaded can carry a “performance” tax of anywhere from .05 seconds to .2 seconds, but should average around 0.1 seconds for the faster websites. website owners to be especially careful when it comes to loading third party resources as these can introduce significant delays. Again, the tradeoff is that third party resources can provide content or services that would be impractical or inefficient for the website itself to own (such as social media content and images) or which are tied to revenue generating strategies of the site such as advertising networks, but if they slow the site down to the point where users are abandoning the page, then it defeats the purpose of incorporating the resource.

Consider for example the 3G data for a popular retailer of video games whose home page had 248 resources, a page weight of 4.13 MB, and took just under 30 seconds to be visually complete due to the time it took to download all the images of games whereas another retailer of sports equipment had a homepage with a similar total page weight of 4.12 MB, but only 140 resources and a visually complete time about one third (10.3 seconds) of the video game retailer.

Most browsers only have eight independent simultaneous threads that they can use to download resources, so when your web page design has lots and lots of resources, the browser can get stuck cycling through its thread pool repeatedly, causing it to bog down and take longer than a site of similar size but with fewer resources to load.

A third approach is to carefully manage and optimize the order in which content is loaded on the site to provide the user with the best perceived experience of the website. This is where the speed index measurement of the site comes in as it provides a relative measure of performance that can differentiate among websites with similar visually complete times to show which site will be perceived by the customer to have better performance. It accomplishes this by measure the ratio of percentage of content delivered over time. For example, consider two website with the same visually complete time, yet website A renders 90% of its content in the first 10% of the visually complete time, while the website B only renders 10% of it’s content in the first 90% of the visually complete time and the last 90% of the content in the last 10% of the visually complete time. Website A will have a much faster speed index score since the user will perceive the website performance to be much better since they can see most of the content very quickly, whereas website B will be perceived to perform much worse and will have a much slower speed index score because the user had to wait so long to see most of the content even though both pages had the same visually complete time.

Looking at the 4G data, for example we see a popular furniture retailer with a visually complete time of 10.6 seconds and a speed index of 2.9 seconds compared to a another furniture retailer with a similar visually complete time of 10.5 seconds yet a speed score of 6.6 seconds. They both have similar visually complete times, but the users will perceive the former retailer’s website to be more than twice as performant than the latter retailer’s website.

Summary

Retailers face a host of contradictory requirements in designing and developing their sites to optimize their revenues and other business KPIs. To achieve the best results, retailers need to carefully consider how their design decisions affect the performance of their websites. Poor design decisions or ill-conceived trade-offs can result in web pages that can be very slow to load, causing today’s impatient consumers to abandon them.

To optimize their design process, retailers should consider using a Browser Synthetic Monitoring solution to see how their decisions and trade-offs are affecting their site performance and availability independently of all of the vagaries that can be associated with real-user performance monitoring.

By using Browser Synthetic Monitoring, companies can get a consistent baseline measure of their sites actual and end-user perceived performance to more accurately assess the impact of different design and implementation approaches.

Furthermore, by combining both Browser Synthetic and Browser Real-User Monitoring, companies can get a comprehensive view of their customers real and perceived experience to optimize their business goals, KPIs, and objectives.

Image showing a snapshot of some measurements from the IR Mobile 500 report.