Comparing Healthcare.gov and JCrew.com Performance

Each week, we measure the website home page performance of about 30 consumer sites using our Browser Synthetic Monitoring (Beta) tool. A summary of the results with the complete rankings and data for each of the sites is published each week by MobileStrategies360.com.

It’s incredibly interesting to see how these sites change over time and how those changes affect the performance of the site. This week’s results show an interesting contrast in the changes between performance improvements in the HealthCare.gov home page and a slow down at JCrew.com.

When the HealthCare.gov site launched a couple of years ago, it rightly took a lot of criticism for the overall poor performance of the site. This week, at least the home page made significant improvement, moving up eight spots in the weekly ranking from 18th to 10th with an improvement in the speed score, mostly by optimizing the size to reduce the page load size by over half a megabyte (as seen in the graph below).

There was also a modest reduction in the number of resources loaded on average by six (each additional resource loaded carries a distinct “resource tax” for the performance of the site). The reduction in the page load size had immediate effect on improving the speed of the visually complete time, the first render time, and the overall speed score of the page (as seen in the graph below).

This represents a standard method of improving site performance by optimizing your content to make it as compact as possible to reduce the amount of data that needs to be transmitted, resulting in faster page load times.

The second interesting movement in this week’s data is that J. Crew went from the fourth position in 3G to eighth almost entirely due to an increase in visually complete time from 8.1 to 9.7 seconds.

This diagram shows the sudden increase in the visually complete time mid-week while the first render and speed score stayed fairly consistent.

The degradation in performance occurred despite the fact that the page weight decreased from 1.5 to 0.89 MB. The elements loaded remained constant and the first render time and speed score were consistent at 5.1 and approximately 5.6, respectively. The diagram below shows the decrease in the page weight over the course of the week.

This visual represents a bit of a conundrum since you would normally expect that a fairly reasonable decrease in total page weight of 600 KB would result in a performance improvement.

If we investigate further, examining individual snapshots from before and after, one of the causes could be an increase in the response available time of the server. It includes the server connection time and the time it takes the server to respond with the first byte of data.

Looking at the two timing diagrams below, we can see that there is a significant increase in the J. Crew server available response time:

Looking at this snapshot from earlier in the week, the server response available time is on the order of one second.

The server response available time in this snapshot from later in the week is closer to 12 seconds.

Similarly, if we look at the individual resource timing from earlier in the week versus later in the week, the following two diagrams show a large increase in the time it takes to create the J. Crew index.jsp (Java Server) page–the main container page for the homepage content.

This resource waterfall timing diagram shows the index.jsp page was taking about 1.7 seconds to download.

This resource waterfall diagram shows the index.jsp of the J. Crew homepage taking about 14 seconds to download.

While these particular snapshots show the extreme changes over the week, they are generally indicative of the trend that occurred over the course of the week that contributed to the increase in visually complete time, which resulted in J. Crew moving down in the index.

Without instrumenting the backend of the website, it can’t be known what might have contributed to the increased time it takes to create the index.jsp. That could be due to a change in the logic used to create the page or some other design change, which resulted in the change. It is critical for companies to understand how changes in website design, architecture, or infrastructure changes can affect the performance of their websites, especially as perceived by the end user.