Benchmarking Website Performance on HubSpot COS

[This article was written by Brendan MacArthur]

One of the most popular (and provocative) blogs from Yottaa was the 2013 post,Benchmarking Performance of 8 CMS Platforms: Who Is Slowest? Using Yottaa technology, we measured the performance of 8 CMS platforms (WordPress, WordPress VIP, Joomla, Drupal, Interwoven, Adobe CQ, Demandware and HubSpot) and set out to analyze the results.

We expected to find little variation among content management systems, however, the opposite turned out to be true. HubSpotemerged a clear winner, the most performant content management system in our 2013 study.

HubSpot has since (re)launched a new CMS, theContent Optimization System (COS). Of course, as performance enthusiasts, the Yottaa team was curious to see if the new COS has maintained its predecessor's high standards.

A Recap of the 2013 CMS Study

The original study revealed HubSpot CMS pages to be the lightest and the fastest in our sample. Thislightness was, in part, reflective of a CMS that was simpler than its larger, more established competitors (Wordpress, Joomla, Etc.)

The graphic comparison from 2013 (below) tells the story. In a way, the simplicity of HubSpot's CMS acted as a safeguard to prevent performance killing page bloat. Notice how HubSpot CMS pages are significantly lighter, weighing in at a .7 MB average. This simplicity enabled a more expedient delivery to end users.

A Modern User Experience

While lighter pages are faster, large assets like JavaScript and image files are requisite to meetcurrent user experience standards. The HubSpot COS was built to be a more dynamic competitor, featuring these modern web design elements and also built-in responsive design. Thus, we expected COS pages to contain many more assets than the original CMS sample. That much we could count on.

The real question became: would these changes affect real-world COS website performance?

Testing the Content Optimization System

With cooperation from HubSpot, Yottaa gathered data on a random sample of HubSpot COS websites to seek an answer. The control sample included comparative websites built on a variety of CMS platforms representative of overall market distribution.

Twenty-five URLs from different COS websites were selected for testing; sample companies sites ranged from marketing agencies to software businesses. Next, real-time browser performance tests for multiple locations across North America were run for each site. Testing included cable Internet connections (Chrome browser), as well as, 3G/4G iPhone connections.

(You can use Yottaa's free testing tool to replicate these tests for any web application)

The last step was to crunch some data. The COS results were compared to the Internet-wide testing averages established at Yottaa.

Aggregate Performance on HubSpot COS

True to our hypothesis, we found that HubSpot COS pages have gained weight, jumping from around .7 MB to 2.43 MB, a massive increase. While COS page weight may seem gigantic, this data is actually in line with current web trends. A recent report from Soasta confirms that the average web page ismore than 2 MB in size.

Despite nearly a 250% increase in weight from the 2013 CMS sample, there was no performance degradation of note on the Content Optimization System. On average, COS websites outperformed our control sample by a considerable margin. Below is the average COS Time-to-Display and Time-to-Interact metrics (in milliseconds) compared to our sample.

What About Mobile?

Mobile devices are less powerful than desktop computers can struggle to download heavy pages. Therefore, it is essential to factor in user experience considerations when implementing a mobile strategy like responsive web design. Again, we found the average asset weight for COS mobile pages quite heavy, roughly twice the size of our sample average.

Despite asset-laden mobile pages, the COS pages we tested had lower start render times and a faster time to display.

Why Does All This Matter?

Online users want two things:

1. A rich, dynamic and engaging user experience
2. they want that experience to be fast.

Ironically, the very page elements that deliver an exiting online experience can cause pages to becomeunbearably slow. A balancing act is necessary to meet both goals. Because the COS pages are much more complex, yet still performant, the conclusion is that HubSpot must be doing something to deliver rich and dynamic pages so efficiently. We asked Jeffrey Vocell, Product Marketing Manager at HubSpot, for some additional COS insights. It turns our there is no secret formula. HubSpot simply built in good performance practices with construction of the COS. Here are his examples.

• HubSpot utilizes a global Content Delivery Network (CDN) for all customer sites to quickly deliver all static assets.
• The back-end of the COS is build using Java to quickly adapt and scale for frequent traffic increases to customer sites.
• The COS is fully responsive for mobile compatible design. So regardless of your design process, the COS automatically ensures your site is optimized for mobile.
• The COS features automated image compression which retains image quality, but significantly reduces the file size which helps accelerate page loading time.

Performance is More Than a Fast CMS

When developing a user experience strategy, content management system performance is an important and often overlooked consideration. Overall, HubSpot's COS delivers a high standard for performance. The platform has done well to balance performance with the ability for web designers to create rich user experiences.

It is also important to note that delivering an exemplary experience for end users necessitates a holistic strategy. Having a fast CMS is only part of that effort. Many of the business websites we measured in the COS study are undoubtedly doing more for performance than choosing a content management system. It takes an all-inclusive, team approach to get it right.