Real-World Performance and the Future of JavaScript Benchmarking

DZone 's Guide to

Real-World Performance and the Future of JavaScript Benchmarking

Web workloads are changing. Performance metrics and tooling need to adapt. Limiting the amount of JS proportionally to what’s visible on the screen is a good strategy.

· Performance Zone ·
Free Resource

This article is featured in the new DZone Guide to Performance: Optimization & Monitoring. Get your free copy for more insightful articles, industry statistics, and more!

In the last 10 years, an incredible amount of resources went into speeding up peak performance of JavaScript engines. This was mostly driven by peak performance benchmarks like SunSpider and Octane and shifted a lot of focus toward the sophisticated optimizing compilers found in modern JavaScript engines like Crankshaft in Chrome.

This drove JavaScript peak performance to incredible heights in the last two years, but at the same time, we neglected other aspects of performance like page load time, and we noticed that it became ever more difficult for developers to stay on the fine line of great performance. In addition to that, despite all of these resources dedicated to performance, the user experience on the web seemed to get worse over time — especially page load time on low-end devices.

This was a strong indicator that our benchmarks were no longer a reasonable proxy for the modern web but rather turned into a caricature of reality. Looking at Google’s Octane benchmark we see that it spends over 70% of the overall execution time running JavaScript code.

Image title

Comparing this to profiles we see during startup of some 25 top web pages, we see that those are nowhere near the 70% JavaScript execution of Octane. They obviously spend a lot of time in Blink doing layouting and rendering, but also spend a significant amount of time in parsing and compiling JavaScript.

Image title

On average, the time spent in executing JavaScript is roughly 20%, but more than 40% of the time is spent in just parsing IC (inline cache) Miss and V8 C++ (the latter of which represents the subsystems necessary to support the actual JavaScript execution and the slow paths for certain operations that are not optimized in V8). Optimizing for Octane might not provide a lot of benefit for the web. In fact, parsing and compiling large chunks of JavaScript is one of the main problems for startup of many web pages nowadays, and Octane is a really bad proxy for that.

There’s another benchmark suite named Speedometer, that was created by Apple in 2014, which shows a profile that is closer to what actual web pages look like. The benchmark consists of the popular TodoMVC application implemented in various web frameworks (i.e., React, Ember, and AngularJS).

Image title

As shown in the profile, the Speedometer benchmark is already a lot closer to what actual web page profiles look like, yet it’s still not perfect — it doesn’t take into account parse time for the score, and it creates 100 todos within a few milliseconds, which is not how a user interacts with a webpage, usually. V8’s strategy for measuring performance improvements and identifying bottlenecks thus changed from using mostly traditional JavaScript benchmark methods toward using browser benchmarks like Speedometer and also tracking the real-world performance of web pages.

Performance GuideFind this article and much more in...

DZone's Guide to Performance: Optimization and Monitoring


  • Survey findings from over 500 developer responses

  • Articles written by top Performance experts

  • "What Ails Your Application" Infographic

  • Directory of performance optimization and monitoring tools

Download for FREE

What’s interesting to developers in light of these findings is that the traditional way of deciding whether to use a certain language feature by putting it into some kind of benchmark and running it locally or via some system like jsperf.com might not be ideal for measuring real-world performance. When following this route, it’s possible for the developer to fall into the microbenchmark trap and observe mostly the raw JavaScript execution speedup, without seeing the real overhead cumulated by the other subsystems of the JavaScript engine (i.e., parsing, inline caches, slow paths triggered by other parts of the application, etc.) that negatively affect a web page’s performance. At Chrome, we have been making a lot of the tooling that supported our findings available to developers via the Chrome Developer Tools.

Image title

You can now see parsing and compile buckets in the profiler. And, over the last few years, we’ve introduced another mechanism — called chrome://tracing — which allows you to record traces that collect all kinds of events. For example, you can analyze in detail how much time V8 spends in the different parsing steps, and thereby understand whether it might make sense to consider using a tool like optimize-js to mitigate the overhead of pre-parsing when it’s not beneficial, for example, the function is executed immediately anyway.

Image title

Chrome Tracing provides you with a pretty detailed understanding of what’s going on performance-wise by offering a view into the less obvious places. V8 has a step-by-step guide on how to use this. For most use cases, though, I’d recommend sticking to the Developer Tools, because they offer a more familiar interface and don’t expose an overwhelming amount of the Chrome/V8 internals. But for advanced developers, chrome://tracingmight be the swiss army knife that they were looking for.

Looking at the web today, we’ve discovered that it is important to significantly reduce the amount of JavaScript that is shipped to the browser, as we live in a world where more and more users consume the web via mobile devices that are a lot less powerful than a desktop computer and might not even have 3G connectivity.

One key observation is that most web developers use ECMAScript 2015 or later for their daily coding already, but for backward compatibility compile all their programs to traditional ECMAScript 5 with so-called transpilers, likeBabel, for example. This can have an unexpected impact on the performance of your application because often the transpilers are not tuned to generate high-performance code. Thus the final code that is shipped might be less efficient than the original code. But there’s also the increase in code size due to transpilers: The generated code is usually 200-1000% the size of the original code, which means the JavaScript engine has up to 10 times the work in parsing, compiling, and executing your code.

Since not all browsers support all new language features, there’s a certain period of time where new features require transpilation. But if you are building a web application today, consider shipping as much of the original code as possible. An Intranet application with dedicated clients inside the company, all using some recent browser version, could as well be written and shipped as ES2015 code.

A good rule of thumb currently is to ship amounts of JavaScript proportionally to what’s on the screen. Think about code splitting from the beginning and design your web application with progressive enhancement in mind whenever possible. And independent of what kind of application you are developing, try to be as declarative as possible, using appropriate algorithms and data structures; i.e., if you need a map, use a Map. If it turns out to be slow ina certain browser, file a bug report. Focus optimization work on bottlenecks identified via profiling.

This article is featured in the new DZone Guide to Performance: Optimization & Monitoring. Get your free copy for more insightful articles, industry statistics, and more!

benchmarking ,javascript ,performance

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}