Performance as Craft

Last week my good friend Jonathan Klein at Etsy published a blog about the latest performance achievements at Etsy.

I have always been a big fan of Etsy’s IT team, they are very transparent about their performance achievements and their development process. Jonathan drove the same kind of transparency and knowledge sharing at Wayfair. When he joined Etsy, it was a match made in “Web Performance Heaven”.

Jonathan’s data was very impressive, so I decided to take a look at it myself from outside their organization. I like to keep an eye on my favorite companies through Catchpoint and see how their performance evolves over time.

The Improvement:

Etsy’s improvements are quite amazing, and clearly benefits their end users. You can see a big change on September 25:

Performance metrics – Median – ms

This performance accomplishment clearly shows that in order to achieve great web performance, you have to focus not only Front End development, but also on hardware, network, and even datacenter location. (See Web Performance Pixie Dust.)

Etsy’s hardware upgrade lowered the “Wait” time (or First Byte Time) variability that occurred by “hour of day” and “day of week” .  It made their system more resilient and less prone to worst performance during busy hours.

Pre – Post Upgrade Wait Time by Hour of Day

Pre – Post Upgrade Wait Time by Day of Week

Lessons learned:

• Performance Management is a craft. You need dedicated folks to look at the data and act on it – whether is code changes, infrastructure changes, or new architecture. It’s not a second job for anyone. Companies serious about performance must invest the time, effort, and money into it.
• Performance is the result of HARD work, hard TEAM work! Blood and sweat must be invested not only in development or infrastructure, but to all aspects of your website or web service. You also must find a balance between code and infrastructure investments, just throwing more servers at a problem is not always the best solution, neither is squeezing 2 more milliseconds out of an overloaded server.
• Fast Performance is a Journey, not a Destination. You must be diligent about staying on top of new technology as there is always room for improvement (and if you don’t do it your competitors will).

Kuddos to the Etsy team!

Mehdi – Catchpoint