Over a million developers have joined DZone.

How the NYT Was Up When It Was Down

DZone's Guide to

How the NYT Was Up When It Was Down

· DevOps Zone
Free Resource

Learn more about how CareerBuilder was able to resolve customer issues 5x faster by using Scalyr, the fastest log management tool on the market. 


Yesterday, the big web news story of the day was the NYT outage.  But the bigger performance story of the day was not the fact the NYT was down for a few hours, but rather, how the NYT team did all the right things to handle the outage.

NYT Avail

On our end, Catchpoint synthetic agents started recording 503 server response errors from http://www.nytimes.com at around 11:14 AM EST indicating that the web application was not functioning properly.  During the downtime, there were timeouts/reset errors from across the US, from Washington D.C. to Dallas to San Francisco.  The homepage was back up and running by about 1:38 PM EST.  A post hoc analysis deemed the root cause of the failure to be an issue during internal maintenance.

You Can’t Stop the Presses

Downtime happens and it cannot be avoided.  To make the best out of a bad situation, I always recommend to communicate with your customers throughout the outage and try to be as creative as possible to keep business running – and that is exactly what the NYT did.   They kept their readers informed, not only using social media as a temporary platform to keep delivering the news but also to stay in touch and let everyone know they were working hard to resolve the issue.

NYT FBI commend the team for their swift action and grace during the downtime. From my time at DoubleClick, I know personally how stressful it can be — the office turns into a war room with everyone trying to triage and resolve the issue as fast as possible.  The NYT set an example yesterday that everyone should follow.  They deserve a pat on the back and a nice cold beer.

I bow to the Grey Lady.

Mehdi – Catchpoint

Find out more about how Scalyr built a proprietary database that does not use text indexing for their log management tool.


Published at DZone with permission of Mehdi Daoudi, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}