Yahoo!: From Continuous Debacle to Continuous Delivery

Several years ago, continuous delivery was a new concept being tried by a small group of early adopters. Things have changed. Now, continuous delivery, or CD, is a practice that companies big and small are not only trying out, but embracing as part of a new, faster moving, on-demand business culture.

We’re now hearing the term described often as being a “table stakes” necessity that companies need to pursue to compete in their particular field. Companies that ante up have an advantage. Companies that don’t, fall behind. Continuous Delivery is now seen as the first stepping-stone of a DevOps transformation.

You’re not going to find many advertising and data platforms that are busier and more complex than Yahoo! Ad Exchange. The media titan’s platforms are backed by a massively distributed system that processes over 100 billion events each day. The system consists of hundreds of unique software components and thousands of servers. Hundreds of programmers are working in more than a dozen languages, on different teams with different priorities and schedules.

Today, the system hums – thanks to a process overhaul driven by a move to continuous delivery. The Ad Exchange team generates more than 8,000 builds a day, committing code to production without human intervention.

Before the CD initiative, it was a different story. The Ad Exchange team launched builds to production only three times per year. Builds broke regularly, and they’d take a week or so on average to rebuild. Coding would take place in intense six-week sessions, followed by eight weeks of testing. QA would sign off with a long list of exceptions and it would take another four weeks to launch into production.

“The same team today launches regularly,” Yahoo! senior product architect Stas Zvinyatsokovsky told an audience. The team goes from commit to certification in about six hours and they queue up to launch every day. If there’s a break or a security issue, they fix it the same day.

Zvinyatsokovsky says the Ad Exchange team has gone “from ‘continuous debacle’ to continuous delivery” in the space of two years.

“Continuous delivery has become part of the culture,” he said. “Nowadays, when a new product is delivered we expect then to launch to production multiple times a day.”

A decade ago, continuous delivery was seen as a new and confusing concept – something for early adopters to test and play around with while the rest of the business community stuck to their cycles. Nowadays, many companies are still figuring out the best ways to integrate CD into their operations. But CD is making its way into the mainstream.

Companies small and large – from Choose Digital all the way up to Amazon – across industries as varied as retail, travel and digital music delivery are generating real, tangible results. They’re deploying more often, with less friction. They’re creating whole new business models and they are beginning their DevOps transformation – thanks to continuous delivery.