How Experiment Cultures Lead to Continuous Deployment
In past webinars we’ve talked about Lean startups and what the enterprise can learn from them. We’ve pointed to these guys as examples at the extreme doing crazy things like deploying an app to production 50 times a day.
The standard explanation for lean startups movings towards continuous deployment is that they walked a path of removing wastes. Automated deployments were effective in removing errors and wasted effort. The endeavored to speed features to production because features written but not delivered to the customer is seen as wasteful “inventory”.
This is no doubt at least partly true. But when you read discussions of the practice there’s generally a heavy emphasis on monitoring deployments and rolling them back if they’re not better than what we had before.
Rollback based on business metric monitoring is often glossed over but can be explained by looking at the lean startup culture of experimentation. Talking in terms of experiments rather than user stories was the focal point of a great talk by Adrian Howard at Agile 2012. Unnecessary features are wasteful, bad and very common in software. Adrian observed that the lean startup folks use aggressive experimentation to try to detect unneeded features before investing heavily in them.
For instance, if the business thinks we should add a “login with Facebook” alternative to the standard registration page we should know why they want it so we can test their idea. Perhaps they think the easier option will increase registrations by 5%. Lean startup fans would see an opportunity to experiment. Rather than build the feature, quickly add a Facebook login link (which shows an nice error when clicked), present it to a portion of your traffic and count clicks. If lots of people try that approach, the feature might be worthwhile. But, if nobody clicks the link it’s highly unlikely to improve registrations and we can remove the fake feature and move on to better uses of time.
Automation infrastructure supporting experiments
When you endeavor to experiment like this routinely, you have to build some key features into your runtime environment. You need to be able to easily deploy to a slice of your servers, route fractions of traffic, keep track of concurrent versions in production, and easily track business metrics like how many people are clicking a link or filling out a form. These are the pieces that provide a safety net for more frequenty deployments. All new versions can be released to only a portion of the users at a time. Loud alerts or automatic rollbacks can be in place and catch big problems before catastrophes.
In order to experiment, the lean startup guys often built monitoring and experimentation infrastructures. When they decided delivering more frequently was desirable, they were able to do so more comfortably. While few of us want to live at the extremes of many production deployments of an app per day, many of us want to detect production problems quickly and be able to respond accordingly. Copy the infrastructure of good tests, piecemeal automated deployments, and good monitoring and apply them to your more reasonable goals.