A colleague and I were talking today about what happens when things go badly. I said that I thought that in an ideal world, we should be working to make our deployment pipelines so efficient that even in the event of an emergency, we should be able to make changes and have them fully validated before we release them. After all, the last time that you want to be increasing risk is when things are already bad! Jason made me laugh with, “So, rather like, ‘the first casualty of war is truth,’ ‘the first casualty of a software emergency is validation.’”
This is spot-on. What commonly happens is that when things go badly, teams throw their protections out of the window and start changing things in production without testing them first. Scary!
The ideal answer to this is to concentrate, before the emergency, on shortening your cycle time. Your cycle time is the time it takes you to release the smallest possible change to your code following your normal release validations and procedures. Your cycle time should be short enough that when an emergency hits, you can still release your fixes having validated them fully.
Short cycle times are a general good. They allow you to work in small batches, ensuring that each change is small enough so that when it does go wrong, it is simple to see where the problem lies and to know how to fix it. They provide rapid feedback for your ideas, allowing you to quickly assess their value. They stop you from creating snowflake servers and deploying untested changes into production. They allow you to get valuable software into the hands of your users quickly and efficiently and they allow you have a single, validated route to production for all your changes.
Cycle time is a wonderful tool to drive good behavior. People often ask me how to start with something as complex as the introduction of Continuous Delivery to an organization. The answer is pretty simple: Look to your cycle time.
I think that Continuous Delivery is built on serval important foundations: version control, Continuous Integration, automation, feedback, collaboration, and cycle time.
If you don’t use version control, shame on you. Download a free VCS now and start immediately. If you are not yet doing Continuous Integration, come on — catch-up, this has been a well-known, well-publicized, effective practice for more than a decade now. Cycle time is different than these, though. The others are mechanisms. Cycle time is a metric by which you can measure your performance. As such, it is a great tool to help you identify where to start.
A great tool for understanding your cycle-time is value-stream analysis (AKA value-stream mapping).
Draw a diagram enumerating the steps, laid out on a timeline, that changes destined for production progress through — from idea to working, validated software in the hands of a user. You can do this at various levels of resolution depending on how much effort you want to spend identifying and measuring the steps. You should try to capture any pauses between steps as well as the steps themselves.
Here are some examples of value-stream maps for some real projects that I have seen.
However good or bad you are at software delivery, this should highlight places that you could improve.
This is an excellent tool and I recommend that you try it for your development process. It may surprise you.
I like tools and processes that encourage good behavior — processes that make doing the right-thing easy. Cycle time is one of those tools. If you work to optimize cycle time, this will have a beneficial effect on your whole process — even if that is all that you do. It will make you eliminate waste and it will make you collaborate more because you will need to minimize hand-overs. It will make you reduce the amount of inventory and work in process and thus make the process more efficient. It will make you automate more because human beings are too slow to sustain fast cycle times.