Last time, we looked at how “regular” development practices have made sure that things worked at the development team level. It’s time to move on because, as we know, it’s not working software unless it’s tested.

Let me tell you a story, sonny. When I was young, we built the software all on my machine: compiling, testing, and packaging. Turns out, these days, it’s not really that easy.

A-Testin’ We Will Go

To take it to the next level, we need more DevOps practices. Let’s look at them through the principle spectacles:

• Minimize risk (delivery methodology, environment management).
• Minimize waste (automatic packaging, automatic deployment).
• Early feedback (automatic system tests, non-functional tests).

The delivery methodology is a relative of the branching methodology we discussed last time. It is usually developed ad-hoc-ishly like its relative, which is too bad.

Unless your product is getting deployed for the first time, you will need to be able to do two things: rebuild it from scratch and upgrade it. There are as many ways to do it as there are companies, but someone needs to say how we publish upgrades and patches and even add operations features (metrics, etc.) in a matter that affects how we architect and code.

The idea is not having this methodology invented (or reinvented) on the fly. When we have proper ways of building environments and packages, we reduce the risk of testing a software package that is different than the one we deploy.

This is where we’re entering the realm of environment management: as we define the environment we’re going to test in. There are many of those, and sometimes (hell, most of the time), they are different than what we have in production. So, we also need to understand the differences and their impact.

With great tools like Docker and Puppet, we can easily define and spin environments for testing (and other purposes that we’ll discuss later). DevOps skills include:

• Understanding tools.

• Understanding the economics of environments mapping.

• Maintaining a “map” of which environments are available and what they include, as well as which versions are available and how they differ from each other.

From understanding the  methodology and having an environment management capabilities, we can talk about packaging and deployment processes.

The Package Has Arrived

When looking at skills, I separate the packaging from the deployment. The packaging of the software is not just creating deployable packages, services, and components. The packaging skills include more than just automation. Because software is so complex, there’s dependency management, understanding what should be a prerequisite in the environment and which version.

In my ancient story, deployment was manual. We produced an installation package that the testers would install on their environment. However, with current distributed software, we’re talking about an automated software package deployed in different environment.

Finally, we’re expanding our feedback cycle with more tests.

MOAR Feedback!

Unit and integration tests are quick. They require a low threshold of prerequisites in order to run. As we move on to system and end-to-end tests, we need fully prepare environments to run them in. We may need simulators (i.e., service virtualization) to run them, because a real third party may not be available. Apart from the functional tests, we can add automated performance tests (like load, availability, recovery, etc.) to get more feedback on the product.

That’s not all. We need to report the results back. Usually, red and green are not enough at this point, so we need to allow better data collection and reporting.

Just for testing.

Still think that these are just “developer” practices? We’re not even at the Ops level yet. We’ll pick up next time on delivering to production.