Over a million developers have joined DZone.

If Devs Own Testing, Ops Own the Environment

DZone's Guide to

If Devs Own Testing, Ops Own the Environment

· DevOps Zone ·
Free Resource

Is the concept of adopting a continuous everything model a daunting task for your fast moving business? Read this whitepaper to break down and understand one of the key pillars of this model in Continuous Governance: The Guardrails for Continuous Everything.

The devs are all writing automated tests and some are even experimenting with TDD. Congrats! But what happens when the build server breaks? Who’s taking care that Continuous Integration is running smoothly? Seems to be an awful lot of red in there…

Unlike writing the first basic tests, CI is hard. Did the test fail due to an application bug or is it the environment? Once again, the dreaded chant of “it works locally” is taken up. What most people fail to understand is that the failing test is the first sign of a communication breakdown between developers and sysadmins.

Maybe the dev locally installed the latest version of python because it brings a 25% performance boost to execution. Or the sysadmin upgraded the virtualenv package on test due to a glaring security hole. And never rule out that both these guys made some “quick fixes”! Whatever the cause, your business application doesn’t work anymore. Who’s “fault” is it? And, more importantly, who should fix it?

This situation is the whole point of DevOps. Or, rather, the idea behind DevOps is toprevent these situations from happening in the first place.

An ounce of configuration is worth a pound of troubleshooting

  1. get the build server under control (puppet, chef)
  2. get the dev boxes under control (vagrant => puppet, chef)
  3. failing test on CI means “stop the line” – no more commits, builds from *anybody* until the test passes

By “under control”, the team agrees on a set of tools, applications and version numbers that need to be available on the server. These tools are defined in an automated configuration management tool running on all development, build and test infrastructure. Initially, only the sysadmins have write access to the repository that hosts this tool.

How to handle problems the DevOps way

  • Never blame!
  • Work together to solve problems (helps with situational awareness)
  • Share daily stand-ups and tell folks what you’re doing!
  • Trust, Measure and Publish

Don’t fall into the trap of blaming and arguing – this isn’t a pissing contest! Act like professional adults interested in the success of your company. The other party did what they did for a reason – take interest and find out. Try to work together to explain and understand why the change was made. If at all possible, combine daily stand-ups and tell people you’re going to be patching the test servers.

Avoid these common mistakes

  • don’t give devs write access to the automated configuration management – they don’t have time for sysconfig
  • don’t give devs sudo access – when you have a hammer, everything is a nail!
  • scale down resources for dev boxes – their desktop VMs typically don’t have 16GB of RAM

Ultimately, when development is responsible for testing and sysadmins are responsible for the environment, there is little chance of misconfigurations and misunderstandings. Having clear lines of responsibility empowers a team to focus on what really matters (and what they’re truly competent in). Communicate this rationale to both sides and get these folks pulling together for the business not their individual departments.

Are you looking for greater insight into your software development value stream? Check out this whitepaper: DevOps Performance: The Importance of Measuring Throughput and Stability to see how CloudBees DevOptics can give you the visibility to improve your continuous delivery process.


Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}