Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Infrastructure Tooling Anti-Patterns: Accumulator

DZone's Guide to

Infrastructure Tooling Anti-Patterns: Accumulator

· DevOps Zone
Free Resource

Download “The DevOps Journey - From Waterfall to Continuous Delivery” to learn about the importance of integrating automated testing into the DevOps workflow, brought to you in partnership with Sauce Labs.

As our (or our clients) infrastraucture grows and runs for longer durations, I have noticed that there are certaion parts of our infrastructure that are known only by certain people to a certain extent. Due to the nature of IT operations, most of the engineers stay in firefighting mode, and fix they some of the problem with a manual hotfix (be it stability related issues, security related issues or performance related issues).

Over time these pieces of infrastructure (or infrastructure services) accumulate some feature or functionality that is not automated or documented, and slowly it attains a state where if you kill that server it will be difficult to recreate it, not only because you don't know what exact steps need to be taken to bring it back to the original state, but also there are dependenies with other integration points you need to worry about. In the community we call them 'Works of Art'. 

There are many ways to fix them, but this post is about how to catch them.

An ounce of prevention is worth a pound of cure. 

I prefer to kill the whole environment (staging, pre-production, UAT) every weekend or have non-functional relases where I just recreate the production infrastructure at regular intervals. This does not eliminate the accumulation of manual fixes, but this does indicate if any manual fixes are present that are crtical for the services to run. By doing this more frequently I reduce the risk of large, accumulated manual fixes. To me this is a litmus test or Gold Standard for Automated Infrastructure.

Discover how to optimize your DevOps workflows with our cloud-based automated testing infrastructure, brought to you in partnership with Sauce Labs

Topics:

Published at DZone with permission of Ranjib Dey, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}