Automation Justification

Automation Justification

(as presented in TechWell Agile and DevOps Virtual Conference 11 November 2020)

Ask a group of engineers how to “DevOps,” and many will say, “Automate all the things!” That’s a great answer and definitely covers many of the DevOps processes, but DevOps is so much more than that. I want to discuss why automation is important, how it relates to DevOps, and when automation may or may not be the right tool for the job.

So, let’s hit Wikipedia for some definitions…  What is automation?  “Automation or labor-saving technology is the technology by which a process or procedure is performed with minimal human assistance.” (Wikipedia). Or, as I like to say, it's a way to get computers to do the work I don't want to do, and with minimal supervision. I’m all in

And DevOps? “DevOps is a set of practices that combines software development (Dev) and IT operations (Ops). It aims to shorten the systems development life cycle and provide continuous delivery with high software quality. DevOps is complementary with Agile software development; several DevOps aspects came from Agile methodology.” (Wikipedia). 

And why do I like DevOps so much?  As a "peace, love, and happiness" kind of person, I appreciate the emphasis on communication and cooperation. As a lead, I appreciate the efficiency, consistency, and the fact that there will be fewer stovepipes and communication gaps as a team working together and communicating well.  Oh, and as a developer, I love to automate things!

Continuing on this topic, I’d like to share the way I explain DevOps to folks: So, you have a process to automate — you’ll perform it manually and make sure you have the procedure down, then you need to document it, to make it easier to figure out the steps.  Now the cool thing here is that you may not know it, but you have already begun the DevOps process!  If you were to go home tonight and win the lottery, a teammate would be able to take over for you! After this step, you’ll go ahead and start to code or script your process. Once you are satisfied, you’ll want someone to review your code, and you and/or teammates can write some tests. 

When everything is fine, the code can be released. It is time to monitor, make sure it is working properly, and watch for needed changes, whether it’s bugs in the code, changes in the environment, or updates suggested by the customer. Of course, as we are going along, we retrospect and continue to find ways to improve both the product and the process.  Now, the big takeaway from this article is right here in this paragraph.  Every step in this process brings you improvement, so even if you do not get beyond the documentation step, you are already ahead of where you were before and headed toward DevOps Nirvana!

Now here’s a question I can only begin to answer — why do we automate things? First, there is the magic triangle of speed/cost/quality — all three of these can be improved by automation — automation adds efficiency to your process allowing you to do more with less but in a consistent manner. It also reduces the chance for human error, whether caused by lack of skill, rushing, or just plain boredom. Automating also allows for scaling; if you can automate a process once for one purpose, you can reuse the code, use the same code on multiple systems, or make quick changes on the fly without having to start a whole process over. Automation allows for processes to be part of a larger automated process (i.e., CI/CD). The fact that code and documentation exist allows enterprise knowledge to be maintained and simplify the training of new employees. The folks who are writing code, reviews, and tests are also learning from this process. And last but not least, people’s time is being freed up to do more interesting and meaningful work.

And what are we automating?  Well, I can only begin to answer this question, as it is a never-ending list.  We automate any process that needs to be repeated multiple times. The more mindless, boring, or error-prone, the more need for automation. Testing and deployment can be automated. Status and metrics reporting can be automated, and the sooner that happens, the better!  Some applications beg for automation, such as auto-sorting items in your inbox. 

Now let’s talk about some of the common tools.  We’ll start with the obvious, coding and scripting languages (Ruby, Perl, Java, and many more).  We’ve been automating difficult processes and calculations for a long time using these languages. Taking it one step further, using your text-based tools such as APIs (Application Processing Interface) and CLIs (Command Language Interface) allows you to take complicated GUIs and write them into commands or integrate them into your codebase and reduce some of the complications of navigation.

Continuous Integration tools (such as Jenkins,  Travis-CI, GitLab-CI) allow you to perform the steps of test, integration, and deployment based on a trigger such as a pull request or a timer. Provisioning tools allow you to automate a system’s setup to ensure consistency among your servers. There are also tools like IFTTT (if this, then that) that you can use at home or at work to automate cool things like getting a text message when rain is predicted or adding a line to a spreadsheet when a PR is created on your GitHub repo.

But remember that DevOps is tool-agnostic.  The choice of tools you use may be different from what others select.  You need to balance your needs with your budget; perhaps create an “analysis of alternatives,” whether it is a whitepaper, a spreadsheet, or the back of a napkin.  Keep in mind not only the budget and needs, but the comfort level of your team and your management, and look ahead at what needs may expand. 

Back to automation, or perhaps lack of it…Part of DevOps is knowing the appropriate thing to do and, believe it or not, there are times when it is not appropriate to automate things.  Here are some examples.  Sometimes you have a task to do and only do once, and although it may not be pleasant, it may be quicker to do it than to code it. Sometimes manual intervention is needed, whether it is to add that the second factor of authentication or to get an interim status, metric, or value. In many cases, human creativity is needed, whether it is making words make sense to a human, making an interface human understandable, or running extra tests that involve some edge cases and human qualities that a machine may miss. 

The human touch is also recommended in code reviews — yes, please use the code grammar checkers and test coverage tools, but getting your code reviewed and reviewing others’ code benefits everyone involved. Sometimes folks worry about the cost of tools and labor to get the process started. Lastly, when starting a larger automation project, do not try to do everything at once. Prioritizing and easing into the automation process makes it simpler and increases the probability it can be done with no loss of functionality. 

In terms of naysayers, some of the reasons given by humans are “if it ain’t broke, don't fix it,” some don't feel comfortable if they are not in control, sometimes the person does not understand the tools needed, and some folks feel like a computer will replace their job. So what do we do? Show them the metrics that can show improvements, teach them how to use the tools (this will also allow them to stay in the job they were so worried about), or just let them know that now that their time is freed up; they can do more meaningful, fun, cool stuff with their time.

Alluding back to an earlier slide, here are some metrics that will show your team, your management, and the bean counters some improvement: cost and time savings; test coverage and speedup; customer satisfaction; fewer defects; faster time to release, as well as to recover from issues; and reduced risk.

So what do we want to do to make sure everyone is happy?  Ensure you go through DevOps' proper stages — do it, document, script, test, review, release, monitor, retrospect.  Ensure your team and your management properly vet any tools used. Make sure changes are done gradually.  Lastly, remember there are situations in which it is totally appropriate to use human creativity.

My favorite thing about DevOps is the feeling of community. You will likely reach a point where your product is fully automated, and you can move your creativity in another direction. Still, it is important to ensure that there is interaction, communication, knowledge-sharing, and documentation every step of the way. And feedback — you will definitely have lessons learned to apply to your next creative endeavor. To make an analogy to my second-favorite thing (after DevOps), like baking, even a perfect recipe needs a little creativity, and often a slight tweak, each time you make it… So automate away, but ensure that the human touch remains part of your process.