DevOps and System Security: Best Practices and Security Challenges

If you are a developer, then you likely agree with me that the term “security” has a negative connotation to most of us. No doubt implementing security features increases the programming work you do.

And that’s not all…

Uncertainty, roadblocks and fast development cycles compound the problem. A study by SpiceWorks found that the major risks to IT security are human error, lack of process, and external threats.

If the term “security” worries you, then fret no more. I will show you how DevOps and system security can help you handle these three risk factors without reducing reliability or stability. Let’s jump in!

The Unique Role That DevOps Plays In The Development Cycle

DevOps does a very good job of eliminating the toxicity surrounding software security.

DevOps changes the developers’ minds from adhering to rules and guidelines and helps them find creative ways to solve significant security problems.

In the SpiceWorks survey, IT professionals were asked to state threats based on how risky they are to IT security.

Let’s look at how DevOps helps deal with the three main issues that developers complained about in the Spiceworks survey: human error, lack of process, and external threats.

Human Error

There’s no denying the fact that we sometimes make typos while coding or writing configuration files. While we have linters and tools to catch them, some errors make it into production and cause massive security problems.

Even the titans of the tech industry are sometimes affected by human errors. The 2018 Cost of Data Breach attributes 25% of all breaches to human error

Why is this?

Developers are so familiar with what they type that their minds see what’s expected instead of what’s actually on their screen. This is where DevOps comes to the rescue.

With DevOps, you can introduce templates to standardize common configurations. DevOps also allows you to automate common tasks, and prevent those fat-finger typos.

Lack Of Process

It’s common for scripts to be used to configure, start, change and shut down services in the production environment. If any developer tells you that they don’t use scripts for mundane infrastructural or networking tasks, they are lying.

And there is nothing wrong with that. The problem is that these scripts are rarely reviewed or even versioned like the production software the developers churn out.

These scripts are not reviewed because there is no process in place to review them. DevOps allows you to clearly define the deployment process. You first write down the requirements and dependencies, while eliminating redundancies. It then allows you to use orchestration for deployment, necessitating manual steps when very necessary.

Finally, DevOps allows you to create a process to review scripts used in the process.

External Threats

You may wonder how DevOps can help you deal with external threats. On the surface, DevOps seems to be least capable in dealing with external threats. Trust me, though, it does.

Malware and DDoS attacks are the most serious business threats today. Code reviews and manual detections by developers uncover most of the vulnerabilities pre-production.

Sadly, this does not extend to production, where risk exploitation presents an existential business threat.

DevOps can reduce risk through more thorough testing and security feature development, which can be deployed in production. With DevOps, you develop these policies and treat them like code.

To take advantage of DevOps in minimizing threats, add web app security development into the actual app development cycle. Take security features as seriously as code, and have them reviewed and standardized.

Finally, run many tests frequently even in production. This can be automated, and if possible, integrated into the development life cycle, further reducing risks earlier.

DevOps Best Practices To Reduce the Likelihood And Effect Of Attack

Network Segmentation

By segmenting your network, you minimize the attacker’s line of sight. Start by grouping assets into units that do not trust each other. Where access needs to traverse trust zones, use a secured jump server with several authentication layers and session monitoring.

You should limit system access based on the context, users’ roles, application, and requested data.

Implement DevOps Secret Management

Remove any embedded credentials tucked inside code, files, scripts and service accounts. Keep the code and password separate to ensure that when not in use, they are saved in a central password safe.

The implementation of API calls gives you control over files, embedded keys, and code. This way, based on the policy in place, you can rotate passwords as often as possible.

Collaborate and Understand Your Security Requirements

Many developers are supposed to follow a security policy. This may be a customer security policy, corporate security policy, or a compliance standard. You should use a relevant security policy even when not mandated to do so.

The secret is to identify and vet the sources of information for security expertise and collaborate early. This way, you will understand your security requirements early and incorporate them in your development cycle.

Patch Management

It is important that you have a method to patch your systems quickly and reliably. It’s no secret that missing patches is a known cause of vulnerabilities that attackers take advantage of.

Your ability to deliver a patch quickly and at scale can reduce the number of security gaps and chances of an attack.

Security Challenges In DevOps and How To Solve Them

The Focus On Speed In DevOps Usually Leaves Teams Helpless And Reactive

DevOps allows developers to push and modify large batches of code over periods as little as days or hours.

The result? The development speed usually outpaces the speed at which security teams can do code reviews. Without proper automation of and configuration of code analysis, DevOps will be painfully slow or there will be no security.

If this misalignment persists, the result could be insecure code, serious vulnerabilities, hardcoded passwords and other application security weaknesses. This causes downtime at best, and destruction at worst.

Developer Resistance To Security

Many developers believe that introducing security will slow or block the development process.

The cost in time and effort that you will spend hunting a security flaw in the design and implementation stages is orders of magnitude smaller than trying to find and fix problematic code way down in the development cycle.

DevOps-Cloud Environments

Very often, DevOps environments rely on cloud deployments. As a consequence, they share many concerns surrounding cloud security. There is no doubt that DevOps teams normally leverage new, immature and at times open source tools to manage server instances.

Since these environments operate at hyperscale, a small configuration error like sharing of APIs, SSH keys, and privileged credentials can be exploited to cause massive operational dysfunction and other serious exploitation problems.

Use Of Containers and Other Tools In DevOps is Risky

Containers and tools to manage them such as Kubernetes, Docker, and CoreOs among others in DevOps present huge productivity and innovation potential. At the same time, they also carry their own security risks. Look at containers: Even as lightweight and portable application packaging platforms, they can be spun up and down instantly and run on any computer or cloud.

Absent of proper controls, these containers pose security risks because of low visibility into the containers. This is more complicated as the container shares an operating system with other containers.

One study by ThreatStack found that a staggering 94% of respondents agree that containers have a negative security implication in their organization.

Uber: A Real Lesson On Caution For The DevOps Culture

Many people argue over which was the worse crime, the fact that the Uber’s information breach of 57 million customers and 600,000 drivers was caused by poor security, or that they paid the hackers to conceal this information from the public for months.

As we have established, the speed that is at the core of the DevOps culture may force developers to take risky shortcuts. Uber’s breach was caused by an employee pushing credentials to GitHub, an open-source code repository used by millions of developers.

A hacker then used those credentials to gain access into Amazon AWS servers used by Uber. As serious as this mistake is, it’s shocking that developers often save credentials and other secrets in a code for easy access.

The worst part?

Hackers know exactly where to look and exploit such negligence with devastating effects. Companies lose public trust, which presents a serious existential threat.

Conclusion

As should be apparent by now, DevOps security plays a major role in creating a productive DevOps ecosystem, while reducing code vulnerabilities and weaknesses before they become serious problems.

By introducing DevOps security early on in the product lifecycle, you ensure that security is at the base of systems and application development. You reduce the chances of a data breach and develop technology that meets user needs.