X.509 Certificate Issuance: Too Slow for DevOps?

The now widespread implementation of DevOps has been a real blessing for enterprises. Better on-premises and cloud network backends help to make it all possible. DevOps is often a much more efficient and effective development methodology than older waterfall and Agile approaches.

A waterfall methodology is completely linear, going from the client’s requirements, to design, to implementation, to verification, to maintenance, with all of the work and communication flowing in one direction. Therefore, the client cannot easily provide feedback about how code is implemented, verified, and maintained. And the whole developmental process can be slow to respond to changing needs.

Agile methodologies, by their very name, are more agile. Usually there’s a linear flow between the client’s requirements, the design, the development, and testing. But the client gets to directly communicate with the testing team to play a more active role in the development process. So, the client’s specific requirements inform the metrics within which the software is tested.

DevOps goes a step further. DevOps completely integrates the development and IT deployment teams so that necessary changes and adjustments can be made in a jiffy. It can also make applications a lot more secure because security patches can be deployed more quickly and efficiently. But every single server, virtual machine, and container is its own entity. And each and every machine entity requires a machine identity so that all of the other components of the application’s system can verify their authenticity, thus keeping the entire infrastructure and toolchain safer from cyber attacks.

Operating system-level virtualization is an excellent means for DevOps, and containers make it all happen! DevOps can also use application-level virtualization, deploying conventional virtual machines. All of that virtualization is wonderful for DevOps because code doesn’t have to be run directly in the kernel, and a computer’s resources can be more dynamically allocated. Machines can be initialized, configured, suspended, and killed all according to the needs of DevOps’ activities. It all makes development, patching, and testing much easier and more responsive.

But when you think of everything that DevOps entails, you can visualize how frequently X.509 certificates, a very important type of machine identity, need to be configured and deployed. A single virtual machine or container could have a lifespan of only a few days, or perhaps even only a few hours. Each and all of those components need a properly configured certificate so that the entire system can be sure of their authenticity and integrity.

Will Sargent has written about how the Chain of Trust pertains to the deployment of X.509 certificates:

“In TLS, the server not only sends its own certificate (known as an ‘end entity certificate’ or EE), but also a chain of certificates that lead up to (but not including) a root CA certificate issued by a certificate authority (CA for short). Each of these certificates is signed by the one above them so that they are known to be authentic. Certificate validation in TLS goes through a specific algorithm to validate each individual certificate, then match signatures with each one in the chain to establish a chain of trust.”

If certificates are improperly signed, they can be forged. That’s a common means of cyber attack.

If the public key within a certificate doesn’t comply with current standards for factors like key length or cipher complexity, they can be easily cracked. That’s another means for man-in-the-middle attacks!

If a developer is expected to manually create, configure, and deploy X.509 certificates, the constant need for many new certificates every day will slow the whole development process down. It also opens up the risk of human error.

Properly-deployed DevOps is great for both security and functionality. The speed and efficiency in which applications can be deployed, adjusted, and patched makes them more responsive to enterprise needs and security hardening. Expecting DevOps teams to submit tickets or require any human intervention for certificate issuance is incompatible with DevOps approaches and will impede software delivery. You absolutely do not want X.509 certificate deployment to be the bottleneck in all of this.

Kevin Bocek makes the problematic nature of the inefficiency of manual certificate deployment very clear:

“The process used to issue and deploy keys and certificates has been slow and complicated, and that conflicts with DevOps’ goal of speed and efficiency. Getting trusted digital certificates typically takes days, rather than the seconds that fully automated and orchestrated DevOps environments require.

As a result, DevOps teams often end up trying to engineer their way around the problem, in some cases using untrusted or unauthorized certificates, such as those offered for free by Let’s Encrypt, GoDaddy and similar services. In other cases, DevOps teams don’t use certificates at all.

Both approaches make it challenging to identify and mitigate threats in a timely manner. Without HTTPS encryption, data may be exposed to attackers. But with HTTPS, it’s difficult for security systems to inspect encrypted traffic for threats and attacks.

It's a troubling paradox: How do you capitalize on the benefits of DevOps without confronting additional security risks? To address this, you need to take a different approach: You must build security into DevOps in a way that is fast and easy, without compromising security.”

If a developer needs to issue a ticket in order for a certificate to be deployed for a virtualization entity, they could be waiting for days. The machines themselves often don’t have a lifespan that long. Therefore, developers applying DevOps practices often circumvent traditional security processes in order to deliver their software and meet SLAs.

There’s a much better way of doing things.

DevOps teams need to collaborate directly with security teams (and vice versa) to ensure that the deployment of X.509 certificates is automated and embedded into the toolchain. Any Kubernetes, Docker, Terraform, Chef, or other platforms must be completely integrated with an automated system of certificate issuance.

Certificate automation also can help ensure the enforcement of organizational policy and regulatory compliance for enterprises. All of the attributes of certificates can be properly standardized in their generation.

*Below, Gartner survey results showing “Automation of manual steps” as a strategy already employed by 40% of surveyed professionals, as a way to overcome hurdles to using DevOps in regulated situations.

Greater automation modernizes the entire DevOps process. With the standardization of the code for generating certificates, PKI and security teams are empowered to remain the experts in controlling the certificate types and sources across the enterprise.

X.509 certificate deployment must be automated for optimal functionality, efficiency, and security of the entire DevOps process and the applications they deliver.