How to Automate Container Security by Using CRDs to Get Security Policy as Code

Security has long been a sticking point for many DevOps teams (including my own, at a Canadian insurance and financial services co-operative). While available tools have enabled automation across plenty of other parts of our CI/CD pipeline — and made automated deployment of our container-based applications the norm — security automation has largely lagged behind.

Like most DevOps teams, we put automated vulnerability scanning into place, but the manual effort of building security policies to safeguard production application workloads remained a pain point.

Enter Kubernetes custom resource definitions (CRDs), which have enabled our team to solve this issue by making it possible for us to declare application security policy as code early on in our pipeline and automate the deployment of security policies into production environments.

We also now utilize CRDs to introduce global security policies that define allowed application behavior and implement safeguards across multiple Kubernetes clusters. By using CRDs to automate and centrally manage security policy as code, I’ve found that we are able to both harden and simplify our policy enforcement while making our application deployments and updates that more efficient, error-proof, and most importantly, secure.

Here’s how our insurance and financial services co-operative went about implementing our application security policy automation:

You may also like: Writing a Kubernetes CRD Controller in Rust.

CRD Benefits

Our team uses NeuVector CRD to define security policies within the NeuVector container security platform, which we selected in order to leverage some specific advantages those tools offered. This CRD enables policies that first capture a full profile of normal and legitimate application behavior.

It whitelists those behaviors, including all network rules, processes, protocols, and file activities that are in-line with the application’s standard operations. It then provides security by only permitting approved network connections within the application’s container environment (as identified using Layer 7 application protocol inspection) and prevents any abnormal external connections.

By doing so, our policies can prevent attackers from seeking to exploit internal or external communications from gaining a foothold into our application environment.

The CRD allows us to define rules on a global or per-service basis. It’s also RBAC-enabled, allowing us to leverage native Kubernetes service accounts and roles to enforce security policies. Versioning then helps keep track of policies for each revision of our application, and integration is supported with security policy management tools, such as Open Policy Agent.

CreatING the NeuVector CRD

The NeuVector CRD lets you use Kubernetes-native YAML files to create security rules.

To begin, you’ll need to create the NeuVector CRD. The following example YAML code creates the CRD, defining NvSecurityRule with a namespaced scope and NvClusterSecurityRule scoped for operations across the cluster.

To then create the CRD, run:

With the NeuVector CRD in place, all custom resources invoked with the kind,  NvSecurityRule, are now processed by the CRD. Therefore, you can proceed with creating custom resources to define your security policies. (Before going forward, refer to the NeuVector documentation and add all necessary clusterroles and  clusterrolebindings.)

Also, using a CRD for NeuVector security rule deployment is RBAC-enabled natively in Kubernetes; security policies declared by a CRD for a certain namespace may only be deployed by user service accounts with permissions to deploy to the workspace. Similarly, service accounts must have the proper cluster admin rights to deploy cluster-wide CRD definitions across namespaces.

Below is a snippet of sample code from demo-security-v1.yaml, restricting nginx-pod containers in the namespace demo to connect to node-pod containers in the same namespace only by using the protocol HTTP.

Beyond this snippet, the yaml file should go on to specify all network connections permitted for containers in the demo namespace, as well as network connections utilizing the redis protocol, and the processes and file activity permitted for each container. Be sure to put these security rules in place by deploying them into NeuVector before deploying your application workload, so that security is active from the beginning of runtime.

To deploy your security policy, simply use a command resembling the following example:

NeuVector interprets security policies in newly-created custom resources and proceeds with REST API calls to the NeuVector Controller that creates rules and configurations in NeuVector as necessary to enact the policies.

Use Cases for Security Policy as Code

I think CRDs and the ability to define security policy as code should open up quite a few possibilities for DevOps/DevSecOps and developer teams. Here are four ideas that come to mind:

Develop and Test Security Manifest That Introduces Security Earlier in the Application Lifecycle

Developers can utilize the CRD to make security top-of-mind early in development by creating both the application deployment manifest and the security manifest. After image building and automated vulnerability scanning are finished and the DevOps review is complete, DevOps can test both manifests and offer developers feedback with an eye to security.

New applications can then be deployed with effective security policy in place from the beginning of production.

Use Behavioral Learning to Create a Security Policy.

DevOps teams can use NeuVector’s behavioral learning capabilities within a testing environment to define security policies and create YAML files formatted for the NeuVector CRD. In the following workflow (beginning in the bottom right of the diagram), your DevOps/QA team deploys your application to a test/QA environment, where a full behavioral profile for your applications completed —producing appropriate network, process, and file access security rules.

The created rules can be exported in YAML format and should then undergo developer review, editing, and DevOps testing before deployment into production.

Define Global Security Policies

Importantly, NeuVector CRD can enable global security policy definitions that are not connected to any specific application workload, nor targeting particular workload groupings within a cluster. For example, your security, compliance, or operations teams may have a need to define global network ingress and egress security rules to block certain processes across all containers or to permit monitoring or diagnostics processes across an entire cluster.

Using global and application-specific policies in tandem, your teams can shape the precise rules your organization requires. For example, here is one we use to deny SSH connections from containers.

Migrate Policies From Staging to Production.

You can control the automated migration of security policies — either in their entirety or only those policies you select — from staging to production using the NeuVector CRD. In the NeuVector console, you can configure a “New Services Mode” to either Discover, Monitor, or Protect settings. Choosing Monitor or Protect ensures that all newly deployed or updated services must include the security rules you require before becoming active.

By taking advantage of Kubernetes CRDs and security policy as code, your dev and DevOps teams should be able to introduce newfound security automation and ensure that your applications are that much better protected, from the beginning of development and on through to deployment in production.

Further Reading

• Create a BOOT.bin, Program an SD Card, and Boot a ZC706 Using Windows.
• Why Kubernetes Operators Are a Game-Changer.
• Introduction to Service Meshes on Kubernetes and Progressive Delivery.