Container Security Essentials: From Images to Runtime Protection

Container security is all about making sure you run an image that is exceptionally low in vulnerability and malware. I would love to say having zero vulnerabilities, but it is rarely possible in the real world.

In the worst case, you at least want to address critical to medium vulnerabilities to have a good night's sleep and avoid potential compromise from bad actors. You could also think of container security like peeling an onion, where each layer adds resilience against potential threats. As part of this article, we will learn what the different steps are that we could take to increase the overall safety of the container infrastructure.

The first step in the process is to have minimal & hardened images.

Minimal and Hardened Images

Use base images with minimal packages because the higher the dependencies, the higher the cost to maintain the image. Constant patching and testing cycles will slow down your feature development. Examples of some of the minimal base images are:

• alpine (5 MB) -> Just busybox and some miscellaneous libraries.
• pything:3.12-slim (40 MB) -> Python runtime + minimal Debian packages.
• distroless (20 MB) -> No shell or packages. Just runtime libraries.

Using these or other similar images in production will make your life a lot easier.

Docker, being at the forefront of containers, provides an offering in this space with Docker Hardened Images.

Docker Hardened Images is a registry of near-zero CVE images, with built-in SBOMs (Software Bill of Materials), digital signatures, and backed by an enterprise SLA. Meaning, Docker will maintain these highly secure images, which can form the baseline images for your organization's security team.

Secure Image Supply Chain

Supply chain threats are not new. Remember the SolarWinds attack? The attackers injected malicious code into Orion, a popular IT monitoring and management software used by thousands of organizations. As a result, more than 18,000 organizations were compromised, which included U.S. government agencies.

That is why it is always important to download from trusted sources, which is where SBOMs come into the picture. With SBOMs, you can validate all the sources a particular software is using. 

While there are several tools to generate SBOMs, such as Trivy and Anchore, Docker has also introduced Docker Scout. 

Docker Scout

What sets Docker Scout apart is that, unlike many open-source scanners, which typically rely solely on community vulnerability databases and often require a commercial upgrade for deeper insights, Docker Scout provides comprehensive vulnerability scanning at no cost and is fully open source. 

What you see below is an example of running the Docker Scout command on a particular image:

docker scout sbom <imageId>

    x HIGH CVE-2022-24675
      https://scout.docker.com/v/CVE-2022-24675
      Affected range : <1.17.9
      Fixed version  : 1.17.9

    x HIGH CVE-2022-23772
      https://scout.docker.com/v/CVE-2022-23772
      Affected range : <1.16.14
      Fixed version  : 1.16.14

    x HIGH CVE-2021-44716
      https://scout.docker.com/v/CVE-2021-44716
      Affected range : <1.16.12
      Fixed version  : 1.16.12

Least Privilege and Rootless Container

Running a container as non-root along with a root file system is another procedure in keeping your infrastructure secure. If an attacker can somehow break out of the container, running as root can translate to elevated privileges, which means full compromise.

Here are some ways you can enforce the rootless container capabilities:

• In the Dockerfile using the USER <username or UID> command
• When using Kubernetes, runAsUser and runAsNonRoot must be made mandatory
• Example of a Kubernetes spec to run as read-only the file system:

apiVersion: v1
kind: Pod
metadata:
  name: readonly-pod
spec:
  containers:
    - name: myapp
      image: myapp:latest
      securityContext:
        readOnlyRootFilesystem: true
        runAsNonRoot: true
      volumeMounts:
        - name: tmp-volume
          mountPath: /tmp

• Docker example of running as read-only on the file system.

docker run --read-only -d nginx

Secrets and Encryption

Even though this is well known, it is important to put it out here. Never ever add secrets to container images. Using cloud-specific key vaults or, if you are running on Kubernetes, storing them in Kubernetes Secrets is the right way to do it.

Like how you encrypt data, one can encrypt Docker images as well. In fact, most of the registers like Harbor, Azure Container Registry, and AWS ECR support encryption at rest using customer-managed keys.

Finally, ensure that container images are digitally signed. Image signing allows you to verify authenticity and integrity by checking the signature against a trusted public key, ensuring that the image you are pulling is legitimate and has not been tampered with. Open-source tools like Cosign can help sign using public and private key infrastructure.

Runtime Security and Monitoring

While build-time security is one part, runtime security also plays a significant role when it comes to container security. How do you know if somebody was able to penetrate your containers and is able to run a bunch of commands? This is where open-source tools like Falco come into the picture. Using these tools, one can detect:

• Privilege escalation attempts inside the container.
• Unexpected network connections
• Writing to sensitive directions like /etc/psswd.

Not only detect, but they are also able to send alerts to your monitoring systems for quick mitigation.

In addition to all the above, I also recommend exploring automation to enforce these security rules using policies. Human error is inevitable, and the most reliable way to prevent misconfigurations or oversights is to implement automated enforcement that consistently applies best practices across your container environments.