The Evolution of Scalable and Resilient Container Infrastructure

Kubernetes has taken over the world by storm. You could unconditionally say that it has won the container war. Docker has similar offering with Docker Swarm but Kubernetes offers a lot more in the form of resiliency, scale, ecosystem and production readiness. 

So, what does Kubernetes solve? And why does its demand grow drastically? In order to understand this we need to go back in history. 

Virtualization

Virtualization is the process of creating a virtual representation of your physical hardware. One well-known example is what we call the Virtual Machine (VM). Previous to VMs, a single server was allocated to one customer and most of the time, the server was underutilized. VM's solved this problem because now:

• One physical machine could host multiple tenant workloads without compromising security.
• Because the underlying infrastructure was cheap, the cost of ownership was reduced.

While this was great, the Internet as we know evolved rapidly. Services that offer e-commerce, movie streaming, and storage are handling billions of transactions daily which leads to the need for dynamic scaling. At the same time around 2006-2010, AWS launched EC2 solving the elastic compute/dynamic scaling problem. You could provision compute with just a mouse click and Amazon would take care of provisioning them.

There was still one problem though. In order to run a service, you need dependencies to be available first. For example, let's say you need to run a Java server. For that, you would need:

• Java Run time Environment (JRE)
• An agent that monitors the service process and restart it in case failures
• Apache Tomcat/ Jetty
• Mvn/Gradle depending on the dependency manager

So, when a VM is launched, all of the required software would have to be installed before it could be used. And when things are required to be set up from scratch, they are error-prone and time-consuming.

Containers

Containers solved this problem beautifully. With Containers, you specify all the requirements to run the software in an image file. Now, when a new VM is spun up, all you need is Docker. The image when provided to the Docker runtime, spins up a container, and your application is up and running in seconds.

What was a multiple hours effort came down to seconds.

Below is the example of a Docker Image:

# 1. Base image
FROM python:3.11-slim

# 2. Set the working directory in the container
WORKDIR /app

# 3. Copy local code to the container
COPY . .

# 4. Install dependencies
RUN pip install --no-cache-dir -r requirements.txt

# 5. Expose the application port
EXPOSE 5000

# 6. Run the application
CMD ["python", "app.py"]

While containers had many advantages they came with some caveats too.

It is common for containers to fail and when it fails, manual intervention is required to bring them up. Now, if a service is running hundreds or thousands of containers, this will quickly go out of hand.

Containers are also lightweight compared to virtual machines (VMs). While a container can start up in just a few seconds, a VM typically takes several minutes. This difference becomes significant when handling spiky or unpredictable traffic.

Kubernetes

Kubernetes solved both these problems. It took care of spinning up new containers, depending on demand and recycle failed containers automatically.

Let's look at some of its core concepts:

• Cluster
  A cluster is nothing but the set of machines on which the containerized applications run.Simply put, these can be your VMs or the physical hardware.
• Node
  Node is a single VM or physical machine in the cluster.
• Pod
  Pod is the smallest deployable unit in the Kubernetes. This is where the container is deployed along with shared storage/network capabilities. While you could deploy multiple containers inside a Pod, the best practices is to just have one container per pod.
• Control Plane
  The control plane is a key component in the Kubernetes cluster. It is responsible for making sure the current state matches the desired state of the cluster. The desired state is provided to the control plane in the form of a spec file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: books-api-deployment
  labels:
    app: books-api
spec:
  replicas: 2  # Desired number of pod replicas
  selector:
    matchLabels:
      app: books-api
  template:
    metadata:
      labels:
        app: books-api
    spec:
      containers:
        - name: books-api
          image: myregistry/books-api:v1.0.0  # Replace with your actual image
          ports:
            - containerPort: 8080
          env:
            - name: ENVIRONMENT
              value: "production"
            - name: DB_HOST
              valueFrom:
                configMapKeyRef:
                  name: books-api-config
                  key: db_host

What we have discussed till now is just the bare bones of Kubernetes. In the next set of articles, we will discuss setting up a hello world Kubernetes cluster, storing secrets and setting up observability using OpenTelemetry where we will emit metrics and logs. 

Similar to package manager for Node, .NET and Java, Helm is the packager manager for Kubernetes. We will discuss it in depth, too.