Openshift Sandbox/Kata Containers
In this tutorial, you will learn Openshift Sandbox containers based on Kata containers and how they are different from traditional Openshift containers.
In this article, I will walk you through Openshift Sandbox containers based on Kata containers and how this is different from the traditional Openshift containers.
Sandbox/kata containers are useful for users for the following scenarios:
- Run 3rd party/untrusted application.
- Ensure kernel level isolation.
- Proper isolation through VM boundaries.
Prerequisites
You will need to install the following technologies before beginning this exercise:
Create the KataConfig
Create the KataConfig CR and label the node on which Sandbox containers will be running. I have used sandbox=true label.
#KataConfig.yaml
apiVersion: kataconfiguration.openshift.io/v1
kind: KataConfig
metadata:
name: cluster-kataconfig
spec:
kataConfigPoolSelector:
matchLabels:
sandbox: 'true'Verify the deployment:
oc describe kataconfig cluster-kataconfig
Name: cluster-kataconfig
…..
Status:
Installation Status:
Is In Progress: false
Completed:
Completed Nodes Count: 3
Completed Nodes List:
master0
master1
master2
Failed:
Inprogress:
Prev Mcp Generation: 2
Runtime Class: kata
Total Nodes Count: 3
Un Installation Status:
Completed:
Failed:
In Progress:
Status:
Upgrade Status:
Events: <none>
Verify a new machine config(mc) and machine config pool(MCP) would have been created with the name Sandbox:
oc get mc |grep sandbox
50-enable-sandboxed-containers-extensionVerify the node configuration. Login to the Node label sandbox=true:
sh-4.4# cat /etc/crio/crio.conf.d/50-kata
[crio.runtime.runtimes.kata]
runtime_path = "/usr/bin/containerd-shim-kata-v2"
runtime_type = "vm"
runtime_root = "/run/vc"
privileged_without_host_devices = trueVerify the Runtimeclass:
→ oc get runtimeclass
NAME HANDLER AGE
kata kata 5d14h
This completes the deployment of the Sandbox container using Operator.
Deploying the Application on Sandbox vs Regular Containers.
Let's try to deploy Sandbox and Regular containers from the same image and will verify the difference.
I have used a sample application image(quay.io/shailendra14k/getotp) based on spring boot for testing.
#Regular POD definition:
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp-deployment-6.0
labels:
app: webapp
version: v6.0
spec:
replicas: 2
selector:
matchLabels:
app: webapp
template:
metadata:
labels:
app: webapp
version: v6.0
spec:
containers:
- name: webapp
image: quay.io/shailendra14k/getotp:6.0
imagePullPolicy: Always
ports:
- containerPort: 8180Version 6.0 is Normal and 6.1 has the runtimeclass=kata.
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp-deployment-6.1
labels:
app: webapp
version: v6.1
spec:
replicas: 1
selector:
matchLabels:
app: webapp
template:
metadata:
labels:
app: webapp
version: v6.1
Spec:
runtimeClassName: kata
containers:
- name: webapp
image: quay.io/shailendra14k/getotp:6.1
imagePullPolicy: Always
ports:
- containerPort: 8180
Deploy the application and verify the status:
➜ ~ oc get pods
NAME READY STATUS RESTARTS AGE
webapp-deployment-6.0-5d78fcd8db-ck7g7 1/1 Running 0 11m
webapp-deployment-6.1-6587f8997b-7f5p5 1/1 Running 0 11mCompare the Uptime of Both Containers
#Regular containers:
➜ ~ oc exec -it webapp-deployment-6.0-5d78fcd8db-ck7g7 -- cat /proc/uptime
416625.14 4640515.30#Sandbox containers:
➜ ~ oc exec -it webapp-deployment-6.1-6587f8997b-7f5p5 -- cat /proc/uptime
670.63 658.26You can observe the difference, which is huge, the uptime of the regular container Kernel is the same as the Node kernel(416625.14s= 4.8 Days). However, the Sandbox container kernel uptime is the time of the creation of the Pod(670.63s=11min)
Compare the Process on the Nodes
Log in to the node, where both containers are running. Use the oc debug node/<node-Name>
#Regular containers:
sh-4.4# ps -eaf |grep 10008
1000800+ 852898 852878 0 07:23 ? 00:00:08 java -jar /home/jboss/test.jar1000800+ is the UID for the container.
#Sandbox containers:
First, fetch the sandbox id using the crictl inspect command:
➜ ~oc get pods webapp-deployment-6.1-6587f8997b-7f5p5 -o jsonpath='{.status.containerStatuses[0]}'
{"containerID":"cri-o://b0768d7fbfd2d656b9900ba0b16b6078eb625b412784809ce516f9111a211e10" …..
#From the node
sh-4.4# crictl inspect b0768d7fbfd2d656b9900ba0b16b6078eb625b412784809ce516f9111a211e10 | jq -r '.info.sandboxID'
7740c8967dd6ad50ecd8c31558c3c844bbe7ac4e7ca1115e7f91eec974737270
Fetch the process id using the SandboxId:
sh-4.4# ps aux | grep 7740c8967dd6ad50ecd8c31558c3c844bbe7ac4e7ca1115e7f91eec974737270
root 852850 0.0 0.1 1337556 34816 ? Sl 07:23 0:00 /usr/bin/containerd-shim-kata-v2 -namespace default -address -publish-binary /usr/bin/crio -id
7740c8967dd6ad50ecd8c31558c3c844bbe7ac4e7ca1115e7f91eec974737270
root 852859 0.0 0.0 122804 4776 ? Sl 07:23 0:00 /usr/libexec/virtiofsd --fd=3 -o source=/run/kata-containers/shared/sandboxes/7740c8967dd6ad50ecd8c31558c3c844bbe7ac4e7ca1115e7f91eec974737270/shared -o cache=auto --syslog -o no_posix_lock -d --thread-pool-size=1
root 852865 0.9 1.8 2465200 603492 ? Sl 07:23 0:15 /usr/libexec/qemu-kiwi -name sandbox-7740c8967dd6ad50ecd8c31558c3c844bbe7ac4e7ca1115e7f91eec974737270 -uuid ae09b8a0-1f89-4196-8402-cdcb471675bd -machine q35,accel=kvm,kernel_irqchip -cpu
…… /run/vc/vm/7740c8967dd6ad50ecd8c31558c3c844bbe7ac4e7ca1115e7f91eec974737270/qemu.log -smp 1,cores=1,threads=1,sockets=12,maxcpus=12
root 852873 0.0 0.2 2514884 75800 ? Sl 07:23 0:00 /usr/libexec/virtiofsd --fd=3 -o source=/run/kata-containers/shared/sandboxes/7740c8967dd6ad50ecd8c31558c3c844bbe7ac4e7ca1115e7f91eec974737270/shared -o cache=auto --syslog -o no_posix_lock -d --thread-pool-size=1For the regular container, the process runs directly on the Node host kernel; However, for the Sandbox, the containers run inside the VMs.
Conclusion
Thank you for reading! We saw how the Sandbox containers are deployed on Openshift and its comparison with the regular containers.
Opinions expressed by DZone contributors are their own.