Using EBS and EFS as Persistent Volume in Kubernetes

If your Kubernetes cluster is running in the cloud on Amazon Web Services (AWS), it comes with Elastic Block Storage (EBS). Alternatively, Elastic File System (EFS) can be used for storage.

We know pods are ephemeral and in most cases, we need to persist the data in the pods. To facilitate this, we can mount folders into our pods that are backed by EBS volumes on AWS using AWSElasticBlockStore, a volume plugin provided by Kubernetes.

We can also use EFS as storage by using efs-provisioner. Efs-provisioner runs as a pod in the Kubernetes cluster that has access to an AWS EFS resource.

In this blog, we will see how to use EBS or EFS as a persistent volume for our Kubernetes cluster in AWS. Before that, we will discuss some common terms and the difference between EBS and EFS.

K8s Clusters in AWS

Kubernetes is an open source system for automating the deployment, scaling, and management of containerized applications. There are multiple ways to create a Kubernetes cluster in AWS. The two most common ways are:

1. The traditional way of installing a master and worker nodes in the EC2 instances.
2. Using the AWS-provided Amazon Elastic Container Service for Kubernetes (EKS), which is a managed service that makes it easy for you to run Kubernetes on AWS without needing maintain your own Kubernetes control plane.

Why Storage Volumes in K8s??

Data in a container are ephemeral, which presents problems for non-trivial applications such as databases when running in containers. First, when a container crashes, Kubernetes will restart the container in a clean state where all the data is lost. Second, when running containers together in a pod, it is often necessary to share files between those containers. The Kubernetes Volume abstraction solves these problems.

Storage Class

A storage class provides a way for administrators to describe the "classes" of storage they offer. Different classes might map to quality-of-service levels or to backup policies or arbitrary policies determined by the cluster administrators.

Storage classes have a provisioner that determines what volume plugin is used for provisioning persistent volumes (PVs). This field must be specified. In our case, for EBS, it will be "kubernetes.io/aws-ebs"— this storage provisioner will take care that a corresponding EBS volume with the correct parameters is created. Data will persist as long as the corresponding PV resource exists. Deleting the resource will also delete the corresponding EBS volume, which means that all stored data will be lost at that point.

For EFS, we will create aN efs_provisioner pod, which manages the EFS resources. The efs-provisioner container reads a configmap, which contains the EFS filesystem ID, the AWS region and the name you want to use for your efs-provisioner. This name will be used later when you create a storage class.

EBS vs. EFS

Persistent Volume in EBS with Default Storage Class

Create a PV with an EBS volume:

Create a PVC to claim the PV:

Persistent Volume in EBS With gp2 Storage Class

Create gp2 the storage class:

Set gp2 storage-class as the default storage-class (this is optional — the storage class can be specified in the PV YAML as well):

Create the PVC using the gp2 storage class:

• Once we create a PVC, the kubelet will automatically create a PV and will be bound with the PVC.
• The PV created is not going to use the existing EBS volume; it will create a new EBS volume.

Persistent Volume in EFS With AWS-EFS Storage Class

• Create EFS (only the first time): It includes the tasks-create the EFS in the right subnets, set up the security groups to allow Kubernetes nodes to access and enable DNS support/resolution in your VPC.
• Create storage class for EFS via efs-provisioner (only the first time): efs-provisioner runs as a container that plays the role of EFS broker. It allows other pods to mount EFS as the persistent volumes. Just be aware that EFS is built on top of NFS4, so you need to have nfs-common packages installed in your Kubernetes nodes.
• Create a PVC to use the storage class for EFS: Just note that EFS has unlimited storage, so the storage size request actually does not take any effects here. But you still have to keep it to pass the syntax check.
• Create a volume for the PVC, then mount the volume inside the pod.
• Download the manifest file yaml.

Conclusion

Handling EBS in AWS has been simplified by the Kubernetes volume plugin. Kubernetes automatically provisions your volume by the type of provisioner provided in the storage class. Even though mounting an EFS as a volume involves more setup, Kubernetes still handles it with ease once all your setup is done.

For more details on EFS provisioner please visit the official site.