DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Because the DevOps movement has redefined engineering responsibilities, SREs now have to become stewards of observability strategy.

Apache Cassandra combines the benefits of major NoSQL databases to support data management needs not covered by traditional RDBMS vendors.

The software you build is only as secure as the code that powers it. Learn how malicious code creeps into your software supply chain.

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

Related

  • The Importance of Persistent Storage in Kubernetes- OpenEBS
  • Which AWS Storage Solution Is Right for Your Elasticsearch Cluster?
  • The Technology Stack Needed To Build a Web3 Application
  • Advanced Kubernetes Deployment Strategies

Trending

  • How to Introduce a New API Quickly Using Micronaut
  • How to Convert XLS to XLSX in Java
  • The Smart Way to Talk to Your Database: Why Hybrid API + NL2SQL Wins
  • How to Use AWS Aurora Database for a Retail Point of Sale (POS) Transaction System
  1. DZone
  2. Software Design and Architecture
  3. Cloud Architecture
  4. Using EBS and EFS as Persistent Volume in Kubernetes

Using EBS and EFS as Persistent Volume in Kubernetes

Take a look at how you can use EBS and EFS in AWS using the Kubernetes volume plugin.

By 
Samson Gunalan user avatar
Samson Gunalan
·
Apr. 23, 19 · Tutorial
Likes (2)
Comment
Save
Tweet
Share
26.4K Views

Join the DZone community and get the full member experience.

Join For Free

If your Kubernetes cluster is running in the cloud on Amazon Web Services (AWS), it comes with Elastic Block Storage (EBS). Alternatively, Elastic File System (EFS) can be used for storage.

We know pods are ephemeral and in most cases, we need to persist the data in the pods. To facilitate this, we can mount folders into our pods that are backed by EBS volumes on AWS using AWSElasticBlockStore, a volume plugin provided by Kubernetes.

We can also use EFS as storage by using efs-provisioner. Efs-provisioner runs as a pod in the Kubernetes cluster that has access to an AWS EFS resource.

In this blog, we will see how to use EBS or EFS as a persistent volume for our Kubernetes cluster in AWS. Before that, we will discuss some common terms and the difference between EBS and EFS.

K8s Clusters in AWS

Kubernetes is an open source system for automating the deployment, scaling, and management of containerized applications. There are multiple ways to create a Kubernetes cluster in AWS. The two most common ways are:

  1. The traditional way of installing a master and worker nodes in the EC2 instances.
  2. Using the AWS-provided Amazon Elastic Container Service for Kubernetes (EKS), which is a managed service that makes it easy for you to run Kubernetes on AWS without needing maintain your own Kubernetes control plane.

Why Storage Volumes in K8s??

Data in a container are ephemeral, which presents problems for non-trivial applications such as databases when running in containers. First, when a container crashes, Kubernetes will restart the container in a clean state where all the data is lost. Second, when running containers together in a pod, it is often necessary to share files between those containers. The Kubernetes Volume abstraction solves these problems.

Storage Class

A storage class provides a way for administrators to describe the "classes" of storage they offer. Different classes might map to quality-of-service levels or to backup policies or arbitrary policies determined by the cluster administrators.

Storage classes have a provisioner that determines what volume plugin is used for provisioning persistent volumes (PVs). This field must be specified. In our case, for EBS, it will be "kubernetes.io/aws-ebs"— this storage provisioner will take care that a corresponding EBS volume with the correct parameters is created. Data will persist as long as the corresponding PV resource exists. Deleting the resource will also delete the corresponding EBS volume, which means that all stored data will be lost at that point.

For EFS, we will create aN efs_provisioner pod, which manages the EFS resources. The efs-provisioner container reads a configmap, which contains the EFS filesystem ID, the AWS region and the name you want to use for your efs-provisioner. This name will be used later when you create a storage class.

EBS vs. EFS

EFS

EBS

Amazon EFS provides a shared file storage for use with compute instances in the AWS cloud and on premise servers.

Amazon EBS is a cloud block storage service that provides direct access from a single EC2 instance to a dedicated storage volume.

Applications that require shared file access can use Amazon EFS for reliable file storage delivering high aggregate throughput to thousands of clients simultaneously.

Application that require persistent dedicated block access for a single host can use EBS as a high available and low-latency block storage solution.

EFS PV provides ReadWriteMany access mode

EBS PV provide only ReadWriteOnce access mode

AN EFS file system can be accessed from multiple availability zones and it is the valuable for multi-AZ cluster

EBS can be accessed by the host it is connected within the zone. EBS volume is automatically replicated within its Availability Zone to protect you from component failure, offering high availability and durability.

It is better to choose EFS when it is difficult to estimate the amount of storage the application will use because EFS is built to elastically scale.

Automatic scaling is not available in EBS but can scaled up down based on the need.

Costly than EBS

Cost efficient

EFS is a file system hence it won’t support some application such as database which require block storage.

Can support all type of application.

EFS doesn’t support any backup mechanism we need to setup backup manually

EBS on the other hand provides point-in-time snapshots of EBS volumes, which are backed up to Amazon S3 for long-term durability.

EFS doesn’t support snapshots.

Amazon EBS provides the ability to copy snapshots across AWS regions, enabling geographical expansion, data center migration, and disaster recovery providing flexibility and protecting for your business.


How to Create PV for EBS

Persistent Volume in EBS with Default Storage Class

Create a PV with an EBS volume:

Create a PVC to claim the PV:

Persistent Volume in EBS With gp2 Storage Class

Create gp2 the storage class:

Set gp2 storage-class as the default storage-class (this is optional — the storage class can be specified in the PV YAML as well):

Create the PVC using the gp2 storage class:

  • Once we create a PVC, the kubelet will automatically create a PV and will be bound with the PVC.
  • The PV created is not going to use the existing EBS volume; it will create a new EBS volume.

Persistent Volume in EFS With AWS-EFS Storage Class

  • Create EFS (only the first time): It includes the tasks-create the EFS in the right subnets, set up the security groups to allow Kubernetes nodes to access and enable DNS support/resolution in your VPC.
  • Create storage class for EFS via efs-provisioner (only the first time): efs-provisioner runs as a container that plays the role of EFS broker. It allows other pods to mount EFS as the persistent volumes. Just be aware that EFS is built on top of NFS4, so you need to have nfs-common packages installed in your Kubernetes nodes.
  • Create a PVC to use the storage class for EFS: Just note that EFS has unlimited storage, so the storage size request actually does not take any effects here. But you still have to keep it to pass the syntax check.
  • Create a volume for the PVC, then mount the volume inside the pod.
  • Download the manifest file yaml.

Conclusion

Handling EBS in AWS has been simplified by the Kubernetes volume plugin. Kubernetes automatically provisions your volume by the type of provisioner provided in the storage class. Even though mounting an EFS as a volume involves more setup, Kubernetes still handles it with ease once all your setup is done.

For more details on EFS provisioner please visit the official site.

Kubernetes AWS Data (computing) Amazon Web Services File system cluster application Manifest file pods

Published at DZone with permission of Samson Gunalan. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • The Importance of Persistent Storage in Kubernetes- OpenEBS
  • Which AWS Storage Solution Is Right for Your Elasticsearch Cluster?
  • The Technology Stack Needed To Build a Web3 Application
  • Advanced Kubernetes Deployment Strategies

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!