Watch Out for These Five Kubernetes Storage Potholes
Watch Out for These Five Kubernetes Storage Potholes
Starting with Kubernetes and container storage is great, but it will take more than that to complete your storage solution.
Join the DZone community and get the full member experience.Join For Free
Kubernetes and containers have been revelatory for application development, but they’re not the complete solutions enterprises might like them to be. Yes, both address organizations’ long-held desires for agility, efficiency, and accelerated application development. However, there remain some missing elements that may hold back Kubernetes’ explosive adoption rate for mission-critical workloads in the modern enterprise.
All of these elements can be traced back to the fact that neither Kubernetes nor containers feature native storage. Although historically seen as an afterthought by developers, storage remains an essential component for enterprise applications, especially in the age of containers.
For Kubernetes to be successful, it must have the ability to handle stateful applications as well as it does stateless applications. This requires persistent storage, which is necessary in the case of a container failure or simply to ensure that data survives even after the container is spun down.
So while Kubernetes has won as the de facto container application platform, there are still a lot of battles ahead for developers and storage administrators. Here are some things they need to be aware of as they make the move toward Kubernetes and containers.
Microservices and cloud-native applications require a different level of scalability than traditional applications. Using legacy tools to handle this type of volume would be like trying to watch Netflix over a dial-up modem.
Refactoring many of today’s applications could result in hundreds, if not thousands, of microservices running simultaneously. How do you efficiently attach persistent storage to those microservices? How do you ensure data integrity and availability? It’s an intractable problem for both developers and storage administrators. The time and effort required to manually provision that storage is unfeasible. You need a scalable and automated approach, particularly when you go into production.
Persistent storage, when served as a native service of the Kubernetes platform, can be coupled with dynamic provisioning. This can allow developers to easily deploy storage without having to submit a request to a storage administrator. This may help accelerate development since developers do not have to wait hours, days, or weeks to have their requests fulfilled.
Being able to provision your own storage provides you with greater control over your storage options, but it doesn’t turn you into a storage administrator, and chances are you’re just fine with that. Most developers do not list storage as a core area of expertise.
But you can use automation to do a lot of the work for you. Automation can be applied to storage provisioning, the migration of data, and other potentially time-consuming tasks. It can be a great tool to help you get the storage you need without jumping through a lot of hoops.
Fortunately, there are several options, including the open source Rook project and API-based storage. Rook is a persistent storage orchestrator that is designed to run as a native Kubernetes service. Consider it the glue between storage and the container, the thing that makes storage automation work. And API-based storage is fundamental for automation. New era storage systems will rely on APIs in order to meet the demands of new microservice applications.
Deploying container-native storage also allows you to set up automated storage management. Kubernetes itself doesn’t have a storage component, but storage can be automated through the Kubernetes orchestration engine just like many other operational tasks. Enterprise Kubernetes platforms should provide a self-service way for you to request whatever storage (i.e., a persistent volume) is needed for your application. You can also use Kubernetes to mount and add storage to containers and have the engine automatically assign storage resources appropriately.
As Operators, designed via the Kubernetes Operator Framework, increase in development, adoption, and maturity, container storage will also reap benefits in the way of automated installations and upgrades. Thus, Kubernetes can take care of the storage housekeeping, allowing you to focus on development efforts.
It’s notoriously difficult to see exactly what’s going on within the container, and that applies to storage as much as it does to anything else. Again, you probably don’t want to become a storage expert, but you’ll still want to be able to determine how much storage is available for your applications, and how it’s being used.
This can be achieved through enhanced storage monitoring. Open source projects like Prometheus enable deep and advanced container visibility and monitoring. Enterprise Kubernetes platforms should have Prometheus enabled by default to make it easy to give you visibility into your application and storage. Just like checking the storage available on your smartphone, you can get a quick view of exactly what is happening with your storage without having to worry about how it’s happening. It’s storage insight and visibility without having to know how the sausage is being made — just enough information to help you focus on your core job.
The confluence of hyperconvergence and Kubernetes is another key trend that is starting to take shape, to meet the demand of infrastructure operators looking for greater flexibility and control without the complexity of managing large distributed systems.
When it comes to data, security will always be an important consideration. How do you ensure that a container running a certain service or application has access to the data it needs, but only access to that data? How can you make storage readily available and not impose limitations without allowing any application in the container to access the data? How do you define access control lists down to the application when a single application can have multiple users with different administration access levels?
These are all relevant questions that are being addressed by the open source community. Enhanced visibility into the container is a step in the right direction. Developers must also ensure their data, both at rest and in flight, is encrypted to ensure optimal security.
One of the benefits of containerization is application portability across many environments, including public or private clouds or traditional data center infrastructures. But you can’t have portability without standardization, and vice-versa. The challenge is that everyone’s needs and use cases are different, which makes standardization difficult.
Fortunately, the Kubernetes community is working toward ways to better enable hybrid cloud management. While most implementations of Kubernetes adhere to the Cloud Native Computing Foundation (CNCF) software conformance program, there are many additional elements that need to run in production. Those can vary from deployment to deployment, or cloud to cloud. Developers need a standardized abstraction layer that exists between the various iterations of Kubernetes so that they can easily and seamlessly “talk” to each other. This layer can also provide a single storage substrate that unifies storage processes.
This is what the Container Storage Interface (CSI) open source project will bring to the table. The CSI is a standardized mechanism for storage across different container orchestration systems, including Kubernetes, Docker, and others. With the CSI, storage should work uniformly across different container orchestrator systems, regardless of the storage provider you may be using. As always with the open source community, the mandate is “use whatever you want, however you want.” The experience and end result will be the same.
These five potholes exemplify the fact that, while Kubernetes and containers are application game changers, they need an “and” to be more complete solutions that developers will rely on. It must be development “and” storage. It must be Kubernetes “and” scalability. It must be visibility “and” security performing hand-in-hand. And it must be container storage, along with applications “and” data, all working together, natively.
Opinions expressed by DZone contributors are their own.