Kubernetes-Native Database Essentials

As more organizations shift operations to the cloud, the benefits of going cloud-native continue to accrue. One critical task in managing a cloud-based microservices architecture is container orchestration. Kubernetes is the container orchestrator of choice for 78 percent of IT Leaders identified in Red Hat’s “State of Open Source 2020” report. Kubernetes is a powerful and flexible tool, but certain issues remain that can make managing stateful applications difficult. Simply put: storage on Kubernetes is not a solved problem.   

This is highly dependent, however, on what kind of database you use alongside Kubernetes. Traditional relational databases that give life — or state — to any application were not architectured to take full advantage of the resources available in the cloud. On the other side, NoSQL databases fail to thrive in Kubernetes-orchestrated environments. A new breed of relational databases — Distributed SQL databases — have emerged over the past few years and are able to take full advantage of the cloud and the operational benefits provided by Kubernetes.  

This is a preview of the Kubernetes-Native Database Essentials Refcard. To read the entire Refcard, please download the PDF from the link above.

Kubernetes is an open-source project, originally created by Google, that allows you to “orchestrate” containerized services. It automates the deployment, scaling, and management of services and applications. This capability, while seemingly simple, delivers huge value to organizations, especially as they scale out their cloud-native initiatives and have to manage and operate hundreds or thousands of services. It delivers a wide range of capabilities that help keep apps and services up and running to scale individual services and meet user demand. 

As a direct descendant of Borg (the Google internal orchestration platform), Kubernetes brings Google-like power to operations. It simplifies and eases delivery for applications architected to take advantage of the resources available in cloud environments. Every public cloud provider offers Kubernetes as a platform.  

This is a preview of the Kubernetes-Native Database Essentials Refcard. To read the entire Refcard, please download the PDF from the link above.

Before we get into the framework for evaluating to what extent a database is Kubernetes-native, it is important first to understand why state and storage are still issues in Kubernetes. By definition, databases are considered stateful applications. Kubernetes, on the other hand, was originally intended to facilitate the management of stateless applications connected to external services to store state, if necessary. The guarantees which Kubernetes offers — for example, high availability of services — were premised on the idea that the underlying pods were effectively ephemeral and interchangeable.  

 Kubernetes manages containers dynamically, meaning that individual containers may be brought up or taken down at any time to match the needs of the application as a whole. This works fine for stateless services, but a stateful, persistent service like a database cannot afford to be restarted arbitrarily. In the past, developers often chose to run databases outside Kubernetes, which was problematic because it required redundant infrastructure management tools for the external database, such as monitoring and service discovery.  

In the early days of Kubernetes, it was an open question whether it would even be possible to run a database in a Kubernetes cluster. Then StatefulSets came along. StatefulSets and related features and methods make running a database in Kubernetes possible. But they don’t necessarily make it easy.

This is a preview of the Kubernetes-Native Database Essentials Refcard. To read the entire Refcard, please download the PDF from the link above.

DASH is a framework used to evaluate Kubernetes-native operations and help assess whether or not a database is Kubernetes-native. DASH stands for the four Kubernetes-native operations a distributed database must provide in order to truly work with Kubernetes: Disposability, API symmetry, shared nothing, and horizontal scaling. The rest of this guide will uncover what these principles mean, and whether or not they work with traditional relational databases, NoSQL databases, and Distributed SQL databases.  

DASH Principles  

• Disposability: Losing things should be a non-event. Because a Kubernetes-native database can handle failures, any disruptions are entirely hidden from the client. 

• API symmetry: Every server in the network provides the same answer to the same question. Though they are distributed, Kubernetes-native databases act as a single logical database with the consistency guarantees of a single-machine system.  

• Shared-nothing: True Kubernetes-native database can operate without any centralized coordinator or single point of failure.  

• Horizontal scaling: Scale up, not out.   

To make these concepts concrete, we walk through the architectures of NoSQL databases, traditional relational databases, Distributed SQL databases, and Kubernetes itself. Let’s get started! 

This is a preview of the Kubernetes-Native Database Essentials Refcard. To read the entire Refcard, please download the PDF from the link above.

DASH provides a necessary framework for evaluating whether a database delivers a truly Kubernetes-native architecture. 

• Disposability enables resilience, ensuring your stateful systems can survive when ephemeral cloud resources cease to exist.  

• API symmetry enables consistency, allowing distributed databases to always provide the up-to-date answer, no matter which process is handling the client request. 

• Shared-nothing properties enable your database to make forward progress without any centralized master or coordinator. 

• Horizontal scalability allows the database to take advantage of the unlimited and on-demand resources available in the cloud. 

Kubernetes-native databases give IT teams an automated database that operates as an always-on, elastic data layer that adds the missing cloud-native foundation to their stacks. 

This is a preview of the Kubernetes-Native Database Essentials Refcard. To read the entire Refcard, please download the PDF from the link above.