Is Namespace-As-A-Service the Evolution of Cluster-As-A-Service?

Choosing a Namespace-as-a-Service strategy will incur cost and development time upfront; cluster-as-a-service defers operational complexity and cost until cluster count grows. However, as cluster counts grow in large organizations, perhaps Namespace-as-a-Service is an evolution of Cluster-as-a-Service.

Kubernetes Cluster-As-A-Service (k8sCaaS)

Each user or team is provided with a cluster that they manage and control. They don’t undertake the installation and build process, which is automated, and depending on the level of development of the k8sCaaS service, additional standardized components and other necessary capabilities such as cluster access and storage are also provisioned. Once provisioned, the day-to-day operation of the cluster is undertaken by the user. The users have the equivalent of superuser access and the most or all of the responsibility for operational security. In some cases, there is oversight from a k8sCaaS team; in other cases, none.

k8sCaaS is usually provisioned on Virtualization. Virtualization provides an easy way to create the nodes necessary to create a cluster, therefore making the cluster and nodes available on demand. Also, hosts can be segmented to provide nodes for more than one user cluster. Two layers of resource sharing are in place with this model, the Container runtime and virtualization.

Kubernetes Namespace-As-A-Service (k8sNaaS)

Each user or team is provided with one or more namespaces on a existing cluster. The cluster is already created and is shared among users; isolation is provided using RBAC and, optionally, other tools. Within the cluster, users can have access to a variety of infrastructure and platform/tool resources. The cluster is managed, not managed by the users; that management is undertaken by the k8sNaaS team. Users simply load their workloads and request platform resources such as storage or access networking provided by the cluster. If two teams need to collaborate, they can provide access to workloads within their namespaces.

k8sNaaS is more often provisioned on Bare Metal hardware. The resource sharing mechanism is the Container Runtime. Virtualization is not required as nodes are added by the k8sNaaS team as demand increases.

The Need for a k8s Platform Strategy

A few years ago, I was engaged by a large European company to build a Namespace-as-a-Service Kubernetes (k8sNaaS) platform. There was already another organization with a Cluster-as-a-Service (k8sCaaS) platform, in addition to Public Cloud, where Cluster-as-a-Service is the Kubernetes solution. At the time, Namespace-as-a-Service was hard to build, especially compared to the Cluster-as-a-Service, which was a packaged product from the Openstack provider and available from Cloud Providers. The k8sNaaS team did a fantastic job developing the platform on bare metal, but it required significant development; there were lots of components that needed to be modified or didn’t exist and needed to be developed. I suspect that after I left the team, unfortunately, the k8sNaaS project was shut down. When I left, the company had more than 300 k8sCaaS clusters and recently boasted that they now have 3x more. The k8sNaaS project was ahead of its time.

It’s worth noting that I spoke with many of the current and future users of k8s at that company; the reality is that there were some who wanted the k8sCaaS because they wanted to control as much as possible, and were willing to do it. There was an equal or greater number who wanted k8sNaaS with as many platform components as possible, so they didn’t need to manage the k8s clusters and could focus on their application projects.

K8s Platform Strategy

Today, what do I think the platform strategy should be? The answer is that you have to do both. Choosing only one solution will increase costs and complexity but, most importantly, have a very high opportunity cost.

This means developing a Cluster-as-a-Service platform that includes Namespace-as-a-Service capabilities and tooling. This leverages that the initial user of the provisioned k8sCaaS will develop tooling that supports their use case, and it's very likely that that tooling is useful for other teams. By providing consistent k8sNaaS tooling and sharing the cluster, other teams benefit from that development and further increase functionality instead of repeating development tasks that have already been undertaken by other teams multiple times.

Having an array of k8sNaaS platforms that evolved from k8sCaaS built for specific use cases captures development IPR, reduces development time for new applications, and focuses and maximizes the infrastructure development effort. A successful strategy should also result in a manageable number of appropriately sized clusters, not lots of very small clusters or a small number of very large clusters.

The initial investment in this strategy is a design pattern that differs from simply installing clusters. It starts by separating the automation used to create the controller nodes and the worker nodes. The controller nodes need to scale independently of the worker nodes and ensure that a larger audience can use the cluster tools, some of that will be specialized for k8sNaaS. The node automation needs to accommodate the possible significant growth of the cluster by supporting both VM based nodes, and bare metal nodes as demands increase or specific use cases such as ML are implemented. Many of k8s standard tools are designed to be cluster-wide resources, in particular LoadBalancers and Ingresses; this issue is addressed with the new GatewayAPI that supports namespaced API Gateways. (for example, the EPIC k8s Gateway) Components deployed using Operators can also be problematic as they require access to all namespaces. Care is also required in establishing secure, reliable control-plane access, ensuring that the cluster can operate with potentially a larger number of users with different security postures while providing cluster stability—finally, a uniform mechanism to access shared storage is potentially also accessible without transiting the a k8s POD.

The “evolution” strategy requires more development effort than a turnkey vendor based Cluster-as-a-Service. However, the effort is less than launching a Namespace-as-a-Service platform that includes all of the features for the organization. In the “evolution” model, k8s adopters are leveraged to build on the platform foundation, guiding the development of those cluster to meet the various needs of different parts of the organization.