Kubernetes Add-On Lifecycle Management With Sveltos

Kubernetes itself is not a complete solution. To build a production cluster, you need various additional add-ons. If you are managing multiple clusters, that is not an easy task.

Sveltos is an open-source project that provides declarative APIs allowing the deployment of Kubernetes add-ons across multiple Kubernetes clusters. Both Helm charts and resource YAMLs can be passed to Sveltos.

For instance, just posting the following ClusterProfile instance is sufficient to deploy the Kyverno Helm chart in a set of managed clusters:

Or if you have Kubernetes resource YAMLs, just create a ConfigMap (or Secret if data is confidential) with the data and then have Sveltos reference such ConfigMap (Secret) instance.

kubectl create configmap contour-gateway --from-file=<File with contour YAML>

To make things more complex, some of the time, the add-ons that you need to deploy depend on the cluster run-time state.

For instance, you have deployed Calico v3.24 in a set of clusters. As those clusters get upgraded to Kubernetes v1.25, you want Calico to be upgraded to v3.25 as well.

If you are managing tens of such clusters, manually upgrading Calico when the Kubernetes version is upgraded is not ideal. You need an automated solution for that.

Then simply create the following Classifier instances:

Above Classifier instances will have Sveltos manage Cluster labels by automatically adding:

• label kubernetes: v1–24 to any cluster running Kubernetes version v1.24.x
• label kubernetes: v1–25 to any cluster running Kubernetes version v1.25.x.

Because of those labels and the above ClusterProfile instances:

• calico version v3.24.5 will be deployed in any cluster running Kubernetes version v1.24.x
• calico version v3.25.0 will be deployed in any cluster running Kubernetes version v1.25.x

No action is required on your side. As clusters are upgraded, Sveltos will upgrade Calico as well.

Things might get more complicated when the add-ons need to be deployed as a result of an event in a managed cluster. For instance, any time a Service in a certain namespace is created, adds an HTTPRoute to expose such a service via Gateway API.

Sveltos Events is an event-driven workflow automation framework for Kubernetes which helps you trigger K8s add-on deployments on various events.

1. Define what an Event is (Sveltos supports Lua script for that);
2. Define what add-ons to deploy when such an event happens. Add-ons can be expressed as templates, and Sveltos will instantiate those at deployment time using information from managed clusters.

This EventSource is defining an event as the creation/deletion of a Service in the namespace eng exposing either port 443 or port 8443.

When such an event happens in a managed cluster, we want to deploy an HTTPRoute instance:

As you can see, what to deploy is defined in a ConfigMap, and it is expressed as a template.

Resources contain all the Service instances in each managed cluster matching the EventSource defined above.