13 Best Practices for Using Helm
Helm is a tool for deploying applications to Kubernetes clusters. Here are 13 best practices to help you create, operate, and upgrade applications using Helm.
Join the DZone community and get the full member experience.
Join For FreeTake Your Helm Charts to the Next Level
Helm is the package manager for Kubernetes. It reduces the effort of deploying complex applications thanks to its templating approach and rich ecosystem of reusable and production-ready packages, also known as Helm charts. With Helm, you can deploy packaged applications as a collection of versioned, pre-configured Kubernetes resources.
Let’s assume you’re deploying a database with Kubernetes — including multiple deployments, containers, secrets, volumes, and services. Helm allows you to install the same database with a single command and a single set of values. Its declarative and idempotent commands make Helm an ideal tool for continuous delivery (CD).
Helm is a Cloud Native Computing Foundation (CNCF) project created in 2015 and graduated in April 2020. With the latest version of Helm 3, it has become even more integrated into the Kubernetes ecosystem.
This article features 13 best practices for creating Helm charts to manage your applications running in Kubernetes.
1. Take Advantage of the Helm Ecosystem
Helm gives you access to a wealth of community expertise — perhaps the tool’s greatest benefit. It collects charts from developers worldwide, which are then shared through chart repositories. You can add the official stable chart repository to your local setup as follows:
$ helm repo add stable https://charts.helm.sh/stable "stable" has been added to your repositories Then you can search for charts, for example, MySQL: $ helm search hub mysql URL CHART VERSION APP VERSION DESCRIPTION https://hub.helm.sh/charts/bitnami/mysql 8.2.3 8.0.22 Chart to create a Highly available MySQL cluster https://hub.helm.sh/charts/t3n/mysql 0.1.0 8.0.22 Fast, reliable, scalable, and easy to use open-...
You will see a long list of results, which shows how big the Helm chart ecosystem is.
2. Use Subcharts to Manage Your Dependencies
Because applications deployed to Kubernetes consist of fine-grained, interdependent pieces, their Helm charts have various resource templates and dependencies. For instance, let’s assume your backend relies on a database and a message queue. The database and message queue are already standalone applications (e.g. PostgreSQL and RabbitMQ). Creating or using separate charts for the standalone applications and adding them to the parent charts is therefore recommended. The dependent applications are named as sub-charts here.
There are three essential elements for creating and configuring sub-charts:
1. Chart structure The folder structure should be in the following order:
backend-chart - Chart.yaml - charts - message-queue - Chart.yaml - templates - values.yaml - database - Chart.yaml - templates - values.yaml - values.yaml
2. Chart.yaml
Additionally, the chart.yaml in the parent chart should list any dependencies and conditions:
apiVersion: v2 name: backend-chart description: A Helm chart for backend ... dependencies: - name: message-queue condition: message-queue.enabled - name: database condition: database.enabled
3. values.yaml
Finally, you can set or override the values of sub-charts in the parent chart with the following values.yaml file:
message-queue: enabled: true image: repository: acme/rabbitmq tag: latest database: enabled: false
Creating and using sub-charts establishes an abstraction layer between the parent and dependency applications. These separate charts make it easy to deploy, debug, and update applications in Kubernetes with their separate values and upgrade lifecycles. You can walk through the folder structure, dependencies, and value files in a sample chart like bitnami/wordpress.
3. Use Labels to Find Resources Easily
Labels are crucial to Kubernetes’ internal operations and the daily work of Kubernetes operators. Almost every resource in Kubernetes offers labels for different purposes such as grouping, resource allocation, load balancing, or scheduling.
A single Helm command will allow you to install multiple resources. But it’s vital to know where these resources originate. Labels enable you to find your resources created by Helm releases quickly.
The most common method is to define labels in helpers.tpl
, like so:
{{/* Common labels */}} {{- define "common.labels" -}} app.kubernetes.io/instance: {{ .Release.Name }} app.kubernetes.io/managed-by: {{ .Release.Service }} {{- end -}}
You then need to use the “include” function with labels in the resource templates:
apiVersion: apps/v1 kind: Deployment metadata: name: my-queue labels: {{ include "labels" . | indent 4 }} ...
Now you should be able to list all resources with the label selectors. For example, you can list all the pods of my-queue deployment with the kubectl get pods -l app.kubernetes.io/instance=[Name of the Helm Release]
command. This step is essential for locating and debugging those resources managed by Helm.
4. Document Your Charts
Documentation is essential for ensuring maintainable Helm charts. Adding comments in the resource templates and the README helps teams with the development and use of Helm charts.
You should use the following three options to document your charts:
- Comments: Template and values files are YAML files. You can add comments and provide helpful information about the fields inside the YAML files.
- README: A chart’s README is a markdown file that explains how to use the charts. You can check the contents of a README file with the following command:
helm show readme [Name of the Chart]
- NOTES.txt: This is a special file located at
templates/NOTES.txt
that provides helpful information about the deployment of releases. The content of theNOTES.txt
file can also be templated with functions and values similar to resource templates:
You have deployed the following release: {{ .Release.Name }}. To get further information, you can run the commands: $ helm status {{ .Release.Name }} $ helm get all {{ .Release.Name }}
At the end of the helm install or helm upgrade command, Helm prints out the content of the NOTES.txt like so:
RESOURCES: ==> v1/Secret NAME TYPE DATA AGE my-secret Opaque 1 0s ==> v1/ConfigMap NAME DATA AGE db-configmap 3 0s NOTES: You have deployed the following release: precious-db. To get further information, you can run the commands: $ helm status precious-db $ helm get all precious-db
5. Test Your Charts
Helm charts consist of multiple resources that are to be deployed to the cluster. It is essential to check that all the resources are created in the cluster with the correct values. For instance, when deploying a database, you should check that the database passwords are set correctly.
Fortunately, Helm offers a test functionality to run some containers in the cluster in order to validate applications. For example, the resource templates annotated with "helm.sh/hook": test-success
are run by Helm as test cases.
Let’s assume you are deploying WordPress with the MariaDB database. The Helm chart maintained by Bitnami has a pod to validate the database connection with the following definition:
... apiVersion: v1 kind: Pod metadata: name: "{{ .Release.Name }}-credentials-test" annotations: "helm.sh/hook": test-success ... env: - name: MARIADB\_HOST value: {{ include "wordpress.databaseHost" . | quote }} - name: MARIADB\_PORT value: "3306" - name: WORDPRESS\_DATABASE\_NAME value: {{ default "" .Values.mariadb.auth.database | quote }} - name: WORDPRESS\_DATABASE\_USER value: {{ default "" .Values.mariadb.auth.username | quote }} - name: WORDPRESS\_DATABASE\_PASSWORD valueFrom: secretKeyRef: name: {{ include "wordpress.databaseSecretName" . }} key: mariadb-password command: - /bin/bash - -ec - | mysql --host=$MARIADB\_HOST --port=$MARIADB\_PORT --user=$WORDPRESS\_DATABASE\_USER --password=$WORDPRESS\_DATABASE\_PASSWORD restartPolicy: Never {{- end }} ...
It is recommended to write tests for your charts and to run them after the installation. For example, you can use the helm test <RELEASE_NAME>
command to run tests. The tests are a valuable asset for validating and finding issues in applications installed with Helm.
6. Ensure Your Secrets Are Secure
Sensitive data, such as keys or passwords, are stored as secrets in Kubernetes. Although it is possible to secure secrets on the Kubernetes side, they are mostly stored as text files as part of Helm templates and values.
The helm-secrets plugin offers secret management and protection for your critical information. It delegates the secret encryption to Mozilla SOPS, which supports AWS KMS, Cloud KMS on GCP, Azure Key Vault, and PGP.
Let’s assume you’ve collected your sensitive data in a file named secrets.yaml as follows:
postgresql: postgresqlUsername: postgres postgresqlPassword: WoZpCAlBsg postgresqlDatabase: wp
You can encrypt the file with the plugin:
$ helm secrets enc secrets.yaml Encrypting secrets.yaml Encrypted secrets.yaml
Now, the file will be updated and all values will be encrypted:
postgresql: postgresqlUsername: ENC\[AES256\_GCM,data:D14/CcA3WjY=,iv...==,type:str\] postgresqlPassword: ENC\[AES256\_GCM,data:Wd7VEKSoqV...,type:str\] postgresqlDatabase: ENC\[AES256\_GCM,data:8ur9pqDxUA==,iv:R...,type:str\] sops: ...
The data in secrets.yaml above was not secure and helm-secrets solves the problem of storing sensitive data as part of Helm charts.
7. Make Your Chart Reusable by Using Template Functions
Helm supports over 60 functions that can be used inside templates. The functions are defined in the Go template language and Sprig template library. Functions in template files significantly simplify Helm operations.
Let’s look at the following template file as an example:
apiVersion: v1 kind: ConfigMap metadata: name: {{ .Release.Name }}-configmap data: environment: {{ .Values.environment | default "dev" | quote }} region: {{ .Values.region | upper | quote }}
When the environment value is not provided, it will be defaulted by the template function. When you check the region field, you’ll see there is no default value defined in the template. However, the field has another function called upper to convert the provided value into uppercase.
Another essential and useful function is required
. It enables you to set a value as required for template rendering. For instance, let’s assume you need a name for your ConfigMap with the following template:
... metadata: name: {{ required "Name is required" .Values.configName }} ...
If the entry is empty, the template rendering will fail with the error Name is required. Template functions are very useful when creating Helm charts. They can improve templating, reduce code duplication, and can be used to validate values before deploying your applications to Kubernetes.
8. Update Your Deployments When ConfigMaps or Secrets Change
It is common to have ConfigMaps or secrets mounted to containers. Although the deployments and container images change with new releases, the ConfigMaps or secrets do not change frequently. The following annotation makes it possible to roll out new deployments when the ConfigMap changes:
kind: Deployment spec: template: metadata: annotations: checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }} ...
Any change in the ConfigMap will calculate a new sha256sum
and create new versions of deployment. This ensures the containers in the deployments will restart using the new ConfigMap.
9. Opt-Out of Resource Deletion With Resource Policies
In a typical setup, after installing a Helm chart, Helm will create multiple resources in the cluster. You can then upgrade it by changing values and adding or removing resources. Once you no longer need the application, you can delete it, which removes all resources from the cluster.
Some resources, however, should be kept in the cluster even after running Helm uninstall. Let’s assume you’ve deployed a database with PersistentVolumeClaim and want to store the volumes even if you are deleting the database release. For such resources, you need to use the resource-policy annotations as follows:
kind: Secret metadata: annotations: "helm.sh/resource-policy": keep ...
Helm commands, such as uninstall, upgrade, or rollback would result in the deletion of the above secret. But by using the resource policy as shown, Helm will skip the deletion of the secret and allow it to be orphaned. The annotation should therefore be used with great care, and only for the resources needed after Helm Releases have been deleted.
10. Useful Commands for Debugging Helm Charts
Helm template files come with many different functions and multiple sources of values for creating Kubernetes resources. It is an essential duty of the user to know what is deployed to the cluster. Therefore, you need to learn how to debug templates and verify charts. There are four essential commands to use for debugging:
- helm lint: The linter tool conducts a series of tests to ensure your chart is correctly formed.
- helm install — dry-run — debug: This function renders the templates and shows the resulting resource manifests. You can also check all the resources before deployment and ensure the values are set and the templating functions work as expected.
- helm get manifest: This command retrieves the manifests of the resources that are installed to the cluster. If the release is not working as expected, this should be the first command you use to find out what is running in the cluster.
- helm get values: This command is used to retrieve the release values installed to the cluster. If you have any doubts about computed or default values, this should definitely be in your toolbelt.
11. Use the Lookup Function to Avoid Secret Regeneration
Helm functions are used to generate random data, such as passwords, keys, and certificates. Random generation creates new arbitrary values and updates the resources in the cluster with each deployment and upgrade. For example, it can replace your database password in the cluster with every version upgrade. This causes the clients to be unable to connect to the database after the password change.
To address this, it is recommended to randomly generate values and override those already in the cluster. For example:
{{- $rootPasswordValue := (randAlpha 16) | b64enc | quote }} {{- $secret := (lookup "v1" "Secret" .Release.Namespace "db-keys") }} {{- if $secret }} {{- $rootPasswordValue = index $secret.data "root-password" }} {{- end -}} apiVersion: v1 kind: Secret metadata: name: db-keys namespace: {{ .Release.Namespace }} type: Opaque data: root-password: {{ $rootPasswordValue}}
The template above first creates a 16-character randAlpha value, then checks the cluster for a secret and its corresponding field. If found, it overrides and reuses the rootPasswordValue as root-password.
12. Migrate to Helm 3 for Simpler and More Secure Kubernetes Applications
The latest Helm release, Helm 3, offers many new features to make it a lighter, more streamlined tool. Helm v3 is recommended for its enhanced security and simplicity. This includes:
- Removal of Tiller: Tiller was the Helm server-side component but has been removed from v3 due to the exhaustive permissions required to make changes on the cluster in earlier versions. This also created a security risk, as anyone gaining access to Tiller would have excessive permissions to your cluster.
- Improved chart upgrade strategy: Helm v2 relies on a two-way strategic merge patch. It compares the new release with the one in the ConfigMap storage and applies the changes. Conversely, Helm v3 compares the old manifest, the state in the cluster, and the new release. So your manual changes will not be lost while upgrading your Helm releases. This simplifies the upgrade process and enhances the reliability of the applications.
There is a helm-2to3
plugin, which you can install with the following command:
$ helm3 plugin install https://github.com/helm/helm-2to3
It is a small but helpful plugin with cleanup, convert, and move commands to help you migrate and clean up your v2 configuration and create releases for v3.
13. Keep Your Continuous Delivery Pipelines Idempotent
Kubernetes resources are declarative in the sense that their specification and status are stored in the cluster. Similarly, Helm is required to create declarative templates and releases. Therefore, you need to design your continuous delivery and release management to be idempotent while using Helm. An idempotent operation is one you can apply many times without changing the result following the first run.
There are two essential rules to follow:
- Always use the
helm upgrade --install
command. It installs the charts if they are not already installed. If they are already installed, it upgrades them. - Use
--atomic
flag to rollback changes in the event of a failed operation during helm upgrade. This ensures the Helm releases are not stuck in the failed state.
Summary
Helm is an indispensable tool for deploying applications to Kubernetes clusters. But, it is only by following best practices that you’ll truly reap the benefits of Helm.
The best practices covered in this article will help your teams create, operate, and upgrade production-grade distributed applications. From the development side, your Helm charts will be easier to maintain and secure. From the operational side, you’ll enjoy automatically updated deployments, save resources from deletion, and learn how to test and debug.
Helm’s official topics guide is another good resource for checking the Helm commands and understanding their design philosophy. With these resources as well as the best practices and examples outlined in this blog, you’ll surely be armed and ready to create and manage production-grade Helm applications running on Kubernetes.
For our latest insights and updates, find us on LinkedIn.
Published at DZone with permission of Kentaro Wakayama. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments