Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

The Depth of Terraforming a Kubernetes Cluster

DZone's Guide to

The Depth of Terraforming a Kubernetes Cluster

Explore the possibilities when you combine the configuration tool Terraform with the cloud management platform Kubernetes.

· Cloud Zone ·
Free Resource

See why enterprise app developers love Cloud Foundry. Download the 2018 User Survey for a snapshot of Cloud Foundry users’ deployments and productivity.

Terraform has become one of the leading tools to manage infrastructure. It allows you to write, plan, and create infrastructure as code using declarative configuration files, shared amongst different team members. From its creation, it has widely been used to provision low-level infrastructure both on-premise and in the cloud.

Image titleKubernetes, on the other hand, is a container orchestration platform running on top of on-premise or cloud infrastructure. It also exposes a rich set of APIs to effectively automate container deployment, scaling, and management. See this wiki page for an overview of Kubernetes.

Terraform for Kubernetes

So you might wonder how Terraform fits with Kubernetes. With version 0.10, Terraform came out with a Kubernetes provider supporting Kubernetes from K8S 1.6 version onwards. This allowed Terraform to provision the Kubernetes Cluster and its containers with different levels of configurations.

There are many advantages in using Terraform to provision Kubernetes Cluster:

  • Allows maintaining Kubernetes Cluster definitions in Code.
  • Modify Kubernetes Cluster configurations through variables.
  • Modularize the infrastructure in code.
  • Dry run the modifications to the Kubernetes Cluster before actually applying them.
  • With the dry run feature, it is possible to observe the execution plan with detailed insight on resource modifications with visibility on how it affects to the Kubernetes Cluster.

Terraform Depth of Responsibility

When using Terraform to define a Kubernetes infrastructure, it is important to understand where we draw the line between Kubernetes and Terraform.

Pods

When setting up a Kubernetes workload, it is possible to use Terraform to directly schedule the pods. After Terraform provisions the pod, Kubernetes is responsible for managing the containers within.

Image title

Replication Controller

If you use Terraform to provision the replication controller, Kubernetes is responsible for managing the underneath pods and containers. This makes the pods out of reach to Terraform and pod failures and recovery are managed by Kubernetes.

Image title

Horizontal Pod Autoscaler

For more dynamic applications, if you use Terraform to provision the Pod Autoscaler, it will also require provisioning Replication Controller with Terraform keeping Pods and Containers to be still managed by Kubernetes.

Image title

However, one of the conflicts arising with this Terraform configuration is that both Pod AutoScaler and Replication Controller can impact the running number of Pod replicas since a minimum and maximum is defined in Pod Autoscaler while an initial number is defined in Replication Controller.

To deal with the overlap, it is possible to define a lifecycle block in Replication Controller resource definition to ignore the changes happening to replicas from outside by the Pod AutoScaler preventing the conflicts.

Storage, Persistence Volume, and Claim

Storage and Persistence Volumes can be directly provisioned using Terraform. However, it is common practice that Storage Class is provisioned using Terraform while allowing Volumes to be managed by Kubernetes for better scalability and control. A similar approach is taken for Persistence volumes and claims as well.

Services

When provisioning Services in Kubernetes, it is possible to provision the Load Balancers using Terraform. But for more dynamic workloads with a timely need of pods as well as dynamic assignments of IPs or Hostnames for Load Balancers it is convenient to allow Terraform to manage the Service while Load Balancers are provisioned by Kubernetes.

Conclusion

When using Terraform to provision your Kubernetes cluster, it is important to keep in mind that Terraform is not a silver bullet in solving all the DevOps problems you have. It is also a responsibility of us to make sure we identify what are the resources and at which level we provision using Terraform. In addition, using Terraform itself has a learning curve not only to learn Terraform but also to apply the best practices keeping Terraform configuration DRY and maintainable with right modularity. For more details on Terraform best practices refer the Terraform Recommended Practices from HashiCorp.

Cloud Foundry saves app developers $100K and 10 weeks on average per development cycle. Download the 2018 User Survey for a snapshot of Cloud Foundry users’ deployments and productivity. Find out what people love about the industry standard cloud application platform.

Topics:
kubernates ,terraform ,best practices ,cloud ,cloud managment ,cloud tools ,containerization

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}