How to Watch Kubernetes Events on Slack

Microservices applications are made of many moving parts. In Kubernetes Land, these applications are made of various resources (i.e., deployments, services, pods, configmaps, volumes, secrets, etc.). At a large scale, many potential failures exist. Keeping an eye on all the moving parts is a real challenge. 

With ChatOps being embraced in an increasing number of enterprises, it's becoming important to be able to watch your microservices applications in your chat application. In this article, we present a simple way to get Kubernetes notifications sent to your Slack channel.

They are many monitoring solutions already for container-based application frameworks. Kubewatch stands out as a light weight alerting system that leverages the Kubernetes API. Check out the GitHub repository and join the fun.

You run kubewatch in your Kubernetes cluster. It communicates directly with the Kubernetes API server and uses the watch APIs for all the resources. It then sends notifications to configured handlers every time that a resource changes.

The resources that you watch can be configured with a simple configuration file. The following example shows a configuration file where we only turn on watch for Kubernetes services and pods.

resource:
      deployment: false
      replicationcontroller: false
      replicaset: false
      daemonset: false
      services: true
      pod: true

Currently, kubewatch only handles the notifications to Slack. You set up a dedicated Slack channel and a Slack API token, and kubewatch sends the notifications to that channel for you to see.

Running kubewatch is as easy as grabbing a ConfigMap and Pod manifest from the repository and kube-ing it.

$ kubectl create -f kubewatch-configmap.yaml
$ kubectl create -f kubewatch.yaml

Your Slack channel will soon get populated with your k8s events:

After the initial burst of notifications, you will only receive new ones. 

It is yet another small yet powerful tool for Kubernetes. While only focusing on Slack, it can be extended to any other system and powered up with advanced controllers to perform actions based on event triggers.

If you want handlers for HipChat, PagerDuty, or any other system, please don't hesitate to file an issue on GitHub or even better submit a pull request.