DZone
Performance Zone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
  • Refcardz
  • Trend Reports
  • Webinars
  • Zones
  • |
    • Agile
    • AI
    • Big Data
    • Cloud
    • Database
    • DevOps
    • Integration
    • IoT
    • Java
    • Microservices
    • Open Source
    • Performance
    • Security
    • Web Dev
DZone > Performance Zone > Monitoring Kubernetes, Part 4: The Sensu-Native Approach

Monitoring Kubernetes, Part 4: The Sensu-Native Approach

In the fourth part of this series, take a look at how you can use Sensu to monitor Kubernetes inside and out.

Sean Porter user avatar by
Sean Porter
·
Jul. 10, 19 · Performance Zone · Analysis
Like (2)
Save
Tweet
6.82K Views

Join the DZone community and get the full member experience.

Join For Free

This is part 4 in a series on monitoring Kubernetes and Docker. Catch up on parts 1, 2, and 3.

Photo by Nareeta Martin on Unsplash


At this point in our series, you're likely quite familiar with the many opportunities and challenges that Kubernetes presents (especially when it comes to monitoring!). The last couple of posts take at a look at Prometheus for monitoring Kubernetes, with a side-by-side comparison with Sensu, and illustrate how they work in tandem. In this post, I'll show you the Sensu-native approach to monitoring Kubernetes, as well as walk you through monitoring your applications running on Kubernetes, using examples from the sensu-kube-demo repository.

Monitoring Kubernetes With Sensu

While Sensu + Prometheus is a great option (more on that in part 3), you can also use Sensu on its own to monitor Kubernetes. Several folks in our community are already doing so — whether that's an OG approach with Sensu 1.x (see this post from Community Member Andy Repton and watch Lee Briggs' Sensu Summit 2018 talk) or with Sensu Go (which, as it just so happens, is designed with Kubernetes in mind — e.g., our RBAC and sensuctl, the command-line tool). The patterns I'll delve into apply to both Sensu 1.x and Go (and you'll see our demo repo has example configs for both) but we'll focus on Go for our examples, since Sensu 1.x is reaching EOL at the end of this year.

Of the data sources I mention in part 1, kube-state-metrics gives you most of what you need in terms of the Kubernetes cluster state, without inundating you with too much information. (It's a polarizing topic, but there is such a thing as too much data; my experience has taught me that a curated approach to monitoring makes better use of resources and combats alert fatigue.) Kube-state-metrics collects information on the current state of your containers. There are three ways to plug this information into your monitoring event pipeline:

  • Use the sidecar pattern to deploy a Sensu agent alongside your application container. Kubernetes pods can contain the container (say that five times fast!) running your application along with other containers running supportive applications such as the Sensu agent. These all share the same network space so your applications can talk to Sensu as if they were running in the same container or host. The agent can simultaneously talk to localhost in the same fashion — for example, if it's in the pod with a web app, it can check localhost to make sure it's up and running and healthy.
  • Run a daemonset for the Sensu agent that runs the Sensu agent on Kubernetes. This daemonset will deploy a Sensu agent per Kube node that can also do these responsibilities.
  • Run a Sensu agent on the Kube host (the actual physical host or virtual machine) that runs all the containers. It can inspect the Kubernetes services to make sure they're running as well as collect data from kube-state-metrics, local services exporting metrics, and a local node exporter (if you're running one). In this use case, you'd be running the Sensu agent outside of Kubernetes.

The most flexible way for monitoring Kubernetes is the sidecar pattern. My advice is: use the sidecar pattern for your application monitoring and use a Sensu agent on all of the Kube hosts for monitoring Kubernetes itself.

If you have full control of the hardware, run Sensu outside of Kubernetes so you can inspect the state of Kubernetes safely. You shouldn't monitor Kubernetes from Kubernetes, because if Kubernetes goes down, so does your monitoring.

For an example of monitoring Kubernetes with Sensu, check out our sample app, which walks you through setting up multitenancy, creating a Sensu pipeline to Slack and InfluxDB, and collecting Kubernetes metrics.

How to Monitor Your Applications Running on Kubernetes

In addition to monitoring Kubernetes itself, you'll want to keep tabs on your applications running on Kubernetes. The aforementioned daemonset and sidecar method comes into play: in the preceding section I outline how you can run a Sensu agent daemonset which would set an agent on every Kubernetes host. You can make that agent's API available to local containers, so that your applications can emit data to that daemonset agent. Another approach is to have that daemonset agent discover and monitor services on that particular host — for example, you discover several web servers running on your localhost, and check HTTP against them. If they go away, but the container is still scheduled, then there's a failure.

If you go this route, we want to hear about it! Join us in Slack and Discourse to share what you've learned with the community.

Going Forward: A Multi-Generational Future

I hope you found this series useful in terms of starting to reason about monitoring Kubernetes and containerized infrastructure. As infrastructure grows increasingly complex — spanning multiple generations of technology — it's more critical than ever that we keep pace in terms of our tooling and practices. One of our main goals at Sensu is to truly future proof your monitoring, preparing you for what's next while bridging the gap from legacy infrastructure, to give you total visibility — from Kubernetes to bare metal.

Join Sensu CEO Caleb Hailey for a webinar on July 10 @ 10am PT, where he'll demo monitoring Kubernetes workloads and clusters.

Kubernetes Docker (software) application Host (Unix)

Published at DZone with permission of Sean Porter, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • A BGP Guide for the Non-Network Engineer
  • Back to Basics: Accessing Kubernetes Pods
  • What Are Protocol Buffers?
  • Total Bummer: Pivotal Drops Groovy

Comments

Performance Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • MVB Program
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends:

DZone.com is powered by 

AnswerHub logo