DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

How does AI transform chaos engineering from an experiment into a critical capability? Learn how to effectively operationalize the chaos.

Data quality isn't just a technical issue: It impacts an organization's compliance, operational efficiency, and customer satisfaction.

Are you a front-end or full-stack developer frustrated by front-end distractions? Learn to move forward with tooling and clear boundaries.

Developer Experience: Demand to support engineering teams has risen, and there is a shift from traditional DevOps to workflow improvements.

Related

  • 8 Must-Have Project Reports You Can Use Today
  • 5 Kibana Visualizations To Spice Up Your Dashboard
  • Monitoring Apache Ignite Cluster With Grafana (Part 1)
  • Enhancing Business Decision-Making Through Advanced Data Visualization Techniques

Trending

  • The Scrum Guide Expansion Pack
  • Modernizing Financial Systems: The Critical Role of Cloud-Based Microservices Optimization
  • How to Master a DevSecOps Pipeline that Devs and AppSec Love
  • Why Whole-Document Sentiment Analysis Fails and How Section-Level Scoring Fixes It
  1. DZone
  2. Data Engineering
  3. Data
  4. Setting Thresholds With Quantiles

Setting Thresholds With Quantiles

The idea is to look at the shape of your data and use that to decide where you feel is the right threshold. One of the best ways to do that is to plot the quantiles.

By 
B Jones user avatar
B Jones
·
Nov. 29, 15 · Tutorial
Likes (4)
Comment
Save
Tweet
Share
6.0K Views

Join the DZone community and get the full member experience.

Join For Free

i was talking with someone the other day about a visualization i remembered seeing some years ago, that could help set a reasonable value for a threshold on a metric. as i’ve written , thresholds are basically a broken way to monitor systems, but if you’re going to use them, i think there are simple things you can do to avoid making threshold values completely arbitrary.

i couldn’t find the place i’d seen the visualization (if you know prior art for the below, please comment!) so i decided to just blog about it. suppose you start off with a time series:

time series

the idea is that you might want to alert on this metric breaking some threshold, but what’s the right value? there are lots of ways you could do it: a multiple of the average, some quantile, some number of standard deviations from the mean, etc, etc. for example, you could say “i want to alert if the metric exceeds its usual 99.9th percentile.” but isn’t 99.9 an arbitrary number, too? what makes it so special? is there any way to pick a number that isn’t just pulled out of a hat?

i need to preface all of this with a disclaimer. everyone’s systems are different, static thresholds are silly, quantiles are known to the state of california to cause cancer, and so on. what i’m about to show you is only slightly less arbitrary. don’t confuse it for a rule that actually has a strong reason why it’s better than the alternatives.

the idea is to look at the shape of your data and use that to decide where you feel is the right threshold. the “shape” is the distribution of the data–how its values are typically spread.

one of the best ways to do that is to plot the quantiles. in the following plot, the quantiles go from left to right. at the far left is the 0th percentile, and at the far right is the 100th percentile. another way to say that is the left is the minimum value and the right is the maximum value.

quantiles

i’ve made no effort to polish this chart; you’ll notice that i generated the quantiles by taking a vector of 1000 numbers and dividing it by 1000, for example.

now, as you look at this chart, you can see that near the far right, it suddenly jumps–it has an elbow. let’s zoom in on the last 10% of the chart, i.e. the 90th percentile and above:

95th percentile

there’s nothing magical about this elbow. there’s nothing magical about the last 10 percent of values. it just so happens that those last few are disproportionately larger than the rest. in simple terms, it means that the system’s model, or perhaps the parameters for its model, apparently changed. and if you’re going to pick a place to alert on a threshold, maybe a point where the behavior diverges rapidly is as good as any and better than some.

in other words, you could put your threshold just at the point where the quantile plot gets steeper, which is about the value 1600 in this dataset.

not all data behaves this way. some metrics will have a nice line all the way up and to the right, with no elbow. some will jump sooner. some will have a big ledge, even several ledges. you’ll get all kinds of different shapes and sizes of data. the point is to at least know what shapes and sizes your own data has.

visualizations like this have a lot of explanatory power and can show you potentially surprising things very easily. that’s why this kind of visualization, rather than just the standard boring time series plot, is not a bad thing to know how to do. it’s not revolutionary, things like this are used in all sorts of ways and by lots of people. qq plots , for example, are a related technique.

so i’m probably repeating myself too much, but again this is nothing special, but it’s something anyway. with this approach, there still may not be a strong justification for choosing a number as a threshold, but at least there’s a reason and method, and it’s better than cargo cult, a random number generator, or copy-and-paste.

here’s the r code.

perf <- read.csv("/path/to/perf.csv", sep="")
plot(perf$tput)
plot(c(1:1000), quantile(perf$tput, probs=c(1:1000)/1000))
plot(c(900:1000), quantile(perf$tput, probs=c(900:1000)/1000))

related: someone on twitter pointed me to this blog post from dynatrace .

Visualization (graphics) Data (computing) IT Metric (unit) Time series Percentile Chart Blog

Published at DZone with permission of B Jones, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • 8 Must-Have Project Reports You Can Use Today
  • 5 Kibana Visualizations To Spice Up Your Dashboard
  • Monitoring Apache Ignite Cluster With Grafana (Part 1)
  • Enhancing Business Decision-Making Through Advanced Data Visualization Techniques

Partner Resources

×

Comments

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • [email protected]

Let's be friends: