I’ve been waiting a long time to write this blog post. The funny thing is, now that the time is finally here, I’m not really sure how to start.
[This article was written by Nate Heinrich]
Writing this post is like being asked to sum up five seasons of Game of Thrones in 30 seconds. How can you sum up 50 hours of awesome storytelling in 30 seconds? Or in my case, how can you describe our awesome new alerting functionality in “less than 1,000 words?”
The first step is probably to stop talking about TV, and focus on the new cool things that customers can now do in the world of operations and alerting at New Relic. So here we go.
New Relic’s brand new alerting system was built from the ground up with two things in mind:
First: It was designed to give customers a single place to identify and start troubleshooting problems in their software or infrastructure.
Second: It was intended to unlock the operational potential of the New Relic Software Analytics Platform.
So, what do you get when talented New Relic engineers spend a bunch of time building a new alerting system with these goals in mind?
Dedicated experience for alerting and incident response
The New Relic Platform includes a full suite of awesome products, and we wanted to make it easy for customers to configure alerting for all of them.
We also wanted to better support cross-functional and full-stack teams who are responsible for resources ranging from servers, databases, Web applications, and mobile apps, all in one place.
That is why we created alerts.newrelic.com. We hope it will become your one place to manage alerting configuration as well as to see and respond to incidents detected by products across the New Relic Platform.
As of today, the new alerting functionality is integrated with New Relic APM, New Relic Browser, New Relic Mobile, and New Relic Servers, as well as New Relic Plugins! In future releases, we intend to extend this functionality to New Relic Synthetics and New Relic Insights as well.
Each incident gets its own response dashboard to track all related violations until closure. [click to enlarge]
Improved conditions and threshold criteria
The New Relic Platform has experienced rapid growth. We now monitor a wide variety of applications, infrastructures, and architectures. In light of this, creating a highly customized experience that could accommodate an infinite number of alerting scenarios just didn’t make sense. What did make sense was to open everything up and hand the controls over to our customers.
That is why if you jump into the new alerts open beta right now and create an alerting condition, you’ll see many more metrics to alert on with more threshold criteria to control.
Choose a metric, operator, threshold, window type and duration.
Alert on custom metrics.
As you can see, we’ve added more options and controls to allow you to alert on the metrics you likely care about most.
Reduce alert fatigue with incident rollups
Getting alerts sucks, but it’s part of the business of running servers and software. Getting lots of alerts when you are already in the midst of diagnosing or fixing a problem sucks even worse. There you are, trying to focus on the task at hand, and your cell phone just will not shut up. “Buzz, buzz, buzz. Ack, ack, ack.” Super annoying.
We thought there had to be a better way to deal with windows of time where bad stuff is happening to the resources you are responsible for.
In an ideal world, we imagine you’d get one alert per incident, start taking action, and make sure it is resolved, without being hassled by multiple alerts dealing with the same situation.
That’s exactly how New Relic incident alerting works. We roll up all the individual violations that occur after the first alert into a single incident—which we hope will help you keep your sanity and focus on fixing the problem at hand.
More integrated than ever
Whether it’s on-call or escalation needs, chat ops, enterprise event management, or home-brew solutions, alert notifications need to go places. Many, many places. That’s why New Relic now offers first-class integrations with the three of the top SaaS on-call services: PagerDuty, OpsGenie, and VictorOps. We’ve also added our friends over atHipChat and Slack.
What about integrating with “company x” or “service y?” There is a long list of integration requests from our customers, and we plan to continually add new ones to our new alerts platform.
However, we’ve doubled down on webhooks. We think they are super-customizable now, allowing you to do basic auth, custom headers, and even custom payloads. So, even if we don’t yet support a product service directly, our goal is to allow you to craft a webhook to integrate with pretty much anything.
Around now, you might be wondering:“What’s going to happen to the existing alerting functionality sprinkled throughout the products that make up the New Relic Platform?”
Our plan is for the pre-existing functionality to go away in favor of the new unified alerts platform. In the following months we hope to have more information on exactly when and how its end-of-life would occur.
We recommend customers begin using the new alerts immediately, taking advantage of the more flexible alert policy structure to create more meaningful and manageable alerting configurations.
We’re just getting started
Building this new alerts platform took a big effort by some seriously talented folks. New Relic’s goal was to build a solid foundation that would allow us to support all of our products in a consistent and flexible way, without the alert fatigue—and we think we’ve done just that.
What are you waiting for? Come check out our new alerts!
Got questions or feedback on the new alerts? Come chat with us in our new community forum.
For more information see: