Microservices Monitoring and Critical Incident Management

Wolfgang Beer, Technical Product Manager at Dynatrace, co-wrote this article.

Microservices can be game-changing if, as Martin Fowler says and Adam Drake explains, you have rapid provisioning, basic monitoring, and rapid deployment already in place. And when microservices meet containers, they can boost software engineering power to a whole new level. Together, they form architectures that act like living, breathing entities and are much more adaptable than in the past.

But an ensemble of microservices is far more complex to understand, let alone troubleshoot, when it comes to performance. Often hosted in modern cloud platforms such as AWS, Azure, or OpenStack, microservices are dynamically started and scaled depending on actual demands and traffic. As useful as this process is, managing availability, detecting errors, and identifying performance problems become especially demanding for DevOps teams.

These rapidly changing environments and dynamically scaling services mean that the right responders must be notified especially fast when things go wrong, and we need to separate out the critical, actionable alerts, versus shooting over a firehose full of noise.

Fortunately, Dynatrace and VictorOps have a few ideas for how to achieve this goal and give your DevOps teams some relief.

Dynatrace: Full-Stack Monitoring With Artificial Intelligence

First, you need the right notifications. Dynatrace automatically detects all of those microservice dynamic infrastructure changes and learns how the entire service environment normally behaves. The system catches each individual transaction, from your application user action to your backend services and databases.

Then Dynatrace puts all that topological and transactional data into context and uses AI algorithms and analytics to detect the root-cause of complex incidents. What is interrelated? What are baselines versus anomalies that warrant alarms? Without that deep transactional and code-level visibility, it would be impossible for DevOps teams to pinpoint what’s causing errors, slowdowns or even outages.

The screenshot below shows how Dynatrace automatically identifies a CPU spike as the root-cause of web application slowdowns. The "problem details" card also shows the business impact the detected problem causes in terms of impacted real users that were using your web application in the moment of the problem and how many service calls into the backend were also affected.

The attached "visual resolution path" shows the topological dependencies that were discovered while following the problem impacts.

Despite the fact that Dynatrace delivers such in-depth automated analysis about your environment, it’s mission critical to receive problem notifications through a reliable channel such as VictorOps.

Integrating Dynatrace With VictorOps Adds More Intelligence

Next, it’s time to add intelligent categorization, routing, and remediation instructions to the incoming notifications. Enter VictorOps. Whereas Dynatrace detects problems in real-time, VictorOps gives you the tools to create flexible on-call schedules and add intelligence to the incident lifecycle.

By integrating Dynatrace with VictorOps, you can now apply logic to help the right alerts get to the right people. Via the Incident Automation Engine, you can set up VictorOps to do things like:

• Indicate the level of severity of each incoming notification, so you’re only alerted when something is critically wrong, separating the signal from the noise.
• Route the specific alert to the right responder so the expert closest to the problem can solve it faster.
• Deliver remediation steps alongside alerts, to assist with resolution.

Together, Dynatrace and VictorOps speed time to resolution. The intelligence built into each system alleviates some of the stress, false alarms, and frequent burnout that DevOps and on-call teams experience.

Anonymous Dynatrace Customers Say:

“We have been using Dynatrace for over 5 years, and find it an indispensable tool during pre-release functional testing, pre-release load testing, and especially post-production troubleshooting of severity one issues. With a breadth of distributed platforms for key application environments, Dynatrace gives us near-real-time (within a matter of seconds) analysis of end-to-end transactions that are spread across multiple servers and multiple layers of the stack…”
(Source: Gartner peer insights)

“Dynatrace has been spectacular to work with. Technology-wise, we use it primarily for root-cause analysis and performance management from an infrastructure perspective, as opposed to APM. But we’re beginning to use it for more comprehensive APM now, and it’s proving very helpful. Relationship-wise, the Dynatrace team is one of the best I’ve worked with in my 20 years in IT. They view their customer relationship as a true partnership.” – IT Architect
(Source: Gartner peer insights)

Bring More Intelligence to Microservices Monitoring

Does this sound good to you? If you’re curious, take Dynatrace for a free 15-day test drive. See VictorOps in action. And if you already use both systems, follow these steps to install the VictorOps/Dynatrace integration. Then, please give us feedback on your experience.