This is the second blog in a series of three, in which I expand on some of the points raised in O’Reilly Media’s DevOps for Media & Entertainment report. The first post covered the two essential aspects of DevOps that are often overlooked: communication and empathy. Today, we dive into a more technical topic – monitoring.

Monitoring is essential. It tells you if your service is up, down, fast, slow, and functioning as designed. When something inevitably breaks, a monitoring tool can notify you via alerts and help diagnose the problem. An effective monitoring strategy can allow organizations to reap significant benefits including:

• Protecting revenue, brand, and security.
• Identification of issues before customers are impacted.
• Creating feedback loops and stability.
• Gathering information on usage and usability.
• Experimenting with A/B scenarios.
• Collecting information from real users.

But what exactly are the components of an effective monitoring strategy? The report talks about analyzing log files and tracking system resources such as memory, storage, and processing power. This is a good start but to achieve all the benefits listed above, you have to do more than analyze the systems and logs. A comprehensive monitoring strategy must include synthetic and real user monitoring (RUM).

Applications today are complex with components being delivered by first and third parties, APIs, CDNs, the cloud, and physical data centers. If you’re only monitoring your infrastructure and the content you deliver, issues will be missed. Synthetic monitoring from globally distributed locations enables your organization to test not just the infrastructure but all the additional dependencies. Everything can be fine inside your firewall and with your systems, but users may still be experiencing problems. Monitor the application the same way a user accesses it to fully understand the digital experience.

Users are geographically distributed and access applications across a wide range of devices and connectivity. Synthetic monitoring may not cover all geographies, that’s one place where RUM can help fill in the gaps. RUM collects data from real visitors to your application providing insight into how users are interacting with the site, what paths they are taking through an application, and how the pages are performing. RUM can expand the insights you are collecting via synthetic monitoring and logs.

Collecting monitoring data is the easy part, the harder part is determining which data to collect and ensure that all viewpoints are being included.

Here are 10 tips to create a comprehensive monitoring strategy:

1. Monitor the components and the whole. System level, component level, and overall application metrics need to be included to get the full picture.
2. Analyze first and third party performance. Problems with a third party affect the overall digital experience just as much as problems with first party content.
3. Measure individual pages and multi-step transactions. Users are visiting more than a single page, you should be monitoring more than the home page.
4. Configure alerts to be notified when performance varies from a baseline. Early identification of issues can help resolve problems before customers are impacted.
5. Compare your performance to competitors or industry leaders. Performance is relative, you are being compared to other sites on a daily basis, do you know how you stack up?
6. Monitor from the viewpoint of your users. Capture metrics from real users to get the broadest coverage and use those locations to influence where to capture synthetic measurements from.
7. Measure performance across multiple connection types. Performance and availability can vary widely across connection types include a representative sample of your users.
8. Align metrics with business objectives. Why should others in the organization care about a metric? Describe how the monitoring data is relevant to objectives such as increasing customer loyalty, increasing revenue, or reducing costs.
9. Re-evaluate your strategy on a regular basis. As your company grows, and your application changes, your monitoring strategy should be re-evaluated. Are you still measuring from the geographies that matter? Have new components been introduced that need to be monitored?
10. Look for the anomalies and outliers. We can learn more from the unexpected than from the everyday occurrences.

Creating a monitoring strategy isn’t easy, but the time invested will be worthwhile in the long run.

To learn more about building a comprehensive monitoring strategy, download the DevOps for Media & Entertainment report.