Unlocking the Potential of Observability With AI

Observability is essential for developing and running modern distributed applications, but fragmented tools and data often obstruct critical insights. AI and unified observability can overcome these challenges.

Observability is crucial for modern software development, allowing developers to monitor, troubleshoot, and optimize complex distributed applications. However, many organizations struggle to achieve effective observability due to data silos, complex monitoring tools, and fragmented insights. 

Observe aims to overcome these challenges by providing a unified observability platform powered by a graph data layer. I recently spoke with Observe CEO Jeremy Burton to learn more about their approach and how it helps developers improve observability.

Unifying Observability Data

A key challenge with existing observability solutions is that data remains siloed across different tools. As Burton explained, traditionally, companies have used specialized tools for metrics, tracing, and logs that don't interoperate. This fragmentation forces developers, DevOps engineers, and SREs to manually piece together insights. 

Observe tackles this by ingesting and correlating all observability data — metrics, traces, and logs — into a single platform. Their graph data layer links related data points together, providing context and speeding up troubleshooting. Users can start from any data type and pivot across others for a unified view.

According to Burton, "You can attack things at a different point of entry. You can attack things a bit more top-down and bottom-up." Rather than chasing down IDs across dashboards, developers can navigate by logical entities like customers and services.

Observe maintains all raw data in an affordable object store data lake. However, their graph indexes and transforms commonly queried data for fast interactive analysis. This powers rapid troubleshooting while allowing users to fetch older data on demand.

Optimizing Kubernetes and Cloud-Native Apps

Observe provides extensive support for containerized and cloud-native environments like Kubernetes and AWS. The platform auto-discovers infrastructure topology and maps raw Kubernetes data into concepts like pods and containers. 

As Burton noted, "We transform the data into things that people recognize." This accelerates Kubernetes monitoring and troubleshooting by presenting data in familiar terms. Developers can go directly to impacted containers and services during incidents.

Observe also auto-instruments customer applications by scanning code for context like customer IDs. Burton explained how this helped Topgolf quickly resolve issues with their games by linking logs to specific bays. These logical mappings simplify troubleshooting for distributed cloud-native apps.

Leveraging AI and Machine Learning

Observe uses AI techniques like conversational interfaces and code generation to enhance the user experience. Burton sees AI as the key to making observability feel more intuitive.

Their O11y GPT chatbot leverages large language models to understand natural language queries, guide troubleshooting, and generate data transformations. Users can describe problems in plain terms rather than memorizing query syntax. 

Observe also trained Codex to automatically generate data parsing and analysis code in their Opal query language. This co-pilot capability allows engineers unfamiliar with Opal to be productive immediately.

As Burton noted, modern applications have made troubleshooting highly complex, so AI can help "eliminate 130 minutes of difference" in the meantime to resolution. By leveraging machine learning to capture expertise, Observe aims to make observability more accessible.

Improving Economics and Customer Experience

While providing richer functionality, Observe is engineered for cloud scale and economics. Their cloud-native architecture takes advantage of affordable storage and compute. This allows retaining high-resolution observability data for up to 30 months to aid in deep troubleshooting.

Observe also integrates tightly with collaboration tools like Slack. Burton explained how surfacing alerts in incident channels and providing an AI assistant improves coordination and reduces mean time to resolution.

For customers like Blooma, Observe has delivered strong outcomes. Blooma's Director of Technical Operations, Jason Huling, reported dramatically faster troubleshooting and no platform degradation despite a 10x data increase. He attributed this to Observe's ease of use and stellar customer support.

For customers like Reveal, Observe has delivered fast results. As Reveal's Director of Engineering, Stephen Montoya, noted, "We move so fast here like it's a rocket ship over here. We didn't have time to have to devote to really learning Observe. It was easy to learn right out of the box." He also praised Observe's stellar customer support.

The Future of Observability 

When asked about the observability market outlook, Burton highlighted the potential for AI to redefine interactions and blur organizational barriers. He envisions developers initiating and driving incident response via collaboration tools, with machine learning suggesting fixes in real-time.

Observe's investments in applied AI aim to make observability seamless. Burton believes this can reduce the skill gap by codifying tribal knowledge into systems engineers can conversationally query. Integrated and proactive observability will enable developers to focus on higher-value tasks.

Overall, Observe's unified observability platform aims to help engineers better understand and optimize modern applications. Their innovative data architecture provides interconnected insight across metrics, traces, and logs. Combined with usability enhancements like AI, Observe strives to make observability effortless. This enables developers to spend less time firefighting and more time innovating.