Observability for AI Agents and Multi-Agent Systems: When Your System Can't Tell You Why It Did That

The bug report was received as a customer complaint. An AI agent responsible for managing vendor onboarding had sent a rejection email to a supplier the company had been trying to close for three months. Nobody had authorized it. Nobody had configured it to reject vendors in that category. The agent autonomously made the decision after analyzing a compliance document and cross-referencing it with an internal policy database. By the time the complaint arrived, the reasoning chain that produced the decision had been discarded. The agent had no memory of why it did what it did. The logs showed the action but not the thought.

That story is fictional in its specifics but accurate in its structure. This phenomenon represents a class of problems that teams deploying AI agents in production are encountering with increasing frequency: the agent performed an action, the output is visible, but the intermediate reasoning, including the sequence of context retrievals, model calls, tool invocations, and decisions that led to the output, is either absent, incomplete, or stored in a format that renders post hoc investigation nearly impossible. Traditional observability was not designed for systems that exhibit cognitive processes.

Why Agent Observability Is Structurally Different

Conventional service observability is built around a relatively stable model: a request enters a system, passes through a defined set of operations, and produces a response. The execution path may be complex, but it's deterministic and bounded. You can instrument each step, correlate the signals with a trace ID, and reconstruct exactly what happened for any given request.

AI agents break this model in at least three ways. First, the execution path is not determined at design time — it emerges from the agent's reasoning. An agent deciding which tools to call, in what order, based on what it reads in a retrieved document, is making structural decisions at runtime that a static trace can't fully capture. The spans exist, but the semantic reason a particular branch was taken lives inside a model call that returned natural language, which most tracing systems treat as an opaque blob.

Second, agent systems frequently involve state that persists across requests: memory stores, retrieved context, and conversation history, which means the behavior of the system at time T is partially determined by things that happened at times T-1 through T-n. Debugging a poor decision often requires reconstructing not just the current request but the accumulated state that shaped it. Most observability stacks are not built for these scenarios.

Third, multi-agent systems introduce the problem of causal attribution across agent boundaries. When Agent A passes a task to Agent B, which delegates a subtask to Agent C, which calls a tool that returns erroneous data, and that incorrect data propagates back up the chain to produce a wrong output from Agent A, the causal chain is real but fragmented across three separate execution contexts. Without deliberate design, you'll have three separate traces with no shared context that links them.

The Minimum Viable Agent Trace

The starting point for any serious agent observability implementation is defining what the minimum viable trace looks like for a single agent execution. In practice, this means capturing five things that standard OpenTelemetry spans don't cover by default.

The first is the full prompt context, not just the user message but the complete input to each model call, including the system prompt, retrieved documents, tool outputs injected into the context, and the conversation history. The information is costly to store and verbose, but you need it to understand the model's reasoning. Sampling helps here: store full prompt context for a percentage of executions, prioritizing those that result in high-stakes actions or errors.

The second is the model's reasoning output before tool calls. If your agent framework supports it, capture chain-of-thought or scratchpad outputs of the model's intermediate reasoning before it decides to call a tool or produce a final answer. This is the closest thing to a stack trace for a reasoning system. Without it, you can see that a tool was called but not why.

The third is a tool called "provenance" for each tool invocation, recording not just the inputs and outputs but which part of the reasoning chain triggered it. Fourth is the agent's decision points: moments where the agent chose between multiple possible actions. Fifth is cross-agent delegation context: when one agent hands off to another, the receiving agent's trace must carry a reference to the delegating agent's trace ID.

# Minimal agent span instrumentation using OpenTelemetry
from opentelemetry import trace
import json

tracer = trace.get_tracer('agent.core')

def traced_model_call(agent_id, prompt_context, step_label):
    with tracer.start_as_current_span(f'agent.model_call.{step_label}') as span:
        span.set_attribute('agent.id',          agent_id)
        span.set_attribute('agent.step',        step_label)
        # Store truncated prompt for cardinality control
        span.set_attribute('agent.prompt_hash', hash(str(prompt_context)))
        span.set_attribute('agent.prompt_len',  len(str(prompt_context)))

        # Full prompt stored separately in blob storage, keyed by trace+span ID
        store_prompt_context(
            trace_id=format(span.get_span_context().trace_id, '032x'),
            span_id =format(span.get_span_context().span_id,  '016x'),
            context =prompt_context
        )

        response = call_model(prompt_context)
        span.set_attribute('agent.output_len',  len(response))
        span.set_attribute('agent.tool_calls',  extract_tool_calls(response))
        return response

The pattern above separates high-cardinality content (the full prompt) from the trace span itself, storing it in blob storage keyed by trace and span IDs. This keeps the tracing backend manageable while preserving the ability to retrieve full context for any specific execution. The prompt hash allows you to detect when two executions were given identical contexts, which is useful for identifying cases where the same input produced different outputs, which is a diagnostic signal in itself.

Multi-Agent Correlation: The Delegation Chain Problem

Here's where things got genuinely complicated in a system I was involved with: we had three agents — a planning agent, a research agent, and a writing agent that collaborated on generating reports. Each was instrumented individually and produced clean traces. But when a report came out wrong, reconstructing which agent's decision caused the problem required manually cross-referencing three separate trace trees, none of which had a shared parent.

The fix was implementing what we called a "workflow ID," a UUID generated at the entry point of any multi-agent task and propagated explicitly to every agent that participated in that task, regardless of how many hops away from the origin they were. This workflow ID was added as a span attribute on every agent span and as a field in every log line produced during the task. With it, querying all spans and logs associated with a single end-to-end agent workflow became a single filter, not a manual correlation exercise.

# Propagating workflow context across agent boundaries
from dataclasses import dataclass
from opentelemetry import trace, context, propagate

@dataclass
class AgentWorkflowContext:
    workflow_id:    str   # stable across all agents in a task
    parent_agent:   str   # which agent delegated this task
    delegation_depth: int # how many hops from the origin agent

def delegate_to_agent(target_agent, task, wf_ctx: AgentWorkflowContext):
    child_ctx = AgentWorkflowContext(
        workflow_id      = wf_ctx.workflow_id,   # same ID propagates
        parent_agent     = wf_ctx.parent_agent,
        delegation_depth = wf_ctx.delegation_depth + 1
    )
    span = trace.get_current_span()
    span.set_attribute('workflow.id',    child_ctx.workflow_id)
    span.set_attribute('workflow.depth', child_ctx.delegation_depth)
    span.set_attribute('workflow.parent_agent', child_ctx.parent_agent)
    return target_agent.run(task, child_ctx)

The delegation depth attribute turned out to be more useful than expected. In one debugging session, seeing that a particular tool call was happening at delegation depth 4 — four hops from the original request immediately flagged that the agent system had gone significantly deeper into a recursive subtask chain than intended. Without that attribute, the trace looked like any other tool call.

Semantic Logging: What Happened vs. Why

Standard logging captures what happened. For agent systems, you also need to capture the agent's stated reasoning at key decision points. This doesn't require exotic infrastructure; it requires a logging discipline that treats the model's reasoning output as a first-class log field rather than as data to be discarded after use.

In practice, this means that when an agent produces a reasoning step leading to a significant action — such as calling an external tool, delegating to another agent, producing a final output, or deciding to abandon a task — the full reasoning text should be logged alongside the action. Tag it with the workflow ID, the agent ID, and a decision type label. This produces a semantic audit trail that lets you answer the question, "Why did the agent do X?" without having to reconstruct it from indirect evidence.

The objection is storage cost, and it's legitimate. Reasoning outputs from LLMs are verbose. Storing them for every execution at scale is expensive. The practical answer is tiered retention: store full reasoning logs for executions that result in errors, high-stakes actions (anything that sends an external communication, modifies a record, or triggers a financial transaction), or random sampling of normal executions for baseline calibration. For the rest, store only the decision label and the action taken. This keeps costs manageable while preserving investigative capability for the cases that matter.

What I'd Do Differently

In hindsight, the single most important decision to make before deploying an agent in production is defining what a 'high-stakes action' means for that specific agent and ensuring those actions always produce full semantic logs regardless of cost. Initially, we did not define logging requirements; instead, we treated logging as uniform across all action types, which resulted in issues when an agent took an unexpected external action, and we lacked a reasoning log to explain it.

I'd also invest earlier in a replay capability: the ability to take a logged prompt context and re-run the agent over it with a modified model or prompt configuration to verify that a fix actually changes the behavior that caused a problem. Without a replay capability, any changes you make are based on hope rather than verification. With it, you can verify that the reasoning path actually differs before deploying.

When should you not build this level of observability? If you're prototyping or running an agent in a low-stakes, easily reversible context, the overhead of full semantic logging and workflow ID propagation is probably premature. Build it before you go to production with consequential actions, not after. The cost of retrofitting it once an unexplained agent decision has already caused a real problem is significantly higher than building it in from the start.

Key Takeaways

Standard distributed tracing captures what happened in agent systems but not why. Semantic logging of reasoning outputs at decision points is the missing layer; treat it as first-class infrastructure, not optional verbosity.

Propagate a workflow ID across all agents in a multi-agent task. Without it, correlating signals across agent boundaries requires manual effort that fails under incident pressure.

Separate high-cardinality prompt content from trace spans. Store the full prompt context in blob storage keyed by trace and span ID, and reference it from the span. This preserves investigative capability without bloating your tracing backend.

Please define high-stakes actions prior to deployment and ensure they consistently generate complete semantic logs. The executions you most need to investigate are exactly the ones where missing reasoning context is most detrimental.

Conclusion

Observability for AI agents is not a solved problem. The tooling ecosystem is immature, the standards are still forming, and most teams are improvising solutions on top of infrastructure designed for deterministic services. That's not a reason to skip it; it's a reason to be deliberate about what you build, because the defaults will leave you blind at exactly the wrong moment.

The deeper challenge is that agent observability isn't just a technical problem. It's also an accountability problem. When an AI agent takes a consequential action, someone needs to be able to answer the question of why, not just for debugging purposes, but for the humans affected by the decision and for the organization responsible for the system. A vendor who received a rejection email deserves a better answer than "the agent decided that." The infrastructure to produce that answer has to be designed in, not bolted on.

The open question I keep returning to: as agent systems become more capable and their reasoning chains longer and more complex, at what point does the volume and opacity of their decision-making exceed our practical ability to observe and understand it? We may be building systems that are genuinely difficult to audit, not because of missing tooling but because of fundamental limits on human comprehension of long reasoning chains. What does accountability look like then?