What Actually Makes AI Infrastructure Agents More Reliable (It's Not More Agents)
Single AI agents fail during incidents. Four specialized agents — supervisor, telemetry, reasoning, action — handle observability more reliably.
·
Join the DZone community and get the full member experience.
Join For FreeI keep seeing the same pattern. Someone builds an "AI agent" for infrastructure monitoring — it answers questions about Prometheus metrics, pulls logs from ELK, suggests restarts. Impressive in a demo. Then you push on it: what happens when its logs query times out mid-investigation? What happens when the context window fills up while correlating signals across four systems? What happens when a tool call hallucinates a metric name that doesn't quite exist?
Usually it doesn't fail catastrophically. It fails quietly, in ways that are hard to debug. And quiet failures during incident response are the worst kind.
I've spent the last year prototyping multi-agent architectures for infrastructure observability — coming at it from a decade of SRE and network reliability work. My working hypothesis going in was straightforward: split the investigation across specialized agents, and the reliability problems that plague a single agent — a blown context window, a hallucinated tool call — should ease. Below is the architecture pattern I built to test that, the failure modes I watched it run into, and — since I've since put the hypothesis through a more rigorous test than a demo — what actually held up.
Why a Single Agent Hits a Ceiling
A good on-call engineer doesn't open one dashboard and stare. They move between tools, check Prometheus, scan logs, look at deploy history, consult runbooks. Each step informs the next. Investigations have structure.
A single LLM agent trying to replicate that workflow runs into two real constraints.
First, the context window. Every tool call, every metric result, every log snippet goes into that window. Short investigation: fine. Anything complex — multiple services, ambiguous signals, a failure mode the model hasn't seen — and the window fills. Early observations get pushed out. The model loses the thread.
Second, the tool problem. The more tools you give a single agent, the more likely it is to hallucinate one — invoking a function that doesn't exist, or constructing a query with the right name but the wrong parameter. I've reproduced this in my own prototyping: the agent confidently calls a metric query with a typo, gets an empty result, and concludes the metric doesn't exist rather than that the query was wrong.
Split the work across specialized agents, and both pressures ease. Each agent owns a smaller tool set it actually knows. Each agent has a manageable context. And when one agent fails — they will, eventually — it fails in a bounded, debuggable way.
The Architecture: Four Roles, One Investigation
The pattern I keep returning to has four agent roles, communicating through shared state rather than direct message-passing. The state is a typed Python object accumulating findings as the investigation progresses. No agent starts from scratch; each picks up where the last one left off.
class InvestigationState(TypedDict):
incident_id: str
trigger: AlertTrigger
telemetry_findings: list[Finding]
causal_hypothesis: Optional[Hypothesis]
recommended_actions: list[Action]
confidence_score: float
audit_trail: list[AgentStep]Every agent reads from and writes back to this state. That single design choice — shared structured state instead of free-form message-passing — is what makes the system auditable.
The four roles:
Supervisor Agent receives the raw alert. It doesn't investigate — it routes. It classifies the incident, identifies the services involved, and decides which specialist agents to invoke.
Telemetry Investigation Agent is the data-gathering specialist. Given an investigation context, it runs queries against the observability stack — Prometheus, Grafana, ELK, AppDynamics — finds anomalies, and returns structured findings. It doesn't explain what it finds. It just finds things.
def route_to_specialists(state: InvestigationState) -> list[str]:
trigger_type = state["trigger"].classification
if trigger_type == "network":
return ["telemetry", "reasoning"]
elif trigger_type == "application":
return ["telemetry", "logs", "reasoning"]
else:
return ["telemetry", "logs", "infra", "reasoning"]Reasoning Agent takes the Telemetry Agent's findings and tries to answer: what is actually going on? Given a RAG index over historical post-mortems, it can reason like "this pattern resembles a connection-pool exhaustion failure mode I've seen documented before." When it works, the experience is impressive. When it's wrong, it's confidently and eloquently wrong — a class of failure I'll come back to.
Action Agent turns a hypothesis into something executable. For low-risk actions, it could, in principle, act autonomously once confidence crosses a threshold. For anything riskier, it drafts a recommendation with full context and routes to a human for approval. I'd treat that human-in-the-loop gate as non-negotiable for any first deployment.
A Worked Example (Prototype, Not Production)
# Investigation trace — synthetic test environment
# Alert received: payments-service p99 latency anomaly
# Supervisor → routing to Telemetry Agent
# Telemetry: db-proxy connection pool utilization elevated
# Telemetry: db-proxy deployed recently (within last 20 min)
# Telemetry: no downstream dependency anomalies
# Reasoning: hypothesis — connection-pool regression in recent deploy
# Action: recommend rollback to prior db-proxy version
# Action: draft escalation with evidence → human approval requiredThe point of the multi-agent system isn't to replace the engineer. It's to do the legwork before the human even opens their laptop, so the human is reviewing evidence rather than gathering it.
Where This Pattern Breaks
Three failure modes worth naming:
Confidence scores are not well-calibrated. An 87% confidence score sounds authoritative. Language models don't express uncertainty the way a careful engineer would. Any deployment needs a conservative threshold for autonomous action and a generous fallback to human review.
Context grows faster than you expect. Five services, 15 tool calls of data in shared state, and the Reasoning Agent starts dropping things. State summarization helps, but it's lossy.
Agent observability is its own problem. You're building a system that monitors infrastructure, and now you need to monitor the monitor. Without per-step tracing, debugging an agent failure is genuinely painful.
What a Rigorous Test Actually Showed
Everything above is prototype-stage reasoning — the kind you form watching a system work and fail in front of you. I didn't want to leave it there, so I ran the architecture against two real fault-injection benchmarks, AIOps Challenge 2020 and RCAEval, 75 incidents each, across six pre-registered configurations comparing the four-role design against a well-built single agent given the same tools and the same context budget.
Decomposition alone didn't win. Across both benchmarks, the multi-agent version came out statistically indistinguishable from the single agent (McNemar's test, p > 0.05 in every configuration), and a plain rule-based baseline stayed competitive with both. That's not the result I expected going in, and it's worth sitting with rather than explaining away: splitting an investigation into roles does not, by itself, make it more accurate.
What did move the needle was a narrower idea: a Falsifier agent that checks the Reasoning agent's hypothesis against evidence it wasn't shown, instead of taking the hypothesis at its word. That improved accuracy on single-service incidents meaningfully — 33.3% vs. 21.3%, p = 0.023 — and made multi-service incidents worse at first — 24.0% vs. 42.7%, p < 0.001 — because with no notion of which services depend on which, the falsifier mistook a downstream symptom for the root cause. Giving it real service-topology data closed that gap. Then the less comfortable check: I gave the same falsifier to a single agent instead of the four-role pipeline. It scored indistinguishably from the multi-agent version (p = 1.0). The gain wasn't coming from decomposition. It was coming from the verification step, and the verification step doesn't care how many agents are asking the question.
This work is accepted at CNSM 2026 (IFIP); the full benchmark, raw results, and eval scripts are in the repo linked below, and the falsifier design specifically is written up in more depth in the preprint linked at the end of this article.
Implementation Notes
LangGraph fits this pattern well. The explicit graph model lets you define exactly what happens after each agent step. The graph is code — versionable and testable.
For tool management, typed schemas validated before execution eliminate most hallucinated tool calls. The discipline is the same regardless of framework: every tool input is typed, every tool call is validated, nothing executes on an unstructured string.
If I had to give one piece of advice: invest in your tools before you invest in your prompts. The ceiling on what an agent can do is set by the quality of the tool interface, not the eloquence of the system prompt.
So, Why Multi-Agent?
Because single agents fail in ways that are hard to predict and hard to debug, and because bounded roles with structured shared state make an investigation's failures easier to trace, whatever the accuracy numbers say. But I'd stop short of the clean version of this pitch. My own testing didn't support "multi-agent is more reliable" as a general claim — it supported something narrower: a verification step that checks a hypothesis against evidence it hasn't seen is what earns its complexity, and you can bolt that onto a single agent just as well as onto four.
The four-role design is still a reasonable way to build one of these systems — the shared-state pattern, the tiered autonomy, the human gate on risky actions are all still doing real work. Just don't assume the agent count is what's buying you the reliability. Test that part before you ship it.
The working prototype for this architecture is available at: github.com/Kinjal-Oza/multi-agent-observability-demo
Originally published on Medium.
Opinions expressed by DZone contributors are their own.
Comments