Why MCP Servers Lose Session State Behind Load Balancers

Picture an MCP server that keeps “forgetting” what an agent just asked it to do.

The agent starts a long-running tool call, such as a database migration. When it checks back for status, the server has no record of the request. There is no crash and no obvious error. The follow-up request simply landed on a different server instance behind the load balancer.

That instance never saw the original session.

This is a common failure mode once MCP servers move beyond local studio usage and into Streamable HTTP deployments with multiple backend instances. Standard HTTP load balancing treats requests as independent. Stateful MCP sessions expect continuity. When those assumptions collide, the result can be silent state loss.

This article walks through why it happens and three practical ways to handle it: sticky sessions, externalized state, and stateless sessions with resumable streams.

Why This Happens

MCP's client-server model was originally built around a fairly simple mental picture: a host spins up a client, the client talks to a server, and that server sticks around for the life of the session. That works fine when the client launches the server as a local subprocess over stdio, which is still how many local dev setups, IDE integrations, and CLI agents talk to MCP servers. 

The picture changes the moment a deployment moves to Streamable HTTP for remote, multi-tenant use. Now there are:

• Multiple server instances behind a load balancer
• No shared memory between them
• A session expected to persist across multiple requests
• Tool calls that can run long enough for a request to get rerouted mid-session

Standard HTTP load balancing assumes requests are independent. MCP sessions assume continuity. Put those two assumptions in the same system, and the result is exactly the bug described above — silent state loss, not a hard failure, which arguably makes it worse. A crash gets noticed quickly. A hallucinated "sure, that's done" doesn't, until someone downstream asks why the migration never ran.

What a Naive Setup Looks Like

Here's a stripped-down MCP server that will fall over reliably in a multi-instance deployment. It uses in-memory session storage — fine for a demo, a landmine in production:

// naive-mcp-server.js
const sessions = new Map(); // lives only in this process's memory

app.post('/mcp', async (req, res) => {
  const sessionId = req.headers['mcp-session-id'];

  if (!sessions.has(sessionId)) {
    sessions.set(sessionId, { createdAt: Date.now(), context: {} });
  }
  const session = sessions.get(sessionId);
  const result = await handleToolCall(req.body, session);
  res.json(result);
});

Tier 1: Sticky Sessions (Session Affinity)

The simplest fix, and the one most teams should reach for first, is routing every request from a given session to the same backend instance. The MCP session ID becomes the routing key, instead of relying on the load balancer's default algorithm (round robin, least connections, or whatever else).

Here's what that looks like with nginx using a session-aware upstream:

upstream mcp_backend {
    hash $http_mcp_session_id consistent;
    server 10.0.1.10:3000;
    server 10.0.1.11:3000;
    server 10.0.1.12:3000;
}
server {
    listen 443 ssl;
  
    location /mcp {
        proxy_pass http://mcp_backend;
        proxy_set_header Mcp-Session-Id $http_mcp_session_id;
        proxy_read_timeout 300s; # long-running tool calls need headroom
    }
}

The hash ... consistent directive means the same session ID reliably maps to the same upstream server, and — this part matters — if a server is added or removed, only a fraction of sessions get remapped instead of the whole pool shuffling. Without consistent, scaling the fleet up or down would silently relocate a chunk of active sessions, landing right back at square one.

On the application side, it's worth validating that the session actually exists locally, and failing loudly if it doesn't, not silently creating a new one:

app.post('/mcp', async (req, res) => {
  const sessionId = req.headers['mcp-session-id'];
  const session = sessions.get(sessionId);

  if (!session) {
    // Routed somewhere it shouldn't have been. Don't fake it.
    return res.status(409).json({
      error: 'session_not_found_on_instance',
      message: 'Session affinity may have broken. Client should retry or reinitialize.'
    });
  }
  const result = await handleToolCall(req.body, session);
  res.json(result);
});

That explicit failure does real work here. Whether you return a diagnostic 409 for affinity failure or a spec-aligned 404 for an unknown session, the important part is not pretending the session exists. 

It turns a silent data-loss bug into a visible, debuggable failure — far better than letting an agent confidently report on work it never actually did.

Where sticky sessions fall short:

If an instance dies, every session pinned to it dies too. There's no failover. For a lot of teams, that trade-off is completely acceptable — instance restarts are rare, sessions are usually short-lived, and the operational simplicity is worth it. But teams that need real resilience need to look at tier two.

Tier 2: Externalized State

Instead of keeping session data in process memory, it gets pushed out to something every instance can reach — Redis is the obvious choice, though any shared, low-latency store works.

const redis = require('redis').createClient();
app.post('/mcp', async (req, res) => {
  const sessionId = req.headers['mcp-session-id'];
  const raw = await redis.get(`mcp:session:${sessionId}`);
  const session = raw ? JSON.parse(raw) : { createdAt: Date.now(), context: {} };
  const result = await handleToolCall(req.body, session);

  await redis.set(
    `mcp:session:${sessionId}`,
    JSON.stringify(session),
    { EX: 1800 } // 30-minute TTL, adjust to session lifecycle
  );
  res.json(result);
});

Now any instance can pick up any session — routing no longer matters, and an instance dying doesn't take sessions down with it. The cost is exactly what would be expected: a network round trip on every request, serialization overhead, and one more piece of infrastructure to keep healthy and monitored. For tool calls with large context payloads, that serialization cost isn't trivial — worth benchmarking before committing to it at scale.

Tier 3: Stateless Sessions With Resumable Streams 

The most ambitious approach avoids server-owned session state where possible. Instead, enough context gets encoded in the session token itself (signed, so it can't be tampered with) that any instance can reconstruct what it needs from the request. Combined with MCP's resumability features — replaying missed events from a last-event-ID — this produces a server that doesn't care which instance handles which request, because there's no server-side state to lose.

This is the kind of pattern large, multi-region deployments often need, but it's genuinely more work: it requires careful token design, replay handling, and a clear answer for what happens when a tool call's side effects (like that database migration) need to survive a dropped connection mid-execution.

Comparing the Three Approaches

Things Worth Checking Before Deployment

A few details that commonly cause trouble, in no particular order:

• Set a proxy_read_timeout (or the load balancer's equivalent) high enough for the longest expected tool call - the default is almost always too short for anything doing real work.
• Don't let sessions live forever in Redis or wherever they're stored. A TTL that's too generous quietly turns into a memory leak.
• Test what happens when the instance pool scales up or down mid-traffic, not just at steady state. This is exactly where sticky-session setups without consistent hashing fall apart.
• Log session-affinity failures (like that 409 above) separately from normal errors. A spike usually means the load balancer config drifted, not that application logic broke.
• With Streamable HTTP, double-check that origin validation and auth are still enforced per-request, not just at session creation. Session continuity shouldn't come at the cost of skipping checks on the requests that follow.

Make sure clients send MCP-Protocol-Version on subsequent HTTP requests, and validate unsupported versions server-side.

The Bigger Picture

None of this is really MCP-specific, at a systems level — it's the same stateful-session-behind-a-load-balancer problem that's shown up in every generation of web infrastructure, from sticky sessions in classic app servers to sharded WebSocket connections. What's new is that MCP's tool calls can be long-running and consequential (migrations, deployments, write actions against real systems), which makes silent state loss a lot more expensive than it would be for, say, a shopping cart.

Teams still running a single MCP server instance don't need to worry about any of this yet. But the moment horizontal scaling enters the picture — and for any MCP server doing production-critical work, it eventually will — deciding up front which tier to build for beats discovering the gap the hard way, mid-incident.