Solving Session Persistence for Model Context Protocol Servers at Enterprise Scale

Model Context Protocol (MCP) servers that work perfectly in development can fail intermittently once they are deployed across multiple replicas behind a load balancer. The failure mode is a stream of "session not found" errors that appear at random, and the cause is a mismatch between how certain MCP transports hold session state and how load balancers distribute requests. This article explains why the problem occurs, when it applies, and a concrete pattern for solving it using a shared session store.

The problem is easy to miss in early development because it only appears once there is more than one server instance. A single-instance deployment holds every session in local memory, so every request naturally finds its session. Add replicas, and that assumption quietly breaks.

The Failure Pattern

Consider a deployment with four MCP server replicas behind a round-robin load balancer, serving agents that connect over the Server-Sent Events (SSE) transport. In this configuration, roughly three out of four follow-up requests fail with a "session not found" error. That ratio is not random. With four replicas and round-robin distribution, a follow-up request has only a one-in-four chance of returning to the replica that created the session. The other three times it lands on a replica that has no record of that session.

The reason the failures look random at first is that success depends entirely on which replica the load balancer happens to select. The distribution of failures tracks the replica count directly, which is the clearest signal that the load balancer, not application logic, is the source of the problem.

Why MCP Sessions and Load Balancers Conflict

Not every MCP deployment has this problem, so it helps to be precise about when it applies.

A tools-only MCP server can be stateless. Under the streamable HTTP transport, the client caches tool schemas after discovery, and each tool call is a self-contained request that carries everything the server needs to process it. Any replica can handle any request, and load balancing works without special handling.

Two situations make a deployment session-bound.

The first is the SSE transport. SSE was the only remote transport available for a long time and remains widely deployed. It is stateful by design: the client opens a long-lived connection that the server holds open as a stream, and the server delivers responses back through that open stream rather than through the response to each individual request. The stream physically lives on one replica. When a follow-up request is routed to a different replica, that replica is not holding the stream and cannot associate the request with the session. The result is the "session not found" error.

The second is stateful MCP features. Even on a transport that supports stateless operation, an MCP server that must retain per-client state needs sessions. MCP resource subscriptions that push updates when server-side data changes, long-running operations where a client may disconnect and reconnect expecting to resume, and per-client authorization context established at initialization all require the server to hold state across requests. That state must be reachable regardless of which replica receives the next request.

The conflict reduces to a single sentence, which is that the session lives on one replica, but the load balancer distributes requests across all of them.

How the Connection Is Established

The session originates at connection time. The following example uses the Koog framework to connect an agent to an MCP server over SSE, which illustrates where the session comes from:

import ai.koog.agents.core.agent.AIAgent
import ai.koog.agents.mcp.McpToolRegistryProvider
import ai.koog.prompt.executor.llms.all.simpleAnthropicAIExecutor
import ai.koog.prompt.llm.AnthropicModels
import kotlinx.coroutines.runBlocking

fun main() = runBlocking {
    // Open an SSE transport to the MCP server
    val transport = McpToolRegistryProvider.defaultSseTransport("http://mcp-server:3000/sse")

    // Build a tool registry from the tools the MCP server
    val mcpRegistry = McpToolRegistryProvider.fromTransport(
        transport = transport,
        name = "records-client",
        version = "1.0.0"
    )

    val agent = AIAgent(
        executor = simpleAnthropicAIExecutor(),
        llmModel = AnthropicModels.Claude.SONNET,
        toolRegistry = mcpRegistry
    )

    val result = agent.run("Look up the status of record 12345")
    println(result)
}

The relevant detail is the transport and the roles it establishes. The client, Koog in this case, opens the SSE connection, and the session lives on the MCP server. Opening an SSE transport creates a stateful connection: the MCP server creates a session bound to that open stream, and from that point the client and server communicate through a channel anchored to one specific server instance. With a single instance, this is invisible. Behind a load balancer, it is the entire problem. The fix belongs on the server side, not in the client.

The Fix: A Shared Session Store

The solution is to stop storing session state in a replica's local memory and move it to a shared store that every replica can reach. This is an addition to the MCP server implementation. Neither the MCP specification nor the client library provides a distributed session store; the specification defines that sessions exist but does not prescribe how to persist them across instances, so the server-side session handling is the implementer's responsibility. Redis is a natural fit for this role because the access pattern is a simple keyed lookup and the added latency is negligible relative to the rest of an agent request.

The mechanism is straightforward. When any replica creates a session, it writes the session record to the shared store rather than to local memory. When any replica receives a request, it reads the session from the shared store before processing. The session no longer belongs to a replica; it belongs to the store, and every replica can reach it.

The session record contains what the server would otherwise hold in memory - the session identifier, the negotiated capabilities, any accumulated per-client state, and timestamps for expiry. Assigning each entry a time-to-live allows idle sessions to expire automatically rather than accumulating.

The change in the server's request handling can be reduced to the difference between a local map and a shared lookup:

// Before: the session lives in this replica's memory.
// Other replicas have no record of it.
val localSessions = mutableMapOf<String, McpSession>()

fun handleRequest(sessionId: String, request: McpRequest): McpResponse {
    val session = localSessions[sessionId]
        ?: error("session not found")   // fails on any other replica
    return session.process(request)
}

// After: the session lives in a shared store every replica can read.
suspend fun handleRequest(sessionId: String, request: McpRequest): McpResponse {
    val session = sessionStore.get(sessionId)      // shared lookup
        ?: error("session expired or unknown")
    val response = session.process(request)
    sessionStore.put(sessionId, session)           // persist any state change
    return response
}

The SSE transport adds one further requirement. Because the response must travel back through the stream held by a specific replica, the shared store also records which replica holds the stream, and a publish-subscribe channel routes the response to that replica when a request is handled elsewhere: 

// The replica holding the SSE stream subscribes for its sessions
sessionBus.subscribe("mcp:response:$sessionId") { payload ->
    sseStream.send(payload)
}

// Any replica that processes a request publishes the response
sessionBus.publish("mcp:response:$sessionId", response)

In this arrangement, the shared store serves two purposes. It is the session store that allows any replica to handle a request, and it is the message bus that routes each response to the replica holding the open stream. A request may arrive at any replica, while the response is delivered to the connection the client is actually listening on.

Why Not Sticky Sessions

The most immediate alternative is sticky sessions: configuring the load balancer to pin each client to the replica that created its session. This works and is a reasonable temporary measure, but it carries three drawbacks that make it unsuitable as a durable solution.

Sticky sessions undermine load distribution, because a high-volume client is concentrated on a single replica while others remain underused. They reintroduce the single point of failure that multiple replicas were intended to eliminate: if the pinned replica fails, every session on it is lost. And they complicate scaling, because newly added replicas receive no existing traffic and take on load only gradually.

A shared session store avoids all three. The load balancer can use plain round-robin distribution. Any replica can fail without affecting sessions held by the others. A new replica can serve existing sessions immediately, because it reads them from the same shared store as every other replica.

Results

With the shared session store in place, the "session not found" errors are eliminated for active sessions, and requests distribute evenly across replicas. Deliberately terminating a replica no longer interrupts active agents, and their requests are absorbed by the remaining replicas. Adding a replica requires no special handling.

The shared lookup adds a small step to each request, but the cost is minor in context. A session read is well under a millisecond, while an agent request already spends hundreds of milliseconds or more on model inference and downstream calls. The overhead is not observable in practice.

Summary

For teams deploying MCP servers at scale, three points are worth carrying forward.

Keep the MCP server stateless where possible. A tools-only server on the streamable HTTP transport scales horizontally without any of this complexity. Sessions should be introduced only when genuinely required, for subscriptions, resumable operations, or server-held per-client context.

When sessions are required, do not store them on the replica. Move them to a shared store so that any replica can serve any request. This mirrors the lesson web applications settled on years ago for HTTP session state, now recurring in the context of MCP.

Account for the SSE response-routing requirement. A shared session store resolves request handling, but the response must still reach the replica holding the open stream, which a publish-subscribe channel provides.

Session persistence behind a load balancer is a common example of the operational gaps teams encounter when deploying MCP in production, and the shared-store pattern described here is a direct and durable solution.