Handling Large API Responses Without Freezing the Client: A Practical Architecture With Temporal, Kafka, and RAG

A large API response becomes a client problem long before it becomes a network problem. A browser can receive hundreds of megabytes and still become unresponsive while buffering bytes, parsing one enormous JSON document, retaining duplicate object graphs, and rendering too much state on the main thread. 

The reliable solution is not a larger timeout. It is to stop treating the response as a synchronous document and start treating it as a durable, observable job whose data arrives in bounded pieces. Browser streams support incremental consumption and backpressure, while background workers allow long-running processing to remain independent of user-interface scripts. 

The Response Becomes a Job, Not a Payload

The public API should acknowledge work quickly and return a stable job identifier rather than hold an HTTP connection open until every upstream page has been fetched. A 202 Accepted response establishes that contract without implying completion. The client can then subscribe to progress events, request a partial view, or retrieve a final artifact when the job reaches a terminal state. RFC 9110 defines 202 Accepted specifically for requests accepted for processing when processing has not necessarily completed.

@PostMapping("/reports")
public ResponseEntity<JobAccepted> create(@RequestBody ReportRequest request) {
    String jobId = UUID.randomUUID().toString();
    workflowClient.start(reportWorkflow::run, jobId, request);

    return ResponseEntity.accepted()
        .header("Location", "/reports/" + jobId)
        .body(new JobAccepted(jobId, "QUEUED"));
}

This endpoint performs no large download or expensive transformation. It creates an addressable unit of work and returns immediately. The browser remains responsive because the initial response is tiny, while server capacity is protected from long-lived request threads. The job record should expose states such as queued, fetching, indexing, ready, failed, and canceled, with progress kept monotonic and coarse enough to remain trustworthy.

Temporal Owns the Long-Running Control Flow

Temporal fits the control plane because Workflow state survives process crashes and worker restarts, while failure-prone operations such as remote API calls belong in Activities with explicit timeouts and retry policies. Temporal documentation distinguishes deterministic Workflow logic from non-deterministic Activities and provides retry, timeout, heartbeat, and message-passing mechanisms for long-running execution.

@WorkflowMethod
public ResultRef run(String jobId, ReportRequest request) {
    String cursor = null;
    int sequence = 0;

    do {
        PageRef page = activities.fetchAndStore(jobId, cursor, sequence);
        activities.publishChunkReady(jobId, page);
        cursor = page.nextCursor();
        sequence++;
    } while (cursor != null && !canceled);

    activities.buildIndex(jobId);
    activities.publishCompleted(jobId, sequence);
    return new ResultRef(jobId, sequence);
}

@SignalMethod
public void cancel() {
    canceled = true;
}

Only references and counters should cross Workflow boundaries. Passing raw pages through Temporal causes every Activity input and result to accumulate in Event History. Temporal warns that large histories increase Workflow Task latency, documents a 50 MB or 51,200-event history limit, and recommends external storage plus Continue-As-New for large or long-running executions.  The response body therefore belongs in object storage, while Temporal retains keys, checksums, cursors, and status.

The fetching Activity should checkpoint often enough to support retries without restarting the transfer. Heartbeat details can carry the last committed cursor or byte range. Temporal recommends heartbeats for long-running Activities because missed heartbeats can trigger failure detection and retry.

public PageRef fetchAndStore(String jobId, String cursor, int sequence) {
    UpstreamPage page = upstream.fetch(cursor);
    String key = storage.put(jobId + "/" + sequence, page.bytes());

    Activity.getExecutionContext().heartbeat(
        new FetchCheckpoint(sequence, page.nextCursor())
    );

    return new PageRef(
        key,
        sequence,
        page.nextCursor(),
        page.sha256()
    );
}

Kafka Carries Facts, Not Giant Documents

Kafka is most effective as the event backbone, not as a substitute for object storage. Events should describe what happened and point to durable data, ChunkStored, ChunkIndexed, JobProgressed, JobCompleted, or JobFailed. Kafka enforces record-size limits at both producer and broker levels, so pushing multi-megabyte fragments into records creates brittle configuration coupling and expensive retries. 

Every event should use jobId as the key. Kafka partitions are ordered logs, and records sharing a key normally land in the same partition, preserving per-job sequence while allowing unrelated jobs to scale across partitions. Consumer groups distribute partitions across workers and rebalance them when membership changes.

public void publishChunkReady(String jobId, PageRef page) {
    ChunkReady event = new ChunkReady(
        jobId,
        page.sequence(),
        page.storageKey(),
        page.sha256()
    );

    kafkaTemplate.send("report-events", jobId, event);
}

Duplicate delivery must be assumed at every boundary. Kafka producer idempotence prevents duplicate writes caused by producer retries when compatible acknowledgment and in-flight settings are used, but downstream side effects still require idempotent consumers.  An indexer can enforce uniqueness with (jobId, sequence, checksum) and commit its database transaction before acknowledging the Kafka offset.

Backpressure should be expressed through bounded concurrency rather than hidden in memory. An Activity can publish one stored chunk at a time, while indexer lag indicates downstream pressure. Temporal can pause between pages when lag crosses a threshold, or consumers can scale until partition count becomes the limit.

The Client Receives Progress and Bounded Content

Server-sent events are sufficient when communication is primarily server-to-client. The protocol uses text/event-stream, keeps a persistent HTTP connection, and represents each notification as a small text block. A projection service can consume Kafka events, maintain the latest job state, and expose a resumable stream using application event IDs

@GetMapping(
    value = "/reports/{jobId}/events",
    produces = MediaType.TEXT_EVENT_STREAM_VALUE
)
public Flux<ServerSentEvent<JobEvent>> events(
        @PathVariable String jobId) {

    return eventProjection.stream(jobId)
        .map(event -> ServerSentEvent.<JobEvent>builder()
            .id(event.sequence().toString())
            .event(event.type())
            .data(event)
            .build());
}

The client should render status changes and small previews, not append the full raw response into application state. When direct streaming is required, the Fetch API exposes the response body as a ReadableStream, allowing chunk-by-chunk processing rather than waiting for completion.  Parsing should occur incrementally, with CPU-heavy decoding or transformation moved to a Web Worker, whose execution remains separate from user-interface scripts. 

Final delivery should usually be a paginated query API, a range-readable artifact, or a signed download URL. A giant JSON reconstruction endpoint merely recreates the original failure at the last step.

RAG Turns Stored Volume Into a Useful Interface

RAG becomes valuable after chunks are durably stored. Each chunk can be normalized, split along semantic boundaries, embedded, and indexed with metadata containing the job identifier, source sequence, object key, and byte range. The original RAG formulation combines parametric generation with retrieved non-parametric memory, grounding generation in selected passages rather than the entire corpus.

@KafkaListener(
    topics = "report-events",
    groupId = "rag-indexers"
)
public void onChunkReady(ChunkReady event) {
    if (index.exists(
            event.jobId(),
            event.sequence(),
            event.checksum())) {
        return;
    }

    byte[] payload = storage.get(event.storageKey());

    chunker.split(payload).forEach(chunk ->
        index.upsert(
            event.jobId(),
            event.sequence(),
            chunk
        )
    );

    progress.markIndexed(
        event.jobId(),
        event.sequence()
    );
}

The query path retrieves only the most relevant chunks and sends those bounded passages to the model. Raw object references remain attached so generated statements can link back to source material. RAG should not conceal incomplete ingestion; the query service must expose index coverage and reject complete-report requests until all expected chunks are indexed.

public Answer answer(String jobId, String question) {
    List<Passage> context =
        index.search(jobId, question, 8);

    return generator.generate(question, context);
}

This layer changes the client experience from downloading everything before anything is useful to inspecting progress, searching partial results, and retrieving only relevant evidence. It also keeps model context bounded when the source response is extremely large.

A Responsive System Is Built From Explicit Boundaries

The essential boundary is simple: Temporal owns durable intent and recovery, Kafka distributes compact facts, object storage holds large bytes, RAG builds a searchable semantic view, and the client receives only bounded updates or explicitly requested slices. Each component solves a different failure mode, and none is forced to carry the complete response through an interface designed for small messages. The resulting architecture prevents UI freezes, survives retries and restarts, supports cancellation and replay, and makes large upstream results useful before a monolithic download could finish. Large-response handling becomes reliable when completion is modeled as a process rather than a payload.