Freshness Is the Missing SLO in Production Vector Search

The Index Can Be Fast and Still Be Wrong

Vector search teams usually define performance with query latency, recall, and throughput. Those measures matter, but they can all look healthy while the system returns a stale version of a document that changed minutes ago. The index is fast. The answer is still wrong.

This failure is easy to miss because a vector index is normally downstream from the source of truth. Between a database write and a searchable embedding sit event capture, transport, chunking, model inference, index mutation, and cache invalidation. Freshness is the end-to-end property produced by that entire chain.

Freshness Needs a Contract

Saying that updates are processed quickly is not a contract. A useful freshness SLO states which source version must be searchable, how long the pipeline may lag, and what the query path should do when that guarantee cannot be met.

For example, a service might require 99 percent of committed updates to become searchable within 60 seconds, while deletes must disappear within 10 seconds. The distinction matters because showing old text is inconvenient, but returning deleted or access-revoked content can become a security incident.

Capture the Write Without a Dual-Write Gap

The first failure appears when application code writes the business row and publishes an indexing event as two separate operations. If the database commit succeeds and the publish fails, the source changes without any durable instruction to update the index. Retrying the request does not reliably repair that gap.

A transactional outbox avoids the split. The application updates the entity and inserts an outbox record in the same database transaction. A change-data-capture process then publishes the outbox record asynchronously.

BEGIN;
UPDATE documents
SET body = :body, version = version + 1
WHERE id = :id;

INSERT INTO embedding_outbox
  (event_id, entity_id, source_version, operation)
SELECT :event_id, id, version, 'UPSERT'
FROM documents WHERE id = :id;
COMMIT;

Make Every Event Versioned and Idempotent

Delivery systems retry. Partitions rebalance, workers crash after writing but before acknowledging, and older events can arrive after newer ones. The index consumer must therefore treat duplicate and out-of-order delivery as normal behavior, not an edge case.

Each event should carry an immutable event identifier, entity identifier, monotonic source version, operation, and payload reference or hash. The consumer applies a mutation only when the incoming version is newer than the indexed version. That compare-and-set must be atomic in the index or in a strongly consistent metadata store beside it.

def apply(event, index, embed):
    current = index.metadata(event.entity_id)

    if current and current.source_version >= event.source_version:
        return "already_applied"

    if event.operation == "DELETE":
        index.delete_if_newer(event.entity_id, event.source_version)
        return "deleted"

    vector = embed(event.content)
    index.upsert_if_newer(
        id=event.entity_id,
        vector=vector,
        metadata={"source_version": event.source_version}
    )
    return "updated"

Deletes Are First-Class Data

Upserts get most of the design attention because they create embeddings. Deletes are more dangerous because there is no new content to process. A delete event must survive the same durable path and carry a version that prevents an older upsert from resurrecting the record later.

Keep tombstones long enough to cover the maximum replay and recovery window. If a full rebuild reads a snapshot taken before a delete, the rebuild process must also consume the change stream from the snapshot position forward. Otherwise, the old record can quietly return when the new index is promoted.

Model Versions Belong in the Index Schema

Fresh source data can still be semantically stale when query and document vectors were created by different embedding models. Store the embedding model identifier, chunking configuration version, and normalization settings with every indexed item. Treat those fields as part of the index schema.

A model upgrade should normally create a new physical or logical index generation. Dual-write new updates, backfill historical content, validate retrieval quality, then switch query traffic. Mixing vectors from incompatible spaces in one collection creates a failure that looks like weak relevance but cannot be tuned away.

Use Watermarks to Measure What the Pipeline Has Proven

A queue-depth metric shows workload, not freshness. The more useful signal is a source-position watermark: the highest committed database position or entity version that the searchable index has fully applied. Compare that watermark with the source head to measure version lag and event-time lag.

Parallel consumers complicate this because one partition can race ahead while another is stuck. The global searchable watermark is bounded by the slowest required partition. Reporting the fastest worker hides exactly the stale slice users are likely to hit.

Guard Queries When Freshness Matters

Some requests know the minimum version they require. A write API can return the committed source version, and a later search request can send that version as a read-your-writes token. The query layer then checks whether the relevant index watermark has caught up.

The fallback depends on the product. The service can wait briefly, route to a fresher generation, perform a source-of-truth lookup, or return a clear retryable status. Serving an older result without saying so should not be the default.

def search(query, minimum_version=None):
    if minimum_version is not None:
        if index_watermark() < minimum_version:
            raise RetryableFreshnessError(
                "search index has not reached the required version"
            )
    return vector_index.search(query)

Rebuild Without Creating a Freshness Blackout

Large indexes eventually need rebuilding because schemas, models, or partition layouts change. A safe rebuild uses a snapshot plus a change-stream handoff. Record the snapshot position, bulk-load the snapshot into a new generation, replay every later event, and promote only after its watermark reaches the live index.

The promotion itself should be an atomic alias or routing change. Keep the previous generation available for rollback until both correctness and latency checks pass. A rebuild is not complete when bulk loading ends. It is complete when the new generation proves that no committed change was skipped.

Operate Freshness Like Availability

The dashboard should track source-to-index lag percentiles, oldest unapplied event age, consumer retry rate, dead-letter volume, version conflicts, delete lag, model-version distribution, and watermark gaps by partition. Alerts should be tied to the freshness SLO rather than to queue depth alone.

Periodic reconciliation closes the final gap. Sample source entities, compare their versions and hashes with indexed metadata, and repair mismatches through the normal event path. The goal is not to pretend delivery is perfect. The goal is to make drift observable, bounded, and repairable.

Test Failure Modes Before Launch

Also test partial degradation. If one embedding worker pool is unavailable, confirm that lag remains visible and query guards behave as designed. If the dead-letter path fills, verify that alerts fire before the SLO is exhausted. These exercises turn recovery assumptions into executable evidence and reveal whether the pipeline can repair itself without manual database edits.

Freshness behavior deserves fault-injection tests, not only happy-path integration tests. Pause one consumer partition, duplicate a batch, deliver versions out of order, fail an embedding call after the index write, and replay a snapshot across a recent delete. Each test should assert the final indexed version, not merely that the worker returned success.

The Missing SLO

Production vector search is a replicated data system with expensive transformation in the middle. Once that is clear, familiar distributed-systems rules apply: capture changes durably, version every mutation, make consumers idempotent, preserve deletes, expose watermarks, and rebuild from a known log position.

Latency tells you how quickly the index answered. Freshness tells you whether it answered from the world that exists now. A production search system needs both guarantees, because a fast answer from yesterday is still a failure.

References

• Debezium Outbox Event Router
• PostgreSQL Logical Decoding Concepts
• Apache Flink Timely Stream Processing and Watermarks