When Data Quality Checks Pass but the Data Is Still Stale

A pipeline can finish successfully, schemas can match, and null checks can pass, while the business is still looking at yesterday's truth. Freshness deserves its own quality model.

The pipeline succeeded. The schema matched. Required fields were present, ranges were sane, and the dashboard refreshed on schedule. Every quality check was green. The number on the screen was still wrong, because it was built from data that stopped updating two days ago and nobody noticed.

This failure is common, and it is quiet. Most data quality programs are built to answer one question: is this data valid? They check for nulls, types, ranges, uniqueness, and referential integrity. Those checks are necessary, and they catch a real class of problems. They also share a blind spot. A record can be perfectly valid and completely stale. Validity is about whether the data is well-formed. Freshness is about whether it is current enough to trust. They are different properties, and a pipeline that measures only the first will keep serving old truth with a green status next to it.

Freshness Is Not Correctness

Structural quality asks whether a row is shaped correctly. Freshness asks whether the row should still be believed given how much time has passed. A transaction record from Tuesday is structurally identical whether it is read on Wednesday or three weeks later. Its validity never changes. Its usefulness for a decision that assumes current data changes completely.

This is why freshness belongs in the quality model rather than in a separate operations dashboard. Most quality dimensions that teams already track, such as completeness, accuracy, consistency, uniqueness, and validity, describe the data as it sits. Freshness describes the data relative to now. Leaving it out of the quality model means the platform can report high quality on data that is too old to act on, which is not a contradiction the business will find reassuring.

The concept that ties this together is the freshness gap: the distance between when an event actually happened and when a consumer can first see it. Structural checks never measure this gap, because both a fresh record and a stale one are equally valid. The gap is the part of quality that only time reveals.

Why Pipelines Hide Staleness

The reason staleness stays hidden is that pipeline success and data freshness measure different things, and teams routinely treat the first as a proxy for the second. A job can complete successfully while delivering nothing new. Common paths to a green pipeline over stale data include:

• The source sent no new files. The job ran, found the same input as yesterday, processed it correctly, and reported success. Nothing failed. Nothing updated either.
• Only some partitions arrived. The pipeline loaded the partitions it received and completed. The missing region or date range is not an error to a job that was never told those partitions were mandatory.
• A late-arriving upstream delayed the real data. The scheduled run fired on time against data that had not landed yet, so it processed an incomplete or old snapshot and finished cleanly.
• The dashboard cached a stale table. The pipeline updated the table, but the serving layer or BI tool returned a cached result, so the freshest data never reached the screen.
• A backfill overwrote current data with an older snapshot. A correction job ran a historical range and, through a scope error, replaced newer records with older ones. Every row is valid. The table went backward in time.

None of these trip a structural check, because in every case the data that is present is well-formed. The problem is not the shape of what arrived. It is the age of what arrived, and whether anything arrived at all.

Freshness Needs Its Own Contract

Freshness cannot be governed by a single global rule, because different datasets have different tolerances. A five-minute delay is a crisis for fraud detection and irrelevant for a historical archive. Tying freshness to the pipeline schedule is the common shortcut, and it is wrong, because the schedule describes when the job runs, not when the data is expected to be current for a specific use.

The fix is to define freshness expectations per dataset, anchored to the business decision the data supports rather than to the cadence of the job that produces it.

Each expectation is a contract. It states what current means for that dataset, and it gives monitoring something concrete to check against. Without it, freshness is a matter of opinion, and the first time anyone forms an opinion is usually after a stale number has already reached a decision.

Measuring Freshness Correctly

The technical heart of freshness is that there is no single timestamp called "the time." A record carries several distinct times, and confusing them is how freshness monitoring gives false comfort. Four matter:

The freshness gap, which is event time to publish time, is the delay no structural check measures. The relevant times to consider are:

• Event time: When the thing actually happened in the source system. A purchase was made, a sensor fired, an address changed.
• Ingestion time: When the record entered the platform. The moment it landed in the queue or the raw zone.
• Processing time: When the transformation ran over it. The point where it was cleaned, joined, and shaped.
• Publish time: When it became queryable by a consumer. The moment the serving table or dashboard could return it.

The freshness gap that matters to the business is publish time minus event time, because that is the total delay between reality and what a consumer can see. A pipeline that measures only processing time, "the job ran at 06:00," reports a healthy number while the events it processed are hours old, because the delay lived upstream, before ingestion, where the job never looked.

Measuring the wrong timestamp is worse than not measuring, because it produces a confident freshness metric that is disconnected from reality. A dataset can show a two-minute processing lag and a six-hour event-to-publish gap at the same time. The first number looks great on a status page. The second is the one the business feels.

What Freshness Failures Look Like

In practice, freshness failures usually look healthy from the outside. The job finishes, the schema matches, and the records pass validation. The failure lives in the time dimension: no new source data arrived, only some partitions landed, a dashboard served a stale cache, or a backfill moved the table backward. Structural validation sees rows that are well-formed. Freshness monitoring sees that the published dataset no longer reflects the current state of the business. Passing one tells you nothing about the other.

A Practical Freshness Pattern

Making freshness a first-class quality dimension does not require a new platform. It requires treating the age of data as something the pipeline measures, records, and alerts on, the same way it already treats nulls and types. A workable pattern:

• Carry timestamps through the pipeline. Preserve event time from the source, and stamp ingestion, processing, and publish times as the record moves. The freshness gap cannot be measured if the timestamps needed to compute it were discarded early.
• Record freshness in a small audit table. For each dataset and run, store the maximum event time published and the publish time itself. This gives a queryable history of how current each dataset actually was, run over run.
• Attach a freshness SLA to each dataset. Encode the per-dataset expectation from the contract above as a checked threshold, not a comment in a runbook.
• Alert on the gap, not on job status. Trigger when publish-time-minus-event-time crosses the dataset's threshold, independent of whether the job reported success. This is the alert that catches the source-sent-nothing case, which job monitoring cannot see.
• Make freshness visible downstream. Surface the last known freshness next to the data itself, so a consumer can see that a dashboard is running on data from two days ago before they act on it.

Track compliance as a percentage over time rather than as a pass or fail on a single run. A dataset that met its freshness SLA 99 percent of the time last month, and is trending down, is a more honest signal than a single green check, and it is the number a business owner can actually reason about.

A minimal audit table makes this concrete. One row per dataset per run is enough to compute the gap, compare it against the SLA, and keep a history:

The gap column is the one structural checks never produce, and the status column is what the freshness alert reads rather than job success.

The Green Check was Measuring the Wrong Thing

Validity and freshness are independent. A pipeline can watch one perfectly and never look at the other, which is exactly how a dataset ends up well-formed, internally consistent, and two days out of date with a passing status next to it. The structural checks were doing their job. They were just never the checks that would have caught this.

Freshness needs its own contract, its own timestamps, and its own alert that fires on the age of the data rather than the exit code of the job. Decide what current means for each dataset, watch the distance between event time and publish time, and put that distance in front of the people making decisions. A pipeline finishing was never the same claim as the data being fresh, and the sooner a platform stops treating the first as proof of the second, the fewer stale numbers reach a meeting.