DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Newsletter
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Related

  • Threat Modeling Context-Aware AI: A STRIDE Walkthrough
  • Building an AI-Ready Data Layer Without Rebuilding the Enterprise
  • Why Real-Time Data Pipelines Are Becoming the Foundation of Industrial AI
  • Architecting Production AI Across Clouds: Patterns That Decide System Survival

Trending

  • You Don't Have to Pick One: A Hybrid Approach to Enterprise Data Refresh
  • One Database, Two Models: Building AI-Friendly Java Applications With MySQL JSON Duality Views
  • How to Format Articles for DZone
  • How to Migrate From C/C++ to Rust: A Practical Guide for Embedded IoT Systems

Draft This article hasn't been published, so readers can't see it.

In moderation This article is waiting for review, so readers can't see it yet.

Deleted This article has been deleted, so readers can't see it.

  1. DZone
  2. Data Engineering
  3. AI/ML
  4. Your Agents Think Data Federation Means One Truth. It Means a Collage.

Your Agents Think Data Federation Means One Truth. It Means a Collage.

Federation makes data reachable across Snowflake, Databricks, and Iceberg. It doesn't make it trustworthy for an AI agent that acts on it without checking twice.

By 
Sushree Mishra
·
Nov. 16, 26 · Analysis
Post on X (opens in a new tab) Share on LinkedIn (opens in a new tab)
136 Views

Join the DZone community Save articles, join the discussion, and download Refcards and Trend Reports with a free account.

Join for free Already a member? Log in

Data federation was sold on a simple promise: query data where it already lives, across Snowflake, Databricks, and the rest of the stack, without making another copy. No more nightly ETL jobs staging the same customer table in four different warehouses. No more arguments about which copy was authoritative. Point the query at the source and get an answer.

For the workloads it was designed to support, that promise mostly held. A human analyst could pull a report, or a dashboard could refresh each morning, without moving the data. The query ran. If a number looked slightly off, someone could notice, check it, or just shrug and move on.

Federation was never built for an autonomous agent that accepts the answer and acts on it without a second look. That assumption is about to be tested, and it will not survive.

The Copy Problem Was Never the Coherence Problem

Federation solved a real issue: duplicated data creates duplicated cost, duplicated drift, and duplicated arguments about which copy to trust. Querying in place through an open table format like Iceberg, or through Snowflake's and Databricks' respective sharing layers, removes that duplication. What it does not do is guarantee that the answer coming back represents one coherent state of the world.

Those are different problems, and the industry has spent the last few years solving the first while quietly assuming the second came for free. It didn't. Here's where that assumption breaks, specifically once an agent, not a person, is the one issuing the query.

Five Clocks, One Query

A federated join across a warehouse, a lakehouse, and a live API does not pull from one system at one moment. It pulls from three systems, each with its own refresh schedule and consistency model. None of them is coordinated with the others, and no distributed transaction aligns their data to a single point in time. The result looks like a snapshot, but it is closer to a collage assembled from whatever each source had available when the query arrived.

Take a churn-risk agent deciding whether a customer needs a retention offer. It queries the CRM for account status, the support platform for open tickets, and the billing system for payment history. The CRM record was synced four hours ago. A support ticket that just closed has not propagated yet. The billing system runs on a separate pipeline and is current to the last five minutes.

The agent receives one joined row and treats it as one fact: this customer is a churn risk, so act now. It has no way to know that a third of the row was already stale by the time it arrived.

A human analyst who sees an unexpected number will usually pause, check the timestamp, and perhaps run the query again. An agent will not do that unless someone explicitly designed it to. It accepts the federated result as ground truth and acts on it.

Federated queries

Figure 1. Federated queries combine data from sources with different freshness levels, creating a result that appears unified but may not represent one coherent moment.

Authorization for a Session, Not a Stream of Calls

Federated and shared-access models assume a human user: log in, pass an authorization check at the session boundary, and then query within that approved window. The model is intentionally coarse because a human session does not need authorization reevaluated every few seconds.

An agent works differently. A fraud-review agent querying a shared Iceberg table of transaction records may use a service account that received table-level read access once, during setup. From there, it can issue several hundred narrow, autonomous queries an hour while working through flagged cases.

Nothing in the architecture rechecks whether that specific task should be allowed to see each record it requests. A standing grant designed for someone opening a dashboard twice a day now supports a process capable of issuing hundreds of queries without supervision.

Runtime access enforcement has to happen on every call, not just at the start of a session. Most federation architectures deployed today were not built for that level of granularity or volume.

Federation makes the query possible. It doesn't make the answer agreeable.

Enterprises increasingly run multiple agents, often on different vendor stacks: Databricks tooling for one and Snowflake Cortex for another. Each agent reasons independently about shared entities such as customers and orders.

In this setting, the case for federated, in-place access is not efficiency. The question is whether the agents are even seeing versions of the same entity that can be reconciled.

Consider a supply chain agent on one platform and a finance agent on another, both querying the same order record. One must decide whether to expedite a shipment. The other must decide whether to release a payment. The supply chain agent reads the order during a write and catches the status enum mid-transition. The finance agent reads it ninety seconds later, after the write is complete.

Both queries were technically correct based on the data available when they ran. Yet the agents now hold different beliefs about the same order. Neither has any signal that the other agent, working from the same source, reached a different conclusion.

Two agents querying the same shared data

Figure 2. Two agents querying the same shared data can reach conflicting conclusions when they read it at different points in time.

What Actually Has to Change

Three things, in order. This is the same sequencing argument behind the AI-Readiness Quadrant framework I laid out: identity first, then access enforcement, then observability. Skipping ahead doesn't work any better here; it just means patching symptoms instead of fixing what's underneath.

AI Readines Quadrant

Figure 3. Federation places particular strain on runtime access enforcement and lineage and observability within the AI Readiness Quadrant.

Federated results need explicit freshness and consistency metadata that an agent can read and reason about. The system cannot assume that every source is synchronized when that synchronization was never guaranteed. If a query joins three systems, the response should report how stale each source was when the query ran, not just return the joined rows.

Access enforcement also needs to happen at the level of each call. Every request should be evaluated against the data it touches instead of relying on a session that was authorized once and left open. That is a heavier operational burden than most sharing and federation layers were designed to support. It still needs to happen before agent query volumes exceed what today's coarse access models can handle.

When agents from different vendors need to agree on the same entity, that agreement must be established deliberately through the identity and catalog layers. It does not emerge automatically because every system adopted the same open table format. Iceberg makes the data physically reachable by more than one engine. It does not make two independent agents agree on what that data means.

The Trust Problem Federation Was Never Asked to Solve

Federation got adopted to kill a cost and integration problem: too many copies, too much ETL, too much arguing about which table was current. It solved that. What it's being asked to solve now, mostly by default and without anyone deciding it deliberately, is a trust problem: whether an autonomous system acting on a federated answer is acting on something coherent.

It was never built for that job. Fixing the foundation underneath agentic systems means treating that gap as the thing to solve first, before scaling the number of agents that depend on an architecture that was never asked to guarantee coherence in the first place.

AI Data (computing) Federation (information technology)

Opinions expressed by DZone contributors are their own.

Related

  • Threat Modeling Context-Aware AI: A STRIDE Walkthrough
  • Building an AI-Ready Data Layer Without Rebuilding the Enterprise
  • Why Real-Time Data Pipelines Are Becoming the Foundation of Industrial AI
  • Architecting Production AI Across Clouds: Patterns That Decide System Survival

Partner Resources

×

Comments

Loading comments

The comments didn't load

Please try again in a moment.

Likes

Loading

The next likes didn't load. Please try again.

The likes didn't load

Please try again in a moment.

See who liked this

Log in or join DZone for free to see who liked this article.

Log in Join for free
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook