Your AI-Generated Reports Have No Paper Trail

I spent years building data pipelines, mostly in Snowflake, in a regulated banking environment. For most of that time, lineage was straightforward: data moves through a transformation, and you can trace exactly where every number came from. That changed the moment LLM functions started showing up inside those same pipelines.

The pattern showed up the same way every time. A Cortex function would generate a narrative, a summary, a piece of text meant for a report someone downstream would rely on. The data going into the report was fully traceable. The text coming out of the LLM was not. I could tell you which table fed a number. I could not tell you which prompt, which model version, or which configuration produced a specific sentence. That gap kept showing up, and it bothered me enough that I eventually went and checked whether the tools I was using were ever going to close it on their own.

The realization did not come from a single dramatic moment. It came from working backward. After a Cortex function ran and produced output that ended up in a report, I tried to reconstruct what had actually happened: which prompt had been active, what parameters had governed the run. The query history showed the function had executed. It showed the timestamp, the user, the warehouse. What it could not show me was what had been sent to the model or what version of the prompt template had produced the result. I was looking at evidence that something had happened, with no record of what that something actually was.

They are not going to close it. I looked across the major data governance and lineage tools commonly used in this space: dbt, MLflow, Apache Atlas, Snowflake's own native tooling, Informatica. Every one of them is genuinely good at tracking structured data through deterministic transformations. Table versions, transformation logic, pipeline runs, all well covered. None of them, as far as I could find, natively captures what happens the moment an LLM enters the picture: which prompt template was used, what version of it, what parameters the model ran with, or how the output maps back to a specific section of a specific report.

That is not a criticism of those tools. They were built before this problem existed in its current form. But it means that right now, if someone asks you to reconstruct exactly how an AI-generated paragraph in a regulated report came to exist, in most environments, you cannot. You have the output. You do not have the chain that produced it.

The Question That Kept Coming Up

The question that kept surfacing in compliance conversations was some version of: which version of this process produced this output? Not just which data, not just which model, but which version of the entire process, prompt included, was active at the time a specific report section was generated. That question is unanswerable with standard data governance tooling, because prompts are not treated as versioned process components the way SQL transformations are. A dbt model gets a version, a run ID, a test result. A prompt template gets saved somewhere, maybe, by someone, whenever they remember to. The governance gap is not subtle. It is the difference between a process that is version-controlled end to end and one that treats its most consequential step as an untracked artifact.

Regulatory frameworks are beginning to reflect this expectation even if they do not yet spell out the technical solution. The EU AI Act, in Article 12, requires that high-risk AI systems allow for the automatic recording of events over the lifetime of the system. That language is more specific than most summaries suggest: it rules out manual log exports or after-the-fact human notes as a substitute. It requires automatic, system-level capture. That is exactly the kind of infrastructure that does not exist in most LLM reporting pipelines today.

The Fix: Build It Into the Pipeline

The fix, for me, was not to wait for a vendor to solve this. It was to treat prompt lineage as something that belongs inside the pipeline from day one, not something to bolt on after the fact. Concretely, that meant logging the prompt template and its version, the model and its configuration, and a hash of the output, automatically, every time the function ran, as part of the same process that writes the report, not as a separate step someone has to remember to do later.

The architecture has six layers, each capturing a specific category of governed artifact. Source data provenance tracks which tables and rows fed the model. Transformation logic captures which pipeline version prepared the data. Prompt construction records exactly what was sent to the model, including template ID, version, variables, and rendered prompt hash. Model parameters log the specific model version, temperature, and inference settings. Output integrity creates a tamper-evident hash of the generated text. Report context maps the output to a specific filing section, including approval records.

If You Can Only Start With One Thing

If I could only implement one layer first, I would start with output hashing. The reason is practical: everything else in the lineage chain can potentially be reconstructed or approximated after the fact. You can check version control for the prompt template. You can look at model documentation for parameters. But once a generated output has been filed in a regulatory document and time has passed, there is no way to prove retroactively that what was filed matches what the model produced, unless you captured a hash at the moment of generation. Output hashing is the layer that makes the rest of the chain defensible. Without it, even a complete lineage record can be questioned, because you cannot prove the output it describes is the output that was actually filed.

What to Do Starting Now

A few things I would tell another data or IT leader looking at this same gap:

• Inventory every place an LLM touches something that ends up in a regulated or customer-facing document. You cannot fix what you have not mapped.
• Do not assume your existing data governance stack already covers this. Check specifically whether it captures prompt versions and model configuration, not just source data.
• Build the logging into the pipeline itself, not as a side process. If it is optional or manual, people will skip it under deadline pressure, and you will be back where you started.
• Start with output hashing if you have to prioritize. That single layer gives you tamper-evident proof of what was generated, which is the foundation everything else depends on.
• Treat this the same way you treat any other production logging you cannot afford to lose. Once a report goes out, the question is not whether someone will eventually ask how it was produced. It is when.