The New API Contract Is Probabilistic: Building Reliable Systems Around Unreliable Model Outputs

For decades, API design rested on a reassuring assumption: given valid input and a stable dependency, software should return a predictable result. Large language models break that assumption without breaking the API. A request can receive HTTP 200, perfectly valid JSON, and a confidently wrong answer.

That distinction matters. The network contract may still be deterministic, but the semantic contract is now probabilistic. A model endpoint does not promise one correct output; it samples a likely output from a distribution shaped by the prompt, context, model version, retrieval results, and decoding process.

Machine-learning engineer Chip Huyen puts it plainly: “LLMs are stochastic; there’s no guarantee that an LLM will give you the same output for the same input every time.” Reliable AI systems begin when developers stop treating that behavior as an exception.

Redefine What the Contract Guarantees

The traditional contract defines fields, types, status codes, and errors. The AI contract, however, has to specify the acceptable behavior: what kind of evidence the model can accept, what failure categories are allowed, when to abstain, what latency and costs budget are in place, and how to proceed in case there is not enough confidence.

Confidence has to be measured using evidence coverage, validation outcomes, or classifier calibration, but not the self-assessment of the model.

Thus, the goal changes from "function returns correct value" to something measurable: for a certain slice of traffic, the system passes some quality criteria at an acceptable frequency. Different tasks require different criteria. For example, summarizing movies does allow for some awkward sentences. Changing a customer's credit limit does not allow any inventions or ambiguities.

Put a Deterministic Envelope Around the Model

The model should be one component inside ordinary software, not the authority at the center of it. The surrounding application should normalize inputs, constrain outputs, validate results, and choose whether to accept, retry, fall back, or escalate.

Structured generation is the first layer. Use a JSON Schema, enums, required fields, and explicit null states instead of asking for “JSON” in a prompt. OpenAI, for example, reported 100% schema adherence for one model in its complex JSON Schema evaluation. That solves a parsing problem, not a truth problem. A fabricated invoice number can still be a perfectly valid string.

Semantic validation must follow structural validation. Check identifiers against source systems, dates against business rules, citations against retrieved passages, and calculated values with deterministic code. Treat every generated field as untrusted input.

The control flow should be explicit:

generate -> validate schema -> verify evidence -> apply policy

-> accept | bounded retry | fallback | human review

The important output is not merely the model’s answer. It is a typed system decision such as accepted, rejected, or needs_review, accompanied by evidence and a machine-readable failure reason.

Keep Side Effects Behind a Transaction Boundary

Probabilistic text becomes dangerous when it can directly create a refund, delete a record, or send a message. Separate proposing an action from committing it.

Let the model select only from allow-listed tools and produce typed arguments. Then let deterministic code authenticate the user, authorize the operation, verify current state, and enforce limits. Add idempotency keys so a retry cannot repeat a payment or ticket creation. For high-impact actions, show a preview or require human approval.

This architecture also limits prompt injection. Untrusted content may influence a proposal, but it should never grant the model new permissions.

Make Retries a Policy, Not a Reflex

Retries can repair malformed output or a transient timeout. They can also multiply cost, latency, and side effects while reproducing the same semantic error.

Set a small attempt budget and retry only failures that may be recoverable. Feed validation errors back in a structured form, use exponential backoff for provider faults, and stop when the remaining time or token budget is insufficient. If the evidence is missing, another generation is not a remedy; retrieval, clarification, or abstention is.

Fallbacks should match the risk. A smaller model, cached result, or rules engine may preserve availability. A safe refusal or human queue may be the correct degraded mode when correctness matters more than speed.

Test Distributions, Not Favorite Prompts

A handful of convincing demos proves little. Build an evaluation set from real tasks, known edge cases, adversarial inputs, and failures observed in production. Run each important case multiple times when sampling variability matters, and report pass rates with confidence intervals rather than one aggregate score.

Evaluation practitioners Hamel Husain and Shreya Shankar offer excellent advice: “Start with error analysis, not infrastructure.” Review traces with domain experts, classify concrete failures, then automate the checks that matter. Prefer deterministic assertions for schema, policy, and executable code; reserve model-based judges for qualities that rules cannot capture, and calibrate those judges against human labels.

Version the entire behavior-producing system: model identifier, prompt, schema, retrieval corpus, tool definitions and safety rules. Run regression suites and canary traffic before changing any of them. In production, monitor validator failures, abstentions, retries, latency, cost, and user corrections. Store redacted traces where privacy permits, because averages alone rarely explain why a system failed.

Reliability Moves Outward

The model does not need to become deterministic for the product to become dependable. Databases still fail, networks still partition, and users still submit hostile input; engineering makes those systems useful by containing uncertainty.

Generative AI demands the same discipline, applied at the semantic boundary. Define measurable behavior, constrain the output, verify claims, isolate side effects, test continuously, and fail safely. Google’s site reliability literature opens with a durable warning: “Hope is not a strategy.” With probabilistic APIs, it is not a contract either.