Building an AI Incident Response Runbook: What Engineering Teams Should Do in the First 24 Hours

Every mature engineering team has a Security Incident Response Plan, refined over years of postmortems. But when the incident involves AI — a toxic hallucination with real consequences, a discriminatory algorithmic output, PII leaking out of a RAG pipeline — the old playbook stops working. There's no CVE to patch, no clean indicator of compromise. 

What you get instead is a probabilistic failure paired with legal escalation happening in real time. One hallucination, a thousand claims: that's the shape of the risk nobody budgeted for.

Anatomy of an AI Incident

The case everyone cites is Air Canada's chatbot, which in 2024 invented a bereavement fare policy that didn't exist and promised a customer a discount based on it. According to the BC Civil Resolution Tribunal's decision in Moffatt v. Air Canada (2024 BCCRT 149), the airline argued that the bot was "a separate legal entity responsible for its own actions" — a defense the tribunal simply didn't accept, ordering the company to pay damages.

2026 has provided a sharper, messier reality. In May 2026, the Higher Regional Court of Hamm (OLG Hamm) ruled — in a judgment that is not yet final (I-4 UKl 3/25) — that a cosmetic clinic was liable for its website chatbot, which falsely attributed specialist medical titles to two directors. The court established a vital principle: if you deploy a chatbot on your site, you bear the risk of its hallucinations. Just weeks later, the Munich I Regional Court (LG München I) issued a preliminary injunction (26 O 869/26) prohibiting Google from repeating specific false claims about two publishers in its AI Overviews — reasoning that AI summaries are the operator's own content, not third-party search results. Air Canada in 2024 was the warning bell; by 2026, European courts transformed the exception into the rule: the operator owns the output.

Consider a plausible scenario: an LLM-based fintech assistant confidently advises users on a grey-area loophole to dodge transaction fees. Engineers, spotting the bug, instinctively push a hotfix to the system prompt and wipe the logs from the affected sessions to "clean up the mess and move on." When consumer protection regulators eventually come knocking, the company cannot produce evidence of what the model actually told users. A technical mistake has, legally, become something that looks a lot like deliberate spoliation of evidence.

Why This Isn't a Security Incident With a New Label

When a database goes down or a SQL injection lands, the faulty component is obvious. AI systems don't give you that clarity.

• Reproducibility isn't guaranteed – run the same prompt again at a temperature above 0.0, and you might get a perfectly safe, correct answer the second time.
• The "guilty component" is blurred – did the base model degrade, did the vector database pull poisoned context, or did a user run a clever prompt injection that slipped past the guardrails? Nobody knows in hour one.
• Evidence is volatile – the state of the context window, hidden system prompts, and chained API calls disappear without a trace unless you've built AI-grade telemetry into the pipeline from the start.

The Runbook: Hour by Hour

The first 24 hours decide whether you end up writing an internal report or fielding calls from a regulator and a journalist at the same time.

0–1 Hour: Freeze

The most expensive mistake in the first minutes is an engineer quietly trying to fix things on the fly. No rollback without preserving state first.

• Take a full snapshot of the model (if self-hosted), system prompts, weights, and pipeline configuration, including sampling parameters like top-k and temperature.
• Freeze the logs for the exact sessions involved in the incident — this is where solid logging architecture pays for itself.
• The goal is simple: preserve evidence for root cause analysis and for regulators. Without it, you can't prove you weren't negligent, even if you weren't.

1–4 Hours: Containment

Don't try to debug the model live while users keep generating fresh risk. Stop the bleeding first.

• Kill switch: cut the AI functionality at the API gateway level if the risk is assessed as critical.
• Degrade to human-only: route affected traffic to support staff using the old non-AI fallback process.
• Feature isolation: hard-block the specific topic (say, all financial questions) using rule-based filters while the investigation continues.

4–12 Hours: Scope Assessment and Legal Notification

This is where things get genuinely difficult because the incident now has to be classified against real regulatory deadlines. Article 73 of the EU AI Act sets out reporting obligations for serious incidents involving high-risk AI systems, and the timelines are unforgiving:

• Standard cases: report immediately after establishing a causal link, and in any event no later than 15 days after becoming aware of the incident.
• Widespread infringement (or incident types defined under Article 3(49)(b)): report immediately, and no later than two days after becoming aware.
• Death of a person: report immediately once a causal link is suspected, and no later than 10 days after becoming aware.

If the incident involves the model surfacing personal data from training or retrieval, GDPR's 72-hour breach notification window kicks in alongside the AI Act obligations. The team also needs to work out how many users were affected and whether the incident maps onto the Manage and Measure functions of the NIST AI Risk Management Framework, which auditors and, increasingly, regulators expect to see referenced in your own process.

12–24 Hours: Communication

Draft the public postmortem. This window is about being transparent with users and stakeholders without conceding legal liability in the wording. Stick to the facts: safety protocols triggered, the AI feature was moved into a safe fallback mode, and an investigation is underway.

Why Legal Counsel Belongs in the War Room in Hour One, Not Day Three

In a typical DevOps process, lawyers get looped in when it's time to draft an apology to angry customers. With an AI incident, legal and compliance need to be in the room from hour one — and the reason is legal privilege.

If engineers are hashing out the incident on a public Slack channel with messages like "our model's spouting discriminatory nonsense again, we screwed up the dataset," those messages become discoverable in litigation and can be weaponized against the company. In my practice, the very first action I take during an escalation is establishing a privileged communication channel and issuing a strict directive: "Do not diagnose the legal fault in Slack; describe only the technical symptoms." The goal isn't to obscure the truth, but to ensure that a stressed engineer's rushed hypothesis isn't treated as a binding confession of corporate negligence in a courtroom.

Legal counsel redirects that conversation early, and — just as importantly — makes sure the technical rollback doesn't look like spoliation of evidence once regulators or plaintiffs start asking what happened to the original logs.

Runbook Template: First 24 Hours

In generative AI, technical ambition has to be matched by operational discipline. The structural difference between an engineering team that survives an AI incident and one that sinks the company lies entirely in the actions taken during the first few hours: evidence discipline and immediate legal integration. Build this runbook, establish your privileged channels, and rehearse the freeze protocol before your model's output becomes the subject of a breaking news story, not after.