How to Write an AI Incident Runbook Your On-Call Team Will Actually Use

Why Most AI Runbooks Fail at 3 AM

There are two kinds of runbooks. One proves you planned for incidents. The other tells the engineer paged at 2:47 AM exactly what to do in the next 90 seconds.

Most AI runbooks are the first kind. They satisfy an audit. They live in Confluence. They say things like "The team will engage the kill-switch process." That sentence does nothing for an engineer at 3 AM. The engineer needs the kill-switch location, the access credential, the activation command, and the confirmation signal. They need them in that order. They need them right now.

When the runbook does not answer those questions, the team guesses. Guessing during an AI incident is costly. AI failures often stay hidden until the damage is done. Here is what that looks like. A tool-calling agent starts producing bad outputs at 2:47 AM. The engineer checks the runbook. The kill-switch section says "contact the platform team." The platform lead is on vacation. No backup is named. The engineer finds a manual API call in an old Slack thread and uses it to shut down the agent. The audit trail breaks. Forty minutes later, the review team cannot reconstruct what the agent did between 2:47 AM and 3:28 AM. The log preservation step was never written down.

This article gives you the six sections that make the difference.

Before going further: use the free stability auditor tool to check that your pipeline can detect its own failure modes. A runbook section for a failure you cannot see is not a safety net. It is a false one.

What Makes an AI Incident Different From a Standard Software Incident

AI incidents have four traits that normal software incidents do not. Each one needs its own runbook section. A generic SRE template will not include them.

Silent degradation. A database going offline throws an error. An AI model giving bad answers does not. Output quality collapse, model behavior drift, and prompt injection events all produce wrong answers. Those wrong answers look correct to any system watching for HTTP 500s. The on-call team is often paged by a downstream signal long after the failure began. Customer complaints or an SLO breach on a business metric are common triggers. A runbook that starts at "service is down" misses the first 40 minutes of an AI incident.

Blast radius ambiguity. OWASP LLM Top 10 (2025) names Excessive Agency (LLM06) as the failure where a model or agent acts beyond its intended scope. In a tool-calling agent, the blast radius is set by the gap between the first bad action and the kill-switch activation. Without documented tool boundary limits and a defined blast radius per tool, the post-incident review cannot answer: "What did it do?"

Rollback complexity. Rolling back a code deployment undoes a known change. Rolling back a model version is harder. The prompt may have changed. The system context may have changed. The new model may have added behaviors the old version never had. The rollback section of an AI runbook must name the last known good version of both the model and the prompt template. It must also name the rollback command and the signal that confirms the rollback worked.

Prompt injection as an incident class. As detailed in the adversarial validation patterns guide, a behavior change triggered by user input is not a normal bug. It is its own incident class with its own response procedure. The runbook must include a pre-call validation step. It must also include a separate documented response for injection events, apart from the general "service degradation" response.

The 6-Section AI Incident Runbook Template

The six sections below map directly to named controls from the 12-Control AI Incident Readiness Audit. That mapping matters. The runbook is what your on-call team picks up. The audit confirms that the controls the runbook points to actually exist in your production system. A runbook section pointing to a control you have not built is a false safety net.

Section 1: Kill-Switch Location and Activation Procedure

Maps to /incident-readiness/ Control 1 (Kill-switch). This section lives in the runbook itself, not in a separate document. The on-call engineer needs it in the first 60 seconds. There is no time to go looking for it.

Required documentation fields:

• Kill-switch location: exact URL or CLI path, not "the admin panel"
• Access credentials: named secret location (vault path, not the credential itself)
• Activation steps: numbered, imperative commands
• Confirmation signal: what the system emits to confirm the kill-switch activated
• Primary owner: named individual (first name and last name, not a role)
• Backup owner: named individual, with personal contact method outside of Slack

Failure mode when missing: The engineer pages the "platform team" at 3 AM. No one responds for 20 minutes. The engineer improvises a shutdown using a method that breaks the audit trail. The review team has no record of when the agent was actually stopped.

Section 2: Tool Boundary Documentation and Blast Radius Limits

Maps to /incident-readiness/ Control 2 (Tool boundary docs). This section documents what the agent can do, not what it should do. That distinction matters during an incident. The team needs to reconstruct what the agent actually did.

Required documentation fields:

• Tool inventory: every tool the agent can invoke, with the exact scope of each
• Blast radius per tool: maximum irreversible action per tool call (e.g., "can delete up to N records per call")
• Tool call log location: where to find the complete tool call history for the incident window
• Scope enforcement mechanism: how tool boundaries are enforced in production

sincllm-mcp v2.0.0 is sincllm's own production MCP deployment. It has 12 scoped tools. Each tool has a defined scope limit and a pre-call validation gate. The blast radius is bounded before the tool runs, not after the incident is found. That pre-call validation pattern is documented in the adversarial validation guide. It belongs in this runbook section.

Failure mode when missing: The team knows the agent misbehaved but cannot tell which tools it called or in what order. The blast radius is unknown. The review takes days instead of hours.

Section 3: Rollback Procedure for Model and Prompt Versions

Maps to /incident-readiness/ Control 9 (Rollback). Model rollback is not the same as code rollback. Test this section before an incident happens, not during one.

Required documentation fields:

• Last known good model version: version identifier, not "the previous version"
• Last known good prompt template: git SHA or equivalent version tag
• Rollback command: exact CLI command or UI path
• Rollback verification signal: the output the system produces to confirm the rollback succeeded
• Rollback test date: the last date this procedure was executed in a non-production environment

Failure mode when missing: The team tries a rollback during the incident and learns the procedure has never been tested. The prior model version is no longer available. The incident stretches from 40 minutes to 4 hours.

Section 4: Escalation Path and Communication Protocol

Maps to /incident-readiness/ Control 8 (Eval coverage) and the audit criterion 8 (On-call and incident response) from the 10-Point AI Vendor Audit. Escalation paths that name roles instead of real people break at 3 AM when no one is filling the role.

Required documentation fields:

• Severity thresholds: the measurable conditions that trigger each escalation level
• Escalation contacts: named individuals at each level, with out-of-band contact methods
• SLO breach trigger: the specific metric value that triggers customer communication
• Customer communication template: a pre-written status update for the first 30 minutes
• Internal notification channel: where incident updates are posted during the event

Failure mode when missing: The engineer spends 25 minutes deciding who to call. The customer message is made up on the spot. The incident summary written next morning is built from memory, not from a timestamped record.

Section 5: Audit-Trail Check and Evidence Preservation

Maps to /incident-readiness/ Control 3 (Audit-trail completeness). Saving evidence is the first step in most AI incidents, not the last. A rollback that erases the audit trail makes the post-incident review impossible.

Required documentation fields:

• Log location: exact path or service URL for the AI pipeline logs
• Log retention window: how long logs are retained before automated deletion
• Evidence preservation command: the exact command to export and preserve logs before executing a rollback
• State snapshot procedure: how to capture the current model state before rolling back

Failure mode when missing: The team runs the rollback without exporting logs. The rollback succeeds. The incident is marked "resolved." Three days later the review team finds that the rollback overwrote the audit trail for the incident window. The root cause cannot be found.

Section 6: Post-Incident Review and Eval Coverage Check

Maps to /incident-readiness/ Control 8 (Eval coverage). The post-incident review is not a look-back. It is a production control update. Every AI incident not captured in an eval case will happen again.

Required documentation fields:

• Post-incident review template: the standard set of questions to answer within 48 hours
• Eval coverage trigger: the condition that requires adding a new eval case (any incident that could not have been detected by existing evals)
• Eval case owner: the named individual responsible for adding the new eval case
• Eval coverage report: where the current eval coverage state is documented

Failure mode when missing: The incident is resolved. The postmortem is written. The same failure happens again 6 weeks later. No eval case was added. No runbook update was made.

The six sections above are your runbook template. The reference tables follow.

Table 1: 6-Section AI Incident Runbook Template

Table 2: AI Incident Type to Runbook Section Mapping

The 55-Hour Problem: What Unstructured Incident Response Really Costs

A real example shows the cost. In one client engagement, sincllm's own production work recovered 55 hours per month by writing down and operationalizing AI pipeline controls. That number comes from a single engagement on sr-demo-ai.com. It is not a guarantee for any other team or system.

The same logic applies to incident response. Teams with no usable runbook spend that same time guessing through incidents and rebuilding state afterward. The cost is not just the length of the incident. It grows in three ways.

First, every improvised response creates knowledge that lives only in someone's head. The engineer who found the manual API call at 3 AM is now the only person who knows it. When they leave the team, it leaves with them.

Second, every incident without an eval coverage update leaves a gap in the test suite. The failure that caused the incident will not be caught by any automated check. The next time it happens, it will look like a new incident.

Third, every blast radius that cannot be reconstructed requires manual review of downstream systems. Those systems may have acted on the agent's bad output. That review has no clear endpoint. The team does not know when they have reviewed enough until they confirm every system the agent touched.

The AI system stability framing explains why escalation thresholds must be written in the runbook, not just escalation paths. A threshold written before the incident is used in seconds. A threshold figured out during the incident takes minutes of deliberation under pressure. The SLO window may not have those minutes.

Six Things to Test in Your Runbook Before You Need It

A runbook that has never been tested is just a plan. Run these six drills every quarter. Each one has a pass condition.

• 01 Kill-switch test (dry run, no production traffic). Execute the full kill-switch procedure in a staging environment. Pass: named primary owner completes activation in under 60 seconds; confirmation signal fires; backup owner executes the same procedure independently in under 90 seconds.
• 02 Rollback test. Execute the model and prompt rollback procedure against the current last known good version. Pass: rollback completes in under 15 minutes; verification signal fires; the system produces the expected output from the prior version.
• 03 Blast radius audit. Ask every on-call engineer to list the tools the agent can invoke and the blast radius per tool from memory. Then compare to Section 2 of the runbook. Pass: all on-call engineers produce a list that matches the documented tool inventory with no additions or omissions.
• 04 Log access drill. Page an on-call engineer without warning. Ask them to pull the tool call log for the last 30 minutes of the AI pipeline. Pass: the engineer retrieves the log in under 5 minutes using only the information in Section 5 of the runbook.
• 05 Escalation drill. Contact every named individual in Section 4 via their out-of-band contact method. Confirm they know they are on the escalation path. Pass: 100% of named individuals confirm their role; no one responds with "I thought that was handled by the platform team."
• 06 Eval coverage check. Review the eval suite against the last three incidents. For each incident, confirm that at least one eval case was added within 48 hours of the post-incident review. Pass: every incident has a corresponding eval case; the eval suite coverage report in Section 6 is current.

The functional safety engineering standard that production AI runbooks borrow from treats testing procedures as part of the safety case, not as optional extras. A kill-switch that has never been tested under controlled conditions is not a safety control. It is a hypothesis.

What Your Vendor's Runbook Should Cover (and How to Find Out If It Exists)

Your runbook is only as good as your vendor's incident response documentation. If your AI vendor cannot be killed-switched by your team on its own, the kill-switch section of your runbook is incomplete. If your vendor does not give you tool call logs for your agent's production activity, the audit trail section of your runbook cannot be executed.

Ask the vendor directly for their on-call and incident response documentation. A vendor with a production-grade AI system can answer without a multi-week procurement process. They should be able to provide:

• The location of their status page and their SLO commitments for the AI service your team depends on
• Their escalation path for incidents that affect your environment specifically (not a generic SLA letter)
• The rollback procedure for model updates that affect your integration, including the notice period before a model version is deprecated
• Whether your team receives audit-trail exports for AI actions taken in your environment

A vendor who cannot answer these questions in writing does not have a working incident response plan. Their SOC 2 certification shows that controls exist. It does not prove those controls can be used by your on-call team at 3 AM.

Criterion 8 (On-call and incident response) from the 10-Point AI Vendor Audit turns these questions into a repeatable checklist. Use it before you sign, not after the first incident.

Conclusion

The six sections above are not a compliance framework. They are the minimum your on-call team needs to follow a procedure instead of guessing. A runbook that covers kill-switch location, tool boundary limits, rollback procedure, escalation path, audit trail preservation, and post-incident eval coverage is a runbook the team can pick up at 3 AM. No Slack thread needed. No vacation-mode auto-reply to work around.

The NIST AI Risk Management Framework (AI RMF 1.0) RESPOND and RECOVER functions set the operational documentation requirements that incident runbooks fulfill. The EU AI Act (Regulation 2024/1689) requires high-risk AI system operators to maintain technical documentation and post-market monitoring plans. Runbooks are how those requirements become real steps your team can take. The six sections above are where documentation obligations become executable procedures.

The 12 controls from the AI Incident Readiness Audit go further than the six runbook sections. Controls 4 through 7 and 10 through 12 cover sandbox separation, secret access scope, prompt-injection defenses, pre-tool-call gate, production data isolation, vendor breach exposure, and failure-mode visibility. These controls must exist in your production system before your runbook can call on them. The free AI incident readiness checklist is the fastest way to find out which ones are missing.