Incident response when nobody remembers writing it

The first question in most incidents used to be who wrote this and are they awake. It was efficient, because the author carried context that no document held.

In an AI-native team that question often has no useful answer. The change was generated, reviewed at speed, and merged three weeks ago. Nobody understands the code in detail. Response has to work without it, and the practices that make that possible are mostly the ones you set up before the incident.

Stop first, understand second

This ordering is standard incident advice and it matters more with generated code, because the ratio between the two costs has shifted. Understanding a piece of code nobody wrote takes longer than it used to. Turning it off takes exactly as long as it did.

So the sequence is: contain, then diagnose. Flag off, revert, or stop sending traffic to the affected part. Then investigate with the pressure removed. Teams that invert this spend the expensive minutes reading unfamiliar code while customers are still being harmed, and they usually end up rolling back anyway, later.

This is only available if you built it, which is the argument for progressive delivery and a practised rollback. An incident is a bad time to discover that turning a feature off requires a deploy and a person on holiday.

What replaces the author's memory

Three things, in order of usefulness during a live incident.

A change log you can read by time. Not just deploys: flag changes, configuration changes, migrations, and dependency updates. The most valuable question in the first five minutes is what changed and when, and the answer needs to be one query rather than four systems.

Traces and business-level metrics. These let a responder understand the system from evidence instead of from memory. A trace showing the request path with timings is faster than reading the code, which is the point of what to instrument when nobody typed the code.

The runbook line for the flow. One or two sentences per critical flow: what it does, what it depends on, how to turn it off. Not a document nobody has opened, a line that exists where the responder is already looking. Our post on what belongs in a runbook nobody reads covers keeping these short enough to stay accurate.

The first ten minutes

Because the author's memory is gone, it helps to have the opening sequence written down rather than improvised. Ours is five steps and it is deliberately boring.

Declare it, so everyone is working on the same thing. Then answer what changed and when, from the change log, which is one query if you built it and four systems if you did not. Then contain: flag off, revert, or stop sending traffic to the affected part, before anyone has a theory. Then confirm containment worked, using the business signal rather than the error rate, because a system can stop erroring by stopping entirely. Only then start diagnosing.

The order is what matters. Every incident that goes badly follows the same pattern: a promising theory arrived at minute three, and the containment step got postponed while it was tested. Theories are cheap and there will be another one. Customer minutes are not.

One addition specific to AI-native teams: check whether the change was one of several generated from a similar prompt at a similar time. If it was, the same defect probably exists in the other changes from that prompt, and finding that out during the incident is much better than finding it out next week.

Using an agent during an incident

Two roles, and the difference between them matters.

Reading is where it is useful. Give a model the code path and ask what it does, what it assumes, and what it does when the dependency fails. A responder who has never seen this module gets a usable summary in a minute. Treat the answer as a hypothesis to check against traces, not as fact, and check it before you act on it.

Writing under pressure is where it causes harm. Generating a fix during an incident produces a plausible change with no review, no gate run, and a team that is not in a state to evaluate it. If the situation genuinely requires a forward fix, keep it minimal, get a second person to read it, and write down what happened so the postmortem can look at it. Anthropic's guidance applies with extra force here: if you cannot verify it, do not release it [1].

The postmortem when nobody wrote the code

Postmortems in AI-native teams go wrong in a specific way. Because the code was generated, the discussion moves to the tool, and the finding becomes something like "be more careful with AI-generated code". That is not an action. Nothing changes and the same class of incident recurs.

The useful question is not who or what wrote the line. It is which control should have caught it. Every incident should end by naming one of four things:

A missing gate. The pipeline could have caught it and did not. This is the best outcome, because it produces a permanent check. Add it, as covered in the quality gates that should block a merge.

A missing check in the specification. The requirement never said this must not happen, so nothing tested for it. The fix is a criterion, and a test, on the class of change rather than on this instance.

A missing signal. It happened for a while before anyone noticed. The fix is a metric or an alert, and usually a positive signal rather than an error count.

An architectural decision that should have had a human. The change crossed a boundary that ought to be visible, which leads to the decisions that stay with your engineers.

If a postmortem cannot name one of these four, it has not finished. Our post on how to run a postmortem when AI wrote the code has the full format.

Blame moves to the process, honestly

There is an obvious temptation in an AI-native team to treat the model as the culpable party. It is comfortable and it is useless. The model produced output that a process accepted, and the process is the thing you can change.

That framing is also what keeps postmortems honest. Nobody has to defend their own code, because it was not theirs. In our experience this makes AI-native postmortems noticeably better than the ones that used to happen: the conversation moves faster to what was missing and spends less time on whose judgment was at fault. Use the advantage.

Accountability sits with the company releasing the software, not with the tool or with an individual engineer. That is the position we take publicly, and it is the only one that holds up when a customer asks who is responsible for the thing that broke.

After the incident

Two follow-ups get skipped and both matter.

Convert the finding into a mechanism within the week, while the incident still has attention. A control added a month later is a control that was never added, because the priority moved.

And check whether the same pattern exists elsewhere. Generated code repeats itself: the same misunderstanding often produced the same code pattern in three other places, since the same prompt style and the same conventions were in use. Searching for the pattern after the first occurrence is one of the highest-value hours available, and it is a task an agent does well.

Practise the parts you rely on

The controls in this page all assume something works under pressure: the change log is queryable, the flag can be switched, the rollback runs, the person on call has the permissions they need.

Each of those is an assumption until it has been exercised. The cheapest way to test them is a scheduled rehearsal on a quiet day: pick a service, turn a feature off, roll a deployment back, and time it. What you are looking for is not whether it worked but where things were slow or difficult, because those points are what turn a five-minute incident into a forty-minute one when it happens for real.

Two findings come up almost every time this is done for the first time. Somebody lacked a permission and had to wake a person who had it. And the runbook named a command that no longer exists. Both are trivial to fix on a Tuesday afternoon and expensive to discover at two in the morning.

Where this connects

Incident response depends on being able to stop a change (progressive delivery) and see what is happening (observability), and it produces most of the improvements to the gates. The main guide shows how these parts connect.