What does a rollback plan for an AI-written feature need to cover?

Four things: a kill switch that turns the behaviour off through configuration rather than a deploy, a written answer to what data the feature wrote and whether the old code can still read it, a named person with authority to turn it off without calling a meeting, and a practice run in production before the plan is needed. Reverting the code is the part that already works.

Why is reverting the commit not enough?

Because code is reversible and its effects are not. While the feature was live it may have written rows in a new format, sent messages, triggered webhooks that other systems already processed, filled caches with values computed under the new logic, and run a migration that dropped a column, and none of that is undone by a git revert.

What is the difference between a kill switch and a revert?

A kill switch changes behaviour through configuration in seconds without a build or a deploy, while a revert requires the full pipeline to run and is therefore slower at the exact moment speed matters. Build the switch when you build the feature, because adding one during an incident means releasing new code to fix code.

How do you handle data written by a feature you are rolling back?

Decide before launch which of four states applies: the data is compatible with the old code, it is ignorable, it needs a repair script, or it cannot be undone at all. Write a repair script alongside the feature for the third case, and treat the fourth as a reason to require an approval step for the feature rather than a flag.

Does AI-assisted development change rollback planning?

It changes the volume and the confidence around each change. More change reaching production per week raises the chance that any one of them is the wrong one, and the METR study from July 2025 found developers believed they were twenty percent faster while measuring nineteen percent slower, which suggests felt speed is not a reliable guide to how carefully a change was checked.

What is an expand and contract migration?

A pattern where the new column or table is added first, both old and new code paths write to both formats for a period, and only after the change has proved stable is the old format removed. It makes the rollback period real, because during the overlap the previous version of the code can still read everything the new version wrote.

Who should be allowed to trigger a rollback?

A named on-call engineer, with the authority written down before the incident. Requiring consensus or a manager's approval turns a two-minute action into a forty-minute discussion, and the cost of an unnecessary rollback is far lower than the cost of a delayed one.

How do you know when to roll back rather than fix forward?

Roll back when you do not yet understand the cause, and fix forward only when you do and the fix is small. Debugging a live incident under pressure produces its own defects, so the default should be to stop the damage first and find the cause afterwards with the system in a known state.

What should you rehearse before launch?

The actual rollback, in production, on a real feature, at a time you chose. Measure how long it takes from start to finish and what breaks that nobody predicted, because an untested rollback plan is a paragraph in a document rather than a capability the team has.

How do feature flags fit into this?

A flag is the mechanism that makes the kill switch instant, and it is only useful if the code path behind it is genuinely optional. A flag wrapped around a schema change or a message that has already been sent looks like control but gives you none, so check what the flag can actually stop.

What breaks in a rollback that people forget?

Anything that left your system. Emails and notifications are gone, webhooks have been processed by third parties who will not reprocess them, and downstream services may have cached results from the new behaviour. List every outbound effect a feature has before you write the plan.

How long should the rollback period last?

At least as long as it takes for the failure to be discovered, which for defects that show no error is often days rather than minutes. Keeping the old code path readable for a full release cycle costs little and covers the case where the problem appears in a monthly report rather than in a dashboard.

What a rollback plan looks like for AI-written features: Reveneau

"We can just roll it back" is one of the most confidently stated sentences in software, and one of the least tested. In most of the incident reviews we take part in, the plan existed and nobody had ever run it.

Reverting the code is genuinely easy. One command, and it works. That part has been solved for twenty years and it is not what causes the real problems.

What causes them is everything the feature did while it was live. Rows it wrote in a format the previous version cannot read. Emails it sent. Webhooks that other companies' systems have already processed and will not process again. A cache full of values computed under the new logic. A migration that dropped a column and is now the reason the old code will not start.

Code is reversible. Effects are not. A rollback plan that only addresses the first half is a plan for the easy half.

Why this needs saying again now

The mechanics have not changed. Two things around them have.

The first is volume. When more change reaches production every week, the chance that any given change is the wrong one goes up, and the time each one gets before it is released goes down. The 2025 DORA report, drawing on nearly 5,000 respondents, found AI adoption relating positively to throughput and negatively to delivery stability. More changes, less stability. That combination is the case rollback plans exist for.

The second is confidence, and it is the more interesting one. A July 2025 study by METR followed 16 experienced open-source developers working on repositories they knew well. Using AI tools, they took 19 percent longer to complete tasks. They had expected a 24 percent speedup, and afterwards they still believed they had been about 20 percent faster.

Read that last part again, because the plan has to be designed around it. The gap was not between expectation and reality. It was between reality and what people believed after doing the work. Feeling fast changes behaviour: bigger increments, less checking, more confidence at the moment of deploy. Whatever the true productivity effect turns out to be, plan for a team that believes it is moving faster than it is.

The four things a real plan contains

1. A kill switch that is configuration, not a deploy

The difference between turning a behaviour off in ten seconds and turning it off in twenty minutes is the difference between a short problem and an incident.

A configuration change takes effect immediately. A revert has to go through build, test, and deployment, which is exactly the pipeline whose speed you cannot control at the moment you need it most. Build the switch when you build the feature. Adding one during an incident means releasing new code to fix code, under pressure, which is how the second outage starts.

Check what the switch can actually stop. A flag wrapped around a code path that includes a schema change or an outbound message looks like control but gives you none.

2. A written answer to "what did it write?"

Do this before launch, on one page, in plain sentences. Every feature that writes data falls into one of four states:

Compatible. The old code reads the new data correctly. Nothing to do, and this is worth designing for deliberately rather than hoping for.

Ignorable. The new data exists and the old code does not look at it. Fine, as long as somebody remembers to clean it up later.

Needs repair. The old code will misread it, so a script has to fix the rows or move them where the code will not read them. Write that script alongside the feature, not during the incident. It takes an hour when nobody is under pressure.

Cannot be undone. Money moved, a message was sent, an external system was told something. This state is not a rollback problem. It is a reason the feature should have been behind an approval step rather than a flag, which is the decision covered in when a human must stay in the loop.

The pattern that makes the first state achievable is expand and contract. Add the new column, have both paths write both formats for a period, and only remove the old format once the change has proved it works. During that period the rollback option is real rather than theoretical.

3. A named person who can turn it off alone

Write down who is allowed to trigger a rollback, before the incident, and give them the authority to do it without a meeting.

Requiring everyone to agree turns a two-minute action into a forty-minute discussion, and the discussion happens while the system is broken. The cost of one unnecessary rollback is small. The cost of a delayed one grows every minute.

The related decision is when to roll back rather than fix forward. Our rule: roll back when you do not yet understand the cause, fix forward only when you do and the fix is genuinely small. Debugging live under pressure produces its own defects, and a system in a known state is worth more than a system that might be fixed.

4. A practice run, in production, before you need it

This is the one that gets skipped and it is the only one that proves the other three.

Pick a real feature. Roll it back on purpose, at a time you chose, with everyone awake. Time the whole thing from decision to verified recovery. Write down what broke that nobody predicted, because something always does: a dashboard that reads a column the revert removed, a downstream service holding a cached response, an alert nobody had connected to the flag.

An untested rollback plan is a paragraph in a document. A practised one is a capability the team actually has, and the difference only becomes visible on the day something goes seriously wrong.

What people forget every time

Anything that left your system. Emails and notifications are gone. Webhooks were received and processed by parties who will not reprocess them because you changed your mind. Any partner who read your API during that period has the answer from the new behaviour stored somewhere.

Downstream caches. Yours and other people's. Reverting the code does not revert a value another service computed and kept.

The monitoring gap. Metrics and logs that only the new code produces stop when the new code stops, which is precisely when you most want to see what is happening.

How long the rollback period needs to be. Failures that produce errors appear in minutes. Failures that produce no error appear in a monthly report. Keeping the old path readable for a full release cycle costs little and covers the case where the problem is a wrong number rather than an error rate. The related mistake of testing this in an environment that does not behave like production is covered in why your staging environment is lying to you.

Where verification fits

A rollback plan is what you do when the checks did not catch it. The cheaper choice is catching more of it earlier.

Automated checks written from the specification, running in the pipeline and able to block a merge, are what let a team accept generated work at volume without reading every line. The mechanics are in evals in CI for coding agents. They will not catch everything, and a suite that has never failed is telling you nothing, which is the failure described in when evals give false confidence.

So both. Verification reduces how often you need the plan. The plan handles the cases verification could not have anticipated, and those cases are permanent, because a check tests a rule somebody thought of.

The short version to put in a template

Before any feature that writes data or connects to an external system goes live, four lines: how it is turned off, what it wrote and whether the old code can read it, who can turn it off, and the date of the last practice run.

Four lines. Most teams cannot fill in the fourth, and the fourth is the one that makes the other three true.

Thanks to the engineers who explained to us rollbacks that did not go to plan. The revert always worked perfectly. It was the fifty thousand rows written in the new format that took the whole weekend to fix.

You do not have a rollback plan until you have used it once on purpose.

Related guide: Vibe coding production risks.

Sources

METR, Early 2025 AI experienced open-source developer study, July 2025: 16 experienced developers were 19 percent slower with AI tools, expected a 24 percent speedup, and believed afterwards they had been 20 percent faster.
2025 DORA report announcement, Google Cloud: nearly 5,000 respondents; AI adoption relates positively to throughput and negatively to delivery stability.

What a rollback plan looks like for AI-written features