What is the right first question when deciding whether to automate something?

Ask what a wrong answer costs and whether it can be reversed, rather than asking whether the task is technically automatable. Almost everything is automatable now, so that question no longer separates good candidates from bad ones and quietly turns the decision into a default.

Why does detection speed matter more than error rate?

Because an error you find in minutes costs a correction while an error you find in a quarter costs an investigation, a backfill, and a conversation with everyone affected. A process wrong two percent of the time that reports itself immediately is safer than one wrong a tenth of a percent of the time in a way nobody notices.

What kinds of work are clearly safe to automate?

Work where a wrong answer is cheap and visible: gathering information, cross-referencing records, checking formats, drafting a first version, and any deterministic transformation with a known correct output. These are also where most of the hours go, so automating them gets you most of the available benefit.

What should stay with a person even when a model could do it?

Decisions where the cost is asymmetric and irreversible, where someone must justify the outcome to a customer or a regulator, and where the right answer depends on context that exists nowhere in your data. Those three conditions describe a small share of the tasks and a large share of the risk.

Does keeping a human in the loop actually work as a safeguard?

Only when the person can realistically intervene, which means they need the evidence assembled, enough time, and a genuine expectation that rejecting is normal. A reviewer approving forty items an hour is providing a signature rather than oversight, and that arrangement fails exactly when it matters.

What happens if you automate a process that was already broken?

It runs faster and produces more of the same wrong output, which is why the 2025 DORA report describes AI as amplifying what is already there rather than fixing it. Automating a process you cannot describe correctly also removes the human judgment that had been silently compensating for the gaps.

How do you decide which parts of a single workflow to automate?

Split the workflow into the work and the judgment, then automate the work completely and leave the judgment with a person who now receives the evidence pre-assembled. This usually removes most of the elapsed time without moving the accountability, because the slow part was gathering rather than deciding.

What is the most common mistake in automation decisions?

Automating the visible part of a job rather than the expensive part, because the visible part is easier to describe. The expensive part is usually a judgment made in a few seconds by someone experienced, and that judgment is hard to write as a specification precisely because it depends on context nobody wrote down.

How do you know when something you automated should be reverted?

When the correction rate stops falling, when errors are being found downstream by other people rather than by your own checks, or when the reviewers have started approving everything. All three mean the automation is producing work that people accept without checking it.

Should a task be automated if it only happens a few times a month?

Usually not, unless the cost of doing it wrong is high and the automation makes it more reliable rather than merely faster. Rare tasks give you very little data about whether the automation is working, so a defect can persist for a long time before enough cases accumulate to reveal it.

How does this apply to agents rather than fixed automation?

The same questions apply with more force, because an agent chooses its own steps and its failures are less predictable. That makes reversibility and how much breaks when it fails the deciding factors, so permissions should be limited to what the agent needs and destructive actions should need explicit approval.

How to choose what not to automate: Reveneau

The question that used to decide these things has stopped working. For a long time "can this be automated?" did real filtering, because the answer was often no, or yes but at a cost nobody would pay. It sorted the list for you.

That question now returns yes for almost everything. Which means it no longer helps you choose, and a decision procedure that does not help you choose is only a default.

So the useful question is now the opposite one: what to leave alone. It has better answers than it used to, because the reasons for leaving something alone were never about technical feasibility in the first place.

Three questions, in order.

1. What does a wrong answer cost, and can you undo it?

Not how often it will be wrong. Everything is wrong sometimes, including the person currently doing it. The number that matters is what a single wrong answer costs and whether the cost can be reversed.

A misfiled internal document costs a few minutes. A wrongly formatted export costs a rerun. A wrong price on an invoice sent to a customer costs a refund, a support conversation, and a small amount of trust, none of which you get back by fixing the code. A wrongly deleted record may cost nothing or may cost everything, depending on a backup policy somebody set up two years ago.

Reversibility is the most important property here. Wrong-and-undoable is cheap at almost any frequency. Wrong-and-permanent is expensive at any frequency, which is why frequency is the wrong measure to use. A once-a-year mistake that cannot be undone is worse to be responsible for than a daily mistake that corrects itself.

Ask it concretely: if this produces a wrong answer tomorrow, what has happened by Friday, and can we undo it?

2. How quickly would you find out?

This is the question people skip, and it decides more outcomes than the first one.

An error caught in minutes costs a correction. The same error caught in a quarter costs an investigation, a backfill, a set of apologies, and the uncomfortable meeting where someone asks how long this has been going on. The severity is set almost entirely by the delay.

Which produces a result that feels wrong until you think about it: a process that is wrong two percent of the time and shows its errors immediately is safer to automate than a process that is wrong a tenth of a percent of the time in a way nobody notices. The number that matters is the undetected error rate, and it is a different number from the error rate.

So before automating anything, describe how a wrong result becomes visible. If the answer is that someone would eventually spot it, that means no. If the answer involves a customer telling you, that is worse than no. Good answers name a check that fails, a reconciliation that will not balance, or a downstream step that refuses the input.

This is also the strongest argument for automating a task and its verification in the same piece of work rather than in sequence. Automation without detection produces a slower feedback loop, because the mistakes it makes go unnoticed for longer than the manual version's would. The mechanics are in how to write your first eval suite.

3. Who has to explain it afterwards?

Some decisions come with an obligation to justify them to someone with standing to object: a customer who was declined, a regulator asking why, a court.

For those, a person needs to have been in a position to change the outcome. "The system decided" is not an answer any of those three audiences accept, and it is a particularly bad answer to give under pressure with no record of who could have intervened.

This is a legal and design question rather than a technical one, and the honest version has a condition attached. A human in the loop only counts if that human can realistically intervene: they need the evidence assembled, they need enough time, and they need a working culture in which rejecting is a normal outcome. Someone approving forty items an hour is supplying a signature, and the arrangement fails at exactly the moment it was built for. The design considerations are set out in when a human must stay in the loop.

The split that usually resolves it

Most real workflows are not one thing. They are a long stretch of work with a short judgment at the end, and the two parts have completely different answers.

Take a credit review, a clinical intake, a diligence review of a codebase, an incident triage. In each one, most of the elapsed time goes on gathering: pulling records, cross-referencing sources, checking formats, assembling a summary, finding the three items that need attention. Then a small amount of time goes on a decision that draws on experience, context, and an understanding of what the organisation is willing to accept.

Automate the first part completely. It is where the hours are, the cost of a mistake is low because the next step catches it, and errors appear fast because a person is about to read the output.

Leave the second part with a person, and hand them the assembled evidence so the decision takes two minutes instead of forty. The same person is accountable, and the decision now takes two minutes. That is the pattern of most automation projects that are still working a year later.

The common failure is automating the visible part rather than the expensive part, because the visible part is easier to describe. The expensive part is often a judgment made in eight seconds by someone with ten years of context, and it is hard to write as a specification precisely because that context was never written down.

Automating a broken process makes it worse

If nobody can describe how the current process works, automating it will not repair it. It will run the same process faster and produce more of the same output, and it will remove the human judgment that had been quietly compensating for the gaps nobody documented.

The 2025 DORA report, drawn from roughly 5,000 respondents, says this plainly: "AI doesn't fix a team; it amplifies what's already there". The same report finds AI adoption relating positively to throughput and negatively to delivery stability, which is the amplification effect showing up in numbers. A process with a weak point produces more output through the weak point.

There is a related warning about our ability to judge these decisions from the inside. A METR study published in July 2025 followed 16 experienced open-source developers working on their own repositories and found they were 19 percent slower when using AI tools, while expecting a 24 percent speedup and believing afterwards that they had been 20 percent faster. It is a small study on a specific population and it should not be given too much weight. What it does establish is that the feeling of being faster is not evidence of being faster, so an automation decision defended with "it feels quicker" has not been evaluated at all.

When to undo it

Automation decisions are reversible, and reverting one should be an ordinary event rather than an admission of failure. Three signals worth acting on:

The correction rate stops falling. Early corrections are learning. Corrections still arriving at the same rate after a few months mean the process is not converging, and it probably will not.

Errors are being found by other people. If defects appear later in the process, in support tickets or in another team's work, your plan for detecting errors was wrong. That is the condition from question two, and it is worse than the automation being wrong.

Reviewers approve everything. An approval rate near a hundred percent means either the automation is excellent or the review has become a formality. Find out which, because the second one is common and looks identical from the outside.

Thanks to the operators who told us which of their automations they turned off and why. In almost every case the task was automatable exactly as promised, and the thing that broke was the assumption that somebody would notice when it went wrong.

The question was never whether a machine can do it. It is what happens on the day it does it wrong.

Related guide: AI agent permissions and guardrails.

Sources

2025 DORA report announcement, Google Cloud: roughly 5,000 respondents; AI adoption relates positively to throughput and negatively to delivery stability, and amplifies a team's existing strengths and weaknesses.
METR, early-2025 study of experienced open-source developers: 16 developers were 19 percent slower with AI tools while expecting a 24 percent speedup and believing afterwards they had been 20 percent faster.

How to choose what not to automate