What done means when an agent writes the code

Ask a coding agent whether it is finished and it will say yes. That is not dishonesty, it is the absence of a signal. Anthropic's own guidance for Claude Code puts it in one line: Claude stops when the work looks done, and without a check it can run, looks done is the only signal available, so you become the only check and every mistake waits for you to notice it [1].

At one change a day, being the only check is manageable. At twenty, it is the reason your delivery slowed down. The fix is a definition of done that a machine can evaluate, agreed before the work starts.

Why the old definition stops working

Most teams already have a definition of done. It usually reads something like: code written, tests passing, reviewed by a peer, merged, deployed. Every clause in that list quietly assumes a human is applying judgment at some point, and most of them assume the same human wrote the code and therefore understands what it was supposed to do.

Both assumptions break in an AI-native team. The person reviewing did not write the code and has no memory of the reasoning behind it. The tests may have been generated from the same misunderstanding as the implementation, which makes them agreement rather than evidence. And the volume means "reviewed by a peer" degrades into "skimmed by a peer", which catches formatting and misses logic.

Stack Overflow's 2025 developer survey names the failure mode precisely. The single biggest frustration, reported by 66 percent of respondents, is AI output that is almost right but not quite [2]. Almost right is exactly the class of error a skim cannot catch, because it looks correct at the level a skim operates on. A definition of done that relies on a skim is a definition of done that lets almost right through.

What a usable definition of done contains

We write ours as five parts. Every unit of work gets all five before an agent touches anything.

The observable behaviour. What will be true from the outside when this is finished, phrased so that someone who did not attend the discussion could confirm it. "The export includes archived records" is observable. "Improve the export" is not.

A named check per criterion. For each behaviour, the specific automated check that proves it: a test name, an eval case, a script, a query against a fixture. This is the part teams skip, and skipping it is what turns a definition of done into a list of hopes. If you cannot name the check, you have written an opinion rather than a criterion.

What must stay the same. Existing behaviour that must be preserved, interfaces other systems depend on, performance that must not regress. Agents are enthusiastic refactorers and will improve things you did not ask them to touch. Writing down what must not be touched is cheaper than discovering it in review.

What a failure can affect. Which systems, tables, and users this change can affect if it is wrong. This is what decides whether the change can merge when the automatic gates pass or needs a person, and it is the input to the release decision covered in progressive delivery for AI-written changes.

The operational half. What gets instrumented, what the rollback is, and what one line goes in the runbook. A feature that works and cannot be observed is not finished, it is deferred. This is covered in more depth in what to instrument when nobody typed the code.

Who writes it, and when

Before generation, by the person who understands the requirement. Not the agent, and not the engineer afterwards from the diff.

This ordering matters more than it sounds. A definition of done written after the code exists is a description of the code. It will agree with whatever was built, including the parts that are wrong, because the person writing it is reading the implementation and thinking "yes, that seems right". Written first, it is a constraint. Written after, it is a rationalisation.

There is a practical benefit too. A well-formed definition of done is most of a prompt. When the criteria are specific and the checks are named, the instruction to the agent is almost complete already, and the agent has something to check its work against without a person involved. Anthropic's guidance describes the same step from the tool's point of view: give the model verification criteria in the prompt and ask it to run the check after implementing [1]. That is the same artefact, used twice.

The specification problem that looks like something else

When we audit a team whose agents keep building the wrong thing, the prompts are almost never the problem. The problem is that nobody wrote down what right meant, so the agent guessed it, and the guess filled the missing part with something plausible.

This shows up in a specific pattern: a change that satisfies every line of the ticket and still fails the intent behind it. The ticket said add a retry, so it retried, including on the request that charges a customer. The ticket said cache the lookup, so it cached, including the permission check. Nobody wrote down that money and permissions were different, because to the person who wrote the ticket that was too obvious to say.

The correction is not longer prompts. It is a habit of writing down what must stay the same, and a review of the criteria rather than of the code. Reviewing criteria is fast, it happens before the expensive work, and it catches the misunderstanding at the point where it costs nothing. Our post on what a good technical spec looks like covers the format we use, and the companion guide has a full page on writing specs an agent can verify.

How to tell yours is too weak

Four signs, all of them cheap to check.

The first is that your acceptance criteria contain adjectives. Fast, clean, robust, intuitive, secure. None of these can be evaluated by anything except a person with an opinion, which is exactly the resource you are short of.

The second is that the same reviewer keeps finding the same class of problem. If three reviews in a row caught a missing permission check, that is not a review success, it is a missing gate. Move it into the pipeline, as covered in the quality gates that should block a merge.

The third is that nobody can answer, in one sentence, what would prove the work is finished. If the answer takes a paragraph or starts with "well, it depends", the work is not ready to start and generating code now will just produce an expensive draft.

The fourth is rework. Track how often a merged change comes back for correction within two weeks. That number is the cost of your current definition of done, and it is usually the first number to move when a team tightens it.

What this looks like in practice

A definition of done for a change that adds a customer data export, written before any code exists, might name four behaviours, four checks, two things that must not change, a limit of one table and the reporting UI on what it can affect, one new metric, and a rollback done by switching a feature flag off. It fits on half a page. It takes ten minutes to write and it is the difference between an agent that produces something checkable and an agent that produces something plausible.

Here is one, written out, for a change that adds a customer data export.

The behaviours: an admin can export the accounts they have access to; the file contains archived accounts marked as archived; a request for more than fifty thousand rows produces a job rather than a synchronous response; a user with no export permission gets a refusal rather than an empty file. The checks: one integration test per behaviour, named in the ticket, plus an existing permission test that must still pass. What must not change: the current report endpoint, its response shape, and the permission model. What it can affect: one table read, the reporting UI, and the job queue. Operational half: a counter for exports started and completed, a rollback done by switching a flag off, and one runbook line saying the queued export jobs can be finished and stopped safely.

Nothing in that is clever. It is specific, and specific is what an agent can satisfy and a reviewer can check without reading every line.

The two-week rework check

If you want one number to tell you whether your definition of done is working, use this one: of the changes merged two weeks ago, how many have since been corrected. Not enhanced, corrected.

It is easy to collect, it is hard to argue with, and it responds quickly when the standard tightens, which makes it the fastest feedback available on this particular practice. Watch it for a month while you change one thing: writing the acceptance criteria before generation instead of after. If the figure moves, the criteria were the constraint. If it does not, the problem is somewhere else in the pipeline and you have ruled out an expensive guess cheaply.

That is the whole discipline. It is not new, it is not sophisticated, and almost nobody does it consistently, which is why the teams that do stand out. The next question is how big each of those units should be, which is decomposing work for coding agents, and after that the standard has to be enforced by something other than goodwill, which is the gates.

If your team is releasing fast and re-doing a lot of it, this is usually where the money is going. The main guide puts it in the context of the full delivery model, and it is the first thing we set up on an AI development engagement.