The verification gap: why AI made writing code cheap and checking it expensive

For most of software's history, the slow step was typing. Everything about how teams work, from sprint planning to code review, was designed around that assumption. The assumption stopped being true and the practices have not changed to match.

What the research actually says

Start with throughput and stability. Google's DORA program surveyed nearly 5,000 technology professionals for its 2025 report on AI-assisted software development. Ninety percent use AI at work. More than 80 percent believe it makes them more productive. And the report found a positive relationship between AI adoption and delivery throughput alongside a negative relationship with delivery stability [1]. Read that twice. More changes are reaching production, and a higher share of them are going wrong.

DORA's own description is that AI increases the effect of whatever practices the organisation already has. A team with strong checks gets faster. A team without them gets faster at producing rework.

Then take individual judgment. In July 2025, METR published a randomised controlled trial with 16 experienced open-source developers working 246 real tasks in repositories they knew well, averaging around five years of familiarity. The result was that AI-assisted work took 19 percent longer, and the participants estimated afterwards that it had been 20 percent faster [2]. METR labels this result historical, since the tools available in early 2025 are not the tools available now, and that caveat is fair. The lasting finding is that skilled professionals were wrong by 39 points about their own performance, and the error made the tool look better than it was. If your intuition cannot measure your own speed, it should not be the thing that certifies correctness.

Now take the code itself. Veracode's spring 2026 update tested more than 150 language models across 80 coding tasks in four languages. Only 55 percent of generations produced secure code. Java came out at 29 percent. Cross-site scripting was handled correctly in roughly 13 to 15 percent of relevant cases. Meanwhile syntax correctness exceeded 95 percent, and the security number has stayed the same for about two years while the syntax number rose [3]. Their conclusion is the sentence to remember: models have become excellent at writing code that compiles, and have failed at writing code that is safe.

Finally, take what developers themselves report. Stack Overflow's 2025 survey found 84 percent using or planning to use AI tools, 46 percent actively distrusting the accuracy of the output against 33 percent who trust it, and the top frustration at 66 percent being AI solutions that are almost right, but not quite [4].

Almost right is the hard case

Every one of those findings points to the same kind of failure, and it is a kind human review handles badly.

An obviously wrong change is easy. It does not compile, it fails on the first click, someone catches it in a minute. A subtly wrong change is the expensive one: correct structure, plausible variable names, a reasonable-looking guard clause that checks the wrong boundary. It reads as competent because it is competent everywhere except the one place that matters. A reviewer skimming 600 lines on a Friday afternoon will approve it, and a reviewer skimming 6,000 lines will approve it faster.

This is a statement about what humans are good at. People are excellent at asking whether a design will cause problems in a year. They are poor at mentally executing code against 40 unusual cases, and they get worse as the diff gets longer. A machine is the reverse. The gap grows when volume rises and the only checking is done by people, whose attention is limited.

Why the gap keeps getting worse

Three causes make it worse together.

Volume. Generation cost per line keeps falling, so the natural size of a change keeps rising. Review capacity is fixed by headcount and hours.

Uniformity. Human mistakes are idiosyncratic and scattered. Model mistakes cluster, because models reproduce the patterns in their training data. When one flawed pattern appears, it tends to appear in many places at once, which is why the security numbers stay the same as capability improves.

Confidence. Generated code shows no doubt. Hand-written code often signals doubt through a comment, an unfinished part, a variable named tmp. Generated code is uniformly tidy, and tidy code looks careful even when nobody took care.

What fixes the gap

Only one thing scales with volume, and that is automated verification. Checks read as fast as the machine can run them, do not get tired, do not skim, and do not care whether the diff is 60 lines or 6,000.

That is the argument for eval-driven development, and it is a practical argument rather than a moral one. If your team is generating more code, you need proportionally more machine-checkable assertions about what that code must do. More meetings, a stricter review policy, or a rule that all pull requests must be under 400 lines will not solve it. What works is more checks, running on every change, with the power to block a merge.

The human review does not go away. It gets better, because people stop trying to do the checking work that a computer does better. That division is covered in evals vs tests vs code review, and the wider trust question, including the human review discipline itself, is in the companion guide on taking AI-generated code to production. Evals prove a change is correct; the process around them, from sizing the work to releasing it and measuring what arrived, is the AI-native delivery guide.

The honest version for a buyer

If you are paying someone else to build software with AI, this is the thing to ask about. Asking whether they use AI tells you little, since everyone does, and a supplier who avoids it is slower with no gain in safety. Ask what the automated check is, whether it runs on every change, who wrote it, and what happens when it fails on a Friday. A supplier who can answer that specifically has dealt with the verification gap. A supplier who answers by describing their senior review culture has not, however senior the people are.