The business case for evals

Someone has to approve the time, and that person is usually being asked to slow down a team that currently feels fast. So it is worth making the argument in terms of risk and cost rather than engineering taste.

What you are actually buying

You are buying the ability to keep releasing quickly without the failure rate rising with the volume, which is the specific thing that goes wrong when a team adopts AI coding tools.

That failure pattern is documented. Google's DORA program surveyed nearly 5,000 technology professionals in 2025 and found AI adoption has a positive relationship with delivery throughput and a negative relationship with delivery stability [1]. Their view is that AI increases the effect of whatever practices the organisation already has. If the checks are strong, you get faster delivery. If they are weak, you get faster rework.

The second thing you are buying is scarce senior attention. Right now that attention is being spent reading diffs to work out whether they function, which a computer does better. Move that work to the machine and the same people spend their time on design and product judgment instead. That is a real reallocation of your most expensive resource, and it is the part that rarely appears in a business case.

What it costs

Less than people expect, if it is not run as a project. One critical flow is typically a day or two: write the invariants as sentences, turn them into checks, connect them to the pipeline as a blocking check. Three flows is a couple of weeks. After that the suite grows inside normal work, because the rule is that you cover the area you are about to change and you turn every incident into a permanent check.

The expensive version is the one to avoid: a testing initiative with a coverage target, running for three months, producing a large number of tests around the easiest code. That is the version that gets cancelled and then quoted as evidence that testing does not pay. Adding evals to an existing codebase is the staged alternative.

Where the return shows up

Four places, in order of size.

Rework avoided. A defect caught by a check in a pull request costs minutes. The same defect caught by a customer costs a support conversation, a hotfix, a release, a retrospective, and the trust of whoever reported it. The difference in cost between those two is the whole argument, and it is why we do not need an invented percentage to make the case.

Incidents avoided. Most of the failures that matter are ordinary: a permission boundary that stopped working, a migration that dropped a column, an integration whose payload changed structure without notice. All three are cheap to assert and expensive to discover in production.

Review time recovered. Senior engineers reading verified diffs go much faster than senior engineers verifying unverified ones, and they enjoy it more, which matters for retention.

Change confidence. The least visible one, and its value grows over time. Teams with trustworthy checks refactor, upgrade dependencies, and let agents restructure code, because they can prove nothing changed. Teams without them build up code nobody is willing to change, and that is what technical debt mostly is.

What we will not tell you

We will not give you a percentage for how much our own defect rate fell after we adopted this, because we have not run the controlled experiment that would justify the number. A consultancy that sells verification should not be careless with its own claims, and any figure we published would be exactly the kind of confident, unverifiable statement this practice exists to prevent.

What we will say is what changed in kind. The problems we find now are design disagreements found in a pull request, rather than unexpected behaviour reported by a customer. That is a qualitative claim and it is honest, which is the trade we would rather make.

If you want the outside numbers, use the ones in this guide: DORA on throughput and stability [1], Veracode on the unchanged security pass rate of generated code [2], and Stack Overflow on almost-right output being the top developer complaint [3]. Those are measured by people with nothing to gain from your decision.

Questions to ask a development partner

If you are buying software rather than building it, this is where the practice becomes a purchasing criterion. Everyone uses AI now, so asking whether they do tells you nothing. Ask instead:

What automated check must pass before a merge in our repository, and can we see it? Who writes those checks, and are they written before or after the implementation? What happens when a check fails on a Friday afternoon? How do you stop an agent from weakening a check to get a passing result? Which of our flows will have checks in the first two weeks?

Specific answers mean the discipline is real. A pitch about senior review culture, with no automated check behind it, means the verification gap is being handled only by human attention, and human attention does not scale with generated code volume however senior it is.

Our own answer, so you can hold us to the same standard: all of our code is generated, every change has to pass a large eval suite before it reaches your branch, and those checks are written from the specification rather than from the implementation. The companion question set for choosing an AI development partner is in how to evaluate an AI development partner, and if you want to talk about a specific codebase, get in touch.

Translating this into numbers a finance or operations leader will use

The four returns above are real but qualitative, and the person approving budget often needs at least a rough structure for comparing them against the cost of the work, even without a published percentage. Here is how to build that structure honestly, without inventing a figure this guide has already promised not to invent.

Take your own change failure rate and your own defect escape count from the last quarter, both of which your existing tools can usually produce. Multiply the number of defects that reached customers by whatever your team already knows a support ticket and hotfix cycle costs in engineer hours, since most teams already track this informally even if nobody has written it down as a metric. That gives a real cost of the current state, built from your own data rather than an industry average that may not describe your product. The cost of the first three weeks of eval work is comparatively easy to estimate, because it is bounded: a day or two per flow, a small number of flows. Comparing a bounded, known cost against an ongoing, currently unmeasured cost is usually enough to make the case, without needing a published return on investment figure that nobody could defend under scrutiny anyway.

Why the cost argument gets easier once volume is high

There is a detail worth stating plainly for a buyer weighing whether to fund this now or later: the case for eval-driven development gets stronger, not weaker, as the volume of AI-generated code in a codebase rises. A team generating a small fraction of its code with AI tools can plausibly rely on senior review catching most problems, because the volume is still within what careful human attention can cover. That same team, six months later, generating most of its code with an agent, faces the mismatch described in agent debt: review capacity did not grow, generation volume did, and the argument for automated checks that scale with volume rather than with headcount grows stronger with it. The practical implication for someone approving budget is that the cost of building the suite now is close to fixed, while the cost of delaying it rises with every quarter that AI-generated volume keeps growing. Waiting for a clearer business case is, in effect, waiting for the problem to get more expensive before addressing it.

What this looks like in a due diligence conversation specifically

The diligence angle deserves a concrete example, because "an asset in diligence" is easy to state and easy to underrate. A buyer evaluating a company's codebase typically has days, not months, to form a view of what is reliable and what is fragile. A codebase with a real eval suite gives that buyer something to inspect directly: a list of checks, what each one asserts, and a pass or fail history. A codebase with no suite, or with a suite of the kind described in when evals give false confidence, gives the buyer only the seller's word and whatever spot-checking a short engagement allows. The difference shows up in how a diligence report gets written: one version can state specifically which requirements are proven to hold, and the other has to hedge with language about apparent code quality and general impressions, which is precisely the kind of unverifiable claim serious buyers already discount.

The failure mode of approving the budget and then quietly deprioritising it

One more pattern worth naming for whoever holds the budget: teams that get eval work approved sometimes fail to protect the first three weeks from the ordinary pressure of a sprint, and the work slips a little every cycle until it never actually starts. The fix is not more budget. It is treating the first three flows as a fixed, short commitment with its own deadline, the same way a security patch or a compliance deadline would be treated, rather than as background work that competes with every feature request for the same engineer's attention. A commitment that can be deprioritised indefinitely is not a commitment, and the staged plan in adding evals to an existing codebase only works if the first three weeks actually happen on schedule.