What should a non-technical approver ask before approving an AI-built project?

Nine questions covering verification, security boundaries, incident response, accountability after launch, handover, data handling, change cost, what the team was uncertain about, and what they would refuse to build. The pattern that matters is whether the answers name artefacts you could be shown or only describe processes anyone can claim.

How do I know if the answer to "how will you know it works" is good?

A weak answer describes a process: we write tests, we do code review, we have QA. A strong answer names artefacts: the list of guarantees the system must always hold, where those are written down, and the automated check that blocks a release when one fails. Ask to see the list, since a team that has one can produce it in a minute.

Why is "we review every line" not a sufficient answer anymore?

Because human attention degrades across a long diff and generated code is unusually plausible, so a reader finds nothing that looks wrong. Review remains valuable for design judgment. It is not a reliable mechanism for catching whether a boundary condition holds, and a vendor offering it as the primary safeguard is offering the weakest available one.

What should I ask about security specifically?

Ask which permission and access rules are enforced by an automated check rather than by convention, and ask to see one of those checks fail. Veracode's Spring 2026 research found generated code compiles over 95 percent of the time while being secure in about 55 percent of generations, so security is exactly where generated code is weakest and where a specific answer matters most.

How do I evaluate the answer about incident response?

Look for four specifics: who is contacted, what they can see without asking anyone, how long restoration is expected to take, and what happened the last time. A vendor with real operational experience answers in concrete terms because they have experienced it. A vague answer usually means the arrangement ends at launch.

What does a good answer about handover look like?

A named artefact set: running documentation, the check suite, environment setup that a new engineer can complete in a day, and the specific list of things only one person currently knows. The last item is the honest one, and a team that claims there is nothing on that list has either not looked or is not telling you.

Should I ask how much of the code is AI-generated?

It is less useful than it sounds, because the number does not predict quality in either direction. The better question is what verifies the code regardless of who or what wrote it, since a hand-written codebase with no checks carries the same risk as a generated one with no checks.

What is the most revealing question on the list?

What were you uncertain about. A vendor who answers that nothing was uncertain has either not engaged with the problem or is trying to control what you think. Real projects contain genuine unknowns, and a team that names theirs is telling you where to watch, which is worth more than a confident general statement.

How long should this conversation take?

About an hour, and it should happen before the contract rather than after the first milestone. The cost of asking is one meeting. The cost of not asking is discovered somewhere between the first incident and the handover that does not happen.

What if I get good answers to all nine and the project still fails?

Then it failed on product judgment rather than on execution, which is a different risk and one these questions do not cover. They establish whether the thing will work as specified. Whether it was worth specifying is a question for discovery, and it is the one that ends more projects than defects do.

The questions to ask before approving an AI build: Reveneau

Over the past year we have been part of this decision more times than we expected: the meeting where a founder, a CFO, or a head of operations has to approve a software build, knows the code will largely be generated, and is not an engineer.

The advice they usually get is unhelpful. Either "make sure they have good developers", which is unfalsifiable, or a technical checklist they cannot evaluate the answers to. What actually works is a short list of questions where a specific answer and a vague one sound obviously different, even if you cannot assess the technical content.

Nine questions. The pattern running through all of them is the same: a process answer is cheap and every vendor has one, while an artefact answer costs something to be able to give.

1. How will you know it works?

Ask this first and wait quietly for the answer.

A weak answer describes a process. We write tests. We do code review. We have a QA stage. Every vendor says this, including the ones whose last project was released broken, so it carries almost no information.

A strong answer names artefacts. Here is the list of things this system must always be true about. Here is where that list is kept. Here is the automated check that stops a release when one of them breaks. Here is what happened last week when one did.

Then ask to see the list. A team that has one produces it in under a minute. A team that does not will offer to put one together, which is the answer.

Why this is the first question: generated code fails differently from hand-written code. Veracode's Spring 2026 research tested over 150 models and found generated code compiled and ran correctly more than 95 percent of the time while being secure in roughly 55 percent of generations. It works and it is not safe, which means "we ran it and it worked" is no longer evidence of much.

2. Which rules are enforced by a machine rather than by convention?

This is question one made more specific, and it is where the security conversation actually happens.

Every codebase has rules: which users can see which data, who can move money, what gets logged, what never leaves the system. Some of those rules are enforced by an automated check that fails the build. The rest are enforced by everyone remembering.

Ask which is which. Then ask to be shown a check failing. Watching a permission assertion fail when someone deliberately breaks the rule tells you more in thirty seconds than an hour of architecture discussion, and it is a request no competent team finds strange.

The honest answer is always that some rules are conventions, because you cannot check everything. What you want to hear is that the expensive ones are checked and that the team knows which category each rule falls into. A vendor who claims everything is enforced automatically has not looked.

3. What happens when it breaks at two in the morning?

Not whether. When.

A serious answer covers four things: who is contacted, what that person can see without having to ask anyone, how long restoration is expected to take, and what happened the last time. That fourth part is the clearest sign. A team with real operational experience answers it immediately and concretely, usually with a story that does not make them look good.

A vague answer here almost always means the arrangement ends at launch, and the person who will actually be woken up is you.

4. Who is accountable after launch, and for how long?

Listen for whether the answer names a phase or an owner.

"Through the warranty period" is a phase. "We own this project through production and after release" is an owner. The difference matters because the harmful defects appear after the build, in the second month, in the case nobody thought about, under a load pattern that only exists once real customers arrive.

Get the answer in writing, and get it before the contract rather than after the first milestone. Our longer explanation of the distinction is in software development partner vs vendor.

5. What does handover look like if we stop working together?

Ask this even when you have no intention of stopping. The answer describes how easy the work is to understand, which matters from the start regardless of what happens later.

You want a named artefact set: running documentation, the check suite, an environment a new engineer can set up in a day, and an explicit list of things only one person currently knows. That last item is the honest one. Every project has such a list. A team that says theirs is empty either has not looked or is trying to control what you think.

The specific risk with generated code is a codebase that is large, uniform, plausible, and undocumented, which is very hard for a new team to take over safely. We wrote that up in handing over AI-written software.

6. What data does this touch, and where does it go?

Three sub-questions, and you do not need technical knowledge to evaluate the answers.

Which categories of personal or regulated data does the system hold? Which third-party services see that data, including any model provider? And when a customer asks for deletion, what actually gets deleted, including copies in backups, logs, analytics, and caches.

The deletion question is the one that surprises people, because "we delete the record" is often true and insufficient. If the system is in a regulated industry the risk rises, and translating regulation into engineering requirements covers how that conversation should go.

7. What will it cost to change this in a year?

Approvers routinely ask what the build costs and rarely ask what the second year costs. The second year is where most of the money is.

The answer you want describes what makes change cheap or expensive here: whether the guarantees are written down, whether a new engineer can be productive quickly, whether there are parts of the system the team already avoids. That last one exists on every project older than a few months and a candid vendor will name it.

The specific failure mode with a fast generated build is a system that works and that nobody can safely modify, because there is no way to prove a change did not break something. That is how a project that came in under budget becomes expensive, and it is the mechanism we described in the real cost of shipping unverified code.

8. What were you uncertain about?

The most revealing question on the list, and the shortest.

Real projects contain genuine unknowns: a third-party integration whose behaviour is undocumented, a performance requirement nobody has tested at full load, a business rule with two plausible readings. A team that names theirs is telling you where to watch, which is worth more to you than a confident general statement about the whole scope.

A team that says nothing was uncertain has either not engaged with the problem or has decided you are the kind of buyer who wants to be reassured. Both are bad, and the second is worse.

There is a research reason to distrust confidence specifically here. In a randomised controlled trial published in July 2025, METR gave 16 experienced open-source developers 246 real tasks in codebases they knew well. With AI tools they took 19 percent longer, and afterwards estimated AI had made them 20 percent faster. METR labels the finding historical, since tooling has changed since then, and that caveat is fair. What survives is that experienced professionals could not tell which direction their own performance had gone. So "the team feels good about it" is not evidence, from them or from you.

9. What would you refuse to build?

A team with judgment has a boundary. Maybe they will not build a feature they think is legally exposed, or a scope they believe cannot be released in the timeline, or a system whose success criteria nobody has defined.

A team with no boundary will accept your worst idea at full price. That is not a supplier, that is an order-taker, and the difference shows up when your requirements are wrong, which some of them are.

How to read the whole conversation

Two patterns matter more than any individual answer.

Specificity under follow-up. Anyone can give a good first answer. Ask "how" once more and watch what happens. Real practice has detail underneath it, and the detail arrives without effort. A rehearsed answer gets less detailed each time you ask again.

Willingness to say "we do not know yet". A vendor who can say that about the right things is telling you they understand which parts are genuinely uncertain. A vendor who is certain about everything, including the parts that depend on information nobody has yet, is only trying to sell.

You do not need to evaluate the code. You need to evaluate whether anything other than confidence supports it, and these nine questions are enough to establish that in about an hour.

Thanks to the operators and finance leads who tested earlier versions of this list on real vendor calls and told us which questions produced nothing. Three that used to be on it are gone because they only ever generated agreement.

The answer you should worry about is not the one you failed to understand. It is the one that sounded good and named nothing.

Related guide: How to evaluate an AI development partner.

Sources

Veracode, Spring 2026 GenAI Code Security update: across more than 150 models, generated code compiled and ran correctly in over 95 percent of cases while being secure in roughly 55 percent.
METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (July 2025): 16 developers, 246 tasks, 19 percent longer with AI tools against a self-estimated 20 percent speed-up. METR labels the result historical.

The questions to ask before approving an AI build