What good code review looks like when nobody wrote the code

To secure AI-generated code before it goes to production, check three things on every change: that every package it added exists on the public registry and passes a dependency audit, that a person read every file in the change against the specification, and that no secret sits in a file the tool can read. Reveneau writes all of its code with AI, and every change must pass an eval suite written from the specification before it ships, so the human read is where the specification itself gets checked.
Ask ten engineers what code review is for and you will get answers that disagree. Some think it is a checkpoint that blocks bad changes. Some think it is where you enforce your opinions about how code should look. Some think it is a chore you do at the end of the day so a teammate can continue. Those disagreements did little harm when a person wrote the code, because the author was doing most of the quality work anyway and review was a useful second check.
When the implementation is generated, there is no first check. The reviewer is the only check, and a process designed as a backup does not become the main control just because the first check disappeared.
What review is for now
Two of the three traditional jobs remain, and one is gone.
Catching problems early still matters, for the same reason it always did. The cost of a defect grows the longer it stays in the code, a point NIST has documented in its work on the economic cost of software errors, and review remains the cheapest place in the process to catch one.
Building shared understanding matters more. When a person writes a change, at least one human understands it because they wrote it. When a model writes it, that is no longer true. If the reviewer does not understand the change, then nobody does, and you have added something to your system that no human can explain. Six months later, when it misbehaves, the understanding does not exist to be recovered. It was never created.
Helping the author improve is gone. You cannot teach a model to be better on the next pull request. What replaces it: the feedback now goes into the specification. When a reviewer keeps finding the same type of gap, the fix is a sentence in the spec that should have been there, more than a comment on the change. Reviewing generated code teaches you about your spec.
Generated code fails differently
This is the part teams underestimate, and it is the reason a reviewer cannot simply keep doing what they were doing.
Human code fails in human ways. Someone forgets a case, misreads a requirement, gets an index off by one, or runs out of attention at 6pm. Experienced reviewers are trained to notice exactly these, and that skill is mostly unconscious. It is pattern recognition built over years of watching the mistakes people typically make.
Generated code does not make those mistakes, and that trained skill does not help with it. Its characteristic failures are different:
It is plausible and wrong in small ways. The code reads well, follows the conventions, and solves a problem close to yours. Close is the dangerous part. A reviewer skimming for obvious errors finds none, because there are none of that kind.
It fills gaps silently. Anywhere the specification left a decision open, something was still decided, and nothing in the change marks where. There is no comment saying "unclear from the spec, assumed the first case wins." The assumption is just there, in the logic, looking exactly like a requirement.
It does more than was asked. Generated code often handles cases nobody asked about, sometimes elaborately. This looks like thoroughness and is frequently the opposite: it is extra code you did not want, behaviour you did not specify, and code you now have to maintain.
It is confident about the main path and incomplete on everything else. This is the most reliable pattern of all. Normal use is usually handled correctly. The timeouts, the retries, the partial failures, the concurrent case: that is where the gaps are, consistently.
The practical consequence is that the reviewer's question has to change. Asking "is this correct?" does not help, because it will look correct. The question is "is this the thing we specified?", checked case by case against the document it was generated from, and most carefully on the unusual paths.
Small changes are now required
If we could change one thing about how a team reviews generated code, it would be size, and the reason is now the opposite of what it used to be.
Small pull requests were always better. What used to keep them small, though, was partly effort: writing a thousand lines took a person a week, so it did not happen casually. AI removed that limit entirely. A thousand-line change now costs the machine nothing to produce, which means nothing is stopping it from arriving except a rule you enforce.
Meanwhile the cost on the other side went up. A reviewer can read a twelve-line human change quickly, because they can rely on the author's judgment for the parts they skim. On a generated change there is no judgment to rely on. Every line is unverified. Reviewing it properly is slower per line.
So the two costs moved in opposite directions: cheaper to produce, more expensive to check. A team that does not deliberately limit change size will slowly start approving changes without reading them, and a large generated change that gets approved without real reading is the worst result in this whole process. It has the appearance of a control and none of the substance. The DORA research has long found that small batches lower failure rates, and that finding did not stop being true when the author changed.
Automate more, and know what automation cannot do
Everything mechanical should run before a human sees the change: formatting, style, common bug patterns, the full test suite, dependency and licence scanning. This was good practice before and it is close to mandatory now, because the reviewer's attention is the scarcest resource in the process and every minute of it spent on something a tool could check is a minute not spent on behaviour.
Using a model to review generated code belongs in this category. It is a useful filter and it catches real things. It cannot be the last step, because it shares failure modes with the thing it is checking, it is subject to the same confident-and-wrong pattern, and it cannot be accountable for the outcome. Use it to reduce what reaches the human reviewer.
Reveneau takes responsibility for the whole project, through production and after release. Nobody can escalate to a model at two in the morning, and a system whose only author was a model has nobody to ask, so a person reads every change against the specification before the eval suite and the release carry it forward.
How do I secure AI-generated code before it goes to production?
Three checks before the change merges, and one rule about what the tool can reach. All four come from OWASP's Secure Coding with AI cheat sheet and the OpenSSF guide for AI code assistant instructions (dated 1 August 2025), both read on 12 November 2026.
The first check is the packages. OWASP says "AI coding assistants frequently suggest package names that do not exist on public registries" and that "Attackers monitor these hallucinated names and register malicious packages with matching names". Its rule is to "Verify every AI-suggested package exists on the public registry before installing", and to run dependency auditing tools (it lists npm audit, pip audit, govulncheck and cargo audit) "on every AI-generated dependency list before merging". The OpenSSF guide cites one study in which 19.7 percent of the packages an assistant proposed did not exist. The figures from the 2025 USENIX Security paper on made-up package names are in how to do a security review before launch.
The second check is the whole diff. OWASP calls review anchoring "the most common review failure mode in agentic coding": the reviewer reads the file the task named and misses the other files the agent changed. Its rule is to "Review every file in an agent-generated PR individually" and never approve "based on the summary or description without reviewing the full diff". Tests get the same read. The cheat sheet says agents "make CI green by deleting failing tests, weakening assertions, mocking the unit under test instead of fixing the code, or asserting the buggy behavior", so a deleted test or an assertion that got weaker needs a reason a person has checked.
The third check is secrets. OWASP says to keep every secret in an environment variable, a vault service or an encrypted secret store, and never in a file inside the project tree where an AI tool can read it, because a .gitignore entry only affects git and AI tools read the file system directly. The OpenSSF guide's sample instruction for the assistant says "Never include API keys, passwords, or secrets in code output, and use environment variables or secure vault references instead."
The rule about reach: OWASP says do not "Run AI coding agents with your full developer credentials, SSH keys, and cloud access tokens without sandboxing", and do not let an agent reach "production credentials, deployment keys, or org-level secrets from the development environment".
The OpenSSF guide puts the whole thing in one sentence: "AI-generated code isn't a shortcut around engineering processes such as code reviews, testing, static analysis, documentation, and version control discipline." At Reveneau those processes run on every change: AI writes all of our code, the mechanical checks above run first, then every change must pass an eval suite written from the specification, and a person reads the change against that specification before it ships. When a check finds something, how to handle a security finding in generated code is the next step.
What this looks like day to day
Changes arrive small, because a rule keeps them small. The automated checks already pass, so the routine checks are done. A reviewer picks one up and reads it against the specification, checking the error paths first because that is where the gaps are. They are direct about the code, because there is no author to discourage. When they find a gap that came from an ambiguous spec, the comment goes on the change and the fix goes in the spec, so the same gap does not arrive again next week. Then the eval suite runs against that specification, and the change ships only when it passes.
Good review was always about two things: problems caught while they are small, and more than one person understanding the system. Both of those got harder to achieve and more valuable to have. The practice is now where nearly all of the engineering judgment happens, which is the same reason we argue for judging engineers on outcomes rather than volume in measure engineers by outcomes not output.
Sources
- NIST, National Institute of Standards and Technology (research on the economic cost of finding and fixing software defects later in the lifecycle): https://www.nist.gov/
- DORA, DevOps Research and Assessment (research on batch size, deployment frequency, and change failure rate): https://dora.dev/research/
- OWASP Cheat Sheet Series, Secure Coding with AI Cheat Sheet (read 12 November 2026): https://cheatsheetseries.owasp.org/cheatsheets/Secure_Coding_with_AI_Cheat_Sheet.html
- OpenSSF Best Practices and AI/ML Working Groups, Security-Focused Guide for AI Code Assistant Instructions, 1 August 2025 (read 12 November 2026): https://best.openssf.org/Security-Focused-Guide-for-AI-Code-Assistant-Instructions


