Security of AI-written code: what to check in diligence

Security of AI-written code in diligence is two checks. The first is a scan of the code on the current commit, because the measured failure rate of generated code is high enough that reading it is the wrong tool. The second is a review of the risks that exist only because a model is in the product, which no code scanner finds and which the OWASP list for LLM applications sets out.

Both sit on top of the general review in assessing security and compliance, which still runs in full. This page is about what the AI adds, and it keeps the argument short because the AI-generated code guide already makes it at length.

Why scan rather than read?

Scan rather than read because the failure rate of generated code is measured and high, and because generated code is consistent by default, which makes it read as competent whether or not it is secure. Reveneau's diligence protocol runs a static and dependency scan on every AI-written codebase before anyone opens a file, and reads only what the scan and the eval suite point to.

The figure that justifies this comes from Veracode's 2025 GenAI Code Security Report, published 30 July 2025, which tested over 100 large language models on coding tasks in Java, Python, C# and JavaScript and found that 45 percent of the generated samples failed security tests and introduced an OWASP Top 10 vulnerability. By language, Java failed 72 percent of the time, C# 45, JavaScript 43 and Python 38. For cross-site scripting specifically, the models failed to defend in 86 percent of relevant samples. Veracode also reported that security performance stayed flat regardless of model size or training sophistication [1]. Is AI-generated code secure goes through those numbers and what they mean for an engineering team; for a diligence reviewer the reading is simpler. Until scanned, assume the codebase carries that rate.

The second reason is the shape of the errors. Stack Overflow's 2025 survey found 66 percent of developers name output that is "almost right" as their main frustration with AI tools [2]. An almost-right input validator, an almost-right authentication check, an almost-right query builder: each passes the happy path and the eval suite, and each is what the scanner exists to find.

What are the model-specific risks?

The model-specific risks are the ones that exist because untrusted text reaches a model that can act on it, and the OWASP Top 10 for LLM Applications 2025 is the reference list [3]. A conventional scan sees none of them. The table gives each entry and the diligence question for it.

OWASP LLM 2025 entry	Diligence question
LLM01 Prompt Injection	What untrusted text reaches the model, and what can the model do as a result? Ask for the injection test cases in the eval suite.
LLM02 Sensitive Information Disclosure	What customer data is in the context window, and can one customer's data reach another's session?
LLM03 Supply Chain	Which models, versions, libraries and third-party prompts are in the product, and how are they pinned?
LLM04 Data and Model Poisoning	If the product fine-tunes or retrieves from customer data, who can write to that data?
LLM05 Improper Output Handling	Is model output rendered, executed or written to a database without validation?
LLM06 Excessive Agency	What tools can the model call, with what permissions, and what needs a human approval?
LLM07 System Prompt Leakage	Does the system prompt contain anything that matters if it leaks, such as a credential or a rule a user could exploit?
LLM08 Vector and Embedding Weaknesses	If retrieval is used, how is access controlled at the document level?
LLM09 Misinformation	Where does the product state a model output as fact, and what checks it?
LLM10 Unbounded Consumption	What stops one user, or one bug, from running an unbounded number of model calls on the company's bill?

Prompt injection holds the first position for the second consecutive edition of the list, and it is the one to ask about first, because a product that reads documents, emails or web pages and then takes actions is exposed to it by design. The test from how to verify an AI claim can include an injection case: an input that contains an instruction, to see whether the product follows it.

LLM10 is also a margin question. A product with no cap on model calls per user has a cost of goods that a single script can change, and that belongs in rebuilding inference gross margin from invoices as well as here.

How does customer data reach the model provider?

Customer data reaches the model provider in every request that includes it in the context, and the diligence question is which provider, under which terms, with what retention. Anthropic's commercial terms state that the customer retains all rights to its inputs and owns its outputs, and that "Anthropic may not train models on Customer Content from Services" [4]. The equivalent terms for every other provider the product calls, and the account type in force, are what the target should be able to show. A product that sends customer data to a consumer-grade endpoint has a different answer from one on a commercial plan, and the difference is a disclosure to the customers whose data went through.

Retention is the third part of the answer. Some commercial plans retain inputs for a fixed period for abuse monitoring, some offer zero-retention arrangements, and consumer products keep conversations on their own terms. The target should know which applies to each provider and be able to show the setting rather than describe it.

Ask the same question of the tools used to write the code, since the codebase itself may have contained customer data or credentials when it was sent as context. Code provenance and licence exposure covers that side.

How do you run the security check in diligence?

Run the security check in diligence in five steps, and do the first one before the management meeting.

1. Ask for the last scan. A static analysis report and a dependency report from the last quarter, with each finding triaged and the fixes merged. If none exists, ask for permission to run one; if permission is refused, that is the finding.
2. Run a scan on the current commit. Your own tools, on a clean checkout. Compare with the report you were given. A large gap between the two means the company's scan is stale or filtered.
3. Walk the ten OWASP LLM entries with the team. One question each from the table above. Write the answer and the evidence. "We have injection cases in the eval suite" is an answer; "the model is safe" is not.
4. Test one injection case live. In the claim-test session, supply an input that contains an instruction the product should not follow. Watch what happens.
5. Trace the data flow. From the customer's input to each provider and back, with the terms and retention for each hop.

What does a good answer look like?

A good answer is a scan from the last quarter, with findings triaged and the fixes merged; a written account of each of the ten model-specific risks with the control for each; injection cases in the eval suite; a data-flow diagram naming every provider and the terms in force; and a cap on model calls per user in the code, with the number. A team with all five sends them in a day.

A weak answer is any of: "we will run a scan", "the model handles that", a system prompt as the only defence against injection, a provider named without the account type, or no answer to the question of what one user can cost the company in a day.

What goes in the report?

The report separates the two checks, because they have different fixes. Scanner findings are ordinary engineering debt: a count by severity, an estimate of the work to clear the high and critical items, and a note on whether the eval suite would catch a regression. Model-specific findings are design questions: an injection exposure in a product that takes actions is a change to the architecture, and a missing cap on consumption is a line in the margin model. Security review before launch is the engineer's version of the same list. Diligence on an AI-written codebase covers the other artefacts the scan sits beside, and the AI startup due diligence guide puts the security read in its place in the whole review.