AI security risks for a large company: prompt injection, too much autonomy, and data

Among Canadian businesses with 100 or more employees, 30.0% name cybersecurity or privacy concerns as a barrier to using artificial intelligence (AI), according to Statistics Canada's survey of the second quarter of 2026 [13]. At the same time, AI is a security tool: 47.51% of large EU enterprises that use AI use it for the security of information and communication technology (ICT), according to Eurostat [12].

This page covers the security risks that AI systems bring into a company: the OWASP list of risks for applications built on large language models, prompt injection and excessive agency in detail, the data risks, and the controls to put in place. It is part of the guide on enterprise AI adoption.

A large language model (LLM) is an AI model trained on large amounts of text, which writes text in reply to a request. Chat assistants and document search tools are often built on one.

The OWASP Top 10 for LLM Applications

The OWASP Gen AI Security Project publishes a list of the ten main security risks for applications built on LLMs. The 2025 edition was still the current list on its page on 26 October 2026 [1].

Number	Risk	What it means in plain words
LLM01	Prompt Injection	Text in the input changes what the model does [2]
LLM02	Sensitive Information Disclosure	The system reveals data it should keep private
LLM03	Supply Chain	A model, dataset or tool from outside is weak or has been tampered with
LLM04	Data and Model Poisoning	Someone corrupts the data a model learns from
LLM05	Improper Output Handling	Other software trusts the model's output without checking it
LLM06	Excessive Agency	The system can take damaging actions it did not need to be able to take [3]
LLM07	System Prompt Leakage	The hidden instructions given to the model are exposed
LLM08	Vector and Embedding Weaknesses	Weaknesses in how the system stores and searches documents for the model
LLM09	Misinformation	The model gives false information that people act on
LLM10	Unbounded Consumption	Use with no limits runs up cost or blocks the service

The names come from the OWASP list [1]. The plain meanings for LLM02 to LLM05 and LLM07 to LLM10 are this page's short summaries of the names.

Prompt injection: a risk with no known complete fix

OWASP defines it: "A Prompt Injection Vulnerability occurs when user prompts alter the LLM's behavior or output in unintended ways" [2]. A prompt is the text given to the model. The instructions in the prompt can be hidden from a person and still read by the model.

In Reveneau's view, the harder case for a large company is indirect prompt injection. The US National Institute of Standards and Technology (NIST) explains it in its report on attacks against AI, NIST AI 100-2e2025, published in March 2025 and still the newest edition on NIST's site on 26 October 2026 [5]. A third party plants instructions in data the model will read later, such as a web page, an email or a shared document. When the model reads that data, it may follow the planted instructions. NIST notes that "in many cases, it is the primary user of the model who is harmed by the compromise of the integrity, availability, or privacy of the GenAI system" [4]. An AI assistant that reads a customer's email can be turned against the employee who uses it.

OWASP says it is not clear that any method fully prevents prompt injection [2]. NIST says: "Because current mitigations do not offer full protection against all attacker techniques, application designers may design systems" on the assumption that prompt injection attacks are possible whenever a model is exposed to untrusted input [4].

Much of the defence therefore sits in the system around the model. OWASP's mitigations include giving the application the least privilege it needs, and: "Implement human-in-the-loop controls for privileged operations to prevent unauthorized actions" [2]. Human-in-the-loop means a person must approve an action before it happens. NIST describes another design defence: "allowing models to interact with potentially untrustworthy data sources only through well-defined interfaces" [4].

Excessive agency: AI that can do too much

An AI agent is an AI system that takes actions on its own, such as sending emails, changing records or calling other software, to reach a goal. OWASP defines the risk: "Excessive Agency is the vulnerability that enables damaging actions to be performed in response to unexpected, ambiguous or manipulated outputs from an LLM" [3]. The root cause is "typically one or more of: excessive functionality; excessive permissions; excessive autonomy" [3]. That means the agent has tools it does not need, permissions wider than its task, or freedom to act without a check.

The fixes follow from the causes. OWASP's first is "Minimize extensions", which means offering an agent only the extensions it needs, and its fourth is "Minimize extension permissions" [3]. The two risks add up: an injected instruction can do as much damage as the agent's permissions allow.

The Model Context Protocol (MCP), an open standard for connecting AI applications to tools and data, sets the same rule for the software that runs the model. Its specification says tool descriptions "should be considered untrusted, unless obtained from a trusted server", and: "Hosts must obtain explicit user consent before invoking any tool" [11]. The page connecting AI to older systems covers MCP in detail, and the guide on AI agents in production covers agents.

Data risks

In May 2025 the AI Security Center of the US National Security Agency (NSA), the Cybersecurity and Infrastructure Security Agency (CISA), the Federal Bureau of Investigation (FBI) and partners published an information sheet on AI data security [10]. It names "three significant areas of data security risks in AI systems: data supply chain, maliciously modified ("poisoned") data, and data drift" [9].

• The data supply chain is every source the training and input data come from. A problem at any one source can corrupt the result.
• Poisoned data is data changed on purpose to make a model behave badly. The sheet says that for some large public datasets, "In some cases, effective techniques can cost as little as $60 USD" [9], citing 2023 research.
• Data drift is a change over time in the data a running system receives, which can make a model that worked well start to fail.

The sheet recommends "data encryption, digital signatures, data provenance tracking, secure storage, and trust infrastructure" [9]. Data provenance means a record of where each piece of data came from and how it changed.

NIST's Generative AI Profile, NIST AI 600-1, adds a related risk it calls value chain and component integration: "Non-transparent or untraceable integration of upstream third-party components" [6]. Parts that a company cannot trace are parts whose risk it cannot judge. The page getting company data ready for AI covers data quality and drift in detail.

The secure development guidelines

The UK National Cyber Security Centre (NCSC) and the US Cybersecurity and Infrastructure Security Agency (CISA), with partner agencies from other countries, published "Guidelines for secure AI system development" on 27 November 2023 [7]. The partners include the NSA, the FBI and national cyber security agencies of Canada, Germany and France [8]. The guidelines split the work into four areas: "secure design, secure development, secure deployment, and secure operation and maintenance" [7]. The NCSC asks for security at every stage of the system's life cycle [7].

The guidelines are written for more than model builders. They are "aimed primarily at providers of AI systems who are using models hosted by an organisation, or are using external application programming interfaces (APIs)" [7]. An application programming interface (API) is a way for one program to call another. A large company that builds a tool on a model from OpenAI, Anthropic or Google through an API is in the main audience.

What the EU AI Act requires

For high-risk AI systems, Article 15 of the EU AI Act requires that they reach "an appropriate level of accuracy, robustness, and cybersecurity" and perform consistently in those respects through their lifecycle, and says: "High-risk AI systems shall be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities" [14]. It is a duty of the provider, the firm that develops the system and puts it on the market. For Annex III high-risk systems it applies from 2 December 2027, under Regulation (EU) 2026/1744 [15]. The 2026 amendment also says that high-risk systems that meet the conditions of the Cyber Resilience Act, the EU law on cybersecurity for products with digital elements, are "deemed to comply with the cybersecurity requirements set out in Article 15" [15]. The page AI governance for a large company covers the other high-risk duties.

Controls for a large company

The table below is Reveneau's list of controls. Each line points to the source that asks for it or the risk it answers.

Control	What it means	Source or risk
Fewest permissions	Each AI system gets only the tools and access its task needs	OWASP LLM06 [3]; OWASP LLM01 [2]
A person approves risky actions	Changing records, sending money or messages outside the company needs a person's approval	OWASP LLM01 [2]; MCP specification [11]
Outside text is treated as untrusted	Emails, web pages and uploaded files are handled as possible attacks	NIST AI 100-2e2025 [4]
Narrow interfaces	The model reaches other systems only through defined, limited calls	NIST AI 100-2e2025 [4]
Data provenance and signatures	A record of where data came from; trusted versions signed	AI Data Security sheet [9]
Usage limits	Caps on requests and spending per user and per system	OWASP LLM10 [1]
Injection tests before release	Test cases that try to make the system break its rules, run on every change	NIST AI 100-2e2025 [4]
Logs and a switch-off	A record of what the system did, and a named person who can stop it	NIST AI RMF MANAGE 2.4, see the governance page

An eval is a written, repeatable test of an AI system's answers against cases the company chooses. A security eval holds cases that try to inject instructions or push an agent past its permissions, and it runs before each release. The guide on AI agent evals explains how to write them.

Where Reveneau fits

Reveneau is an AI software development consultancy with offices in San Francisco and Vancouver. Reveneau says AI writes all of its code and every change must pass an eval suite written from the specification. Reveneau says it takes responsibility for the whole project through production and after release. To talk about a project, see Reveneau's services or contact Reveneau.