Why are permissions harder in an AI feature than a normal one?

Because a normal feature answers a narrow question about one record for one user, while an AI feature reads across many sources and combines them into a summary. Once the content is merged into a paragraph, there is no field left to attach an access rule to, so the check has to happen earlier.

Where do permissions usually fail in a retrieval feature?

Most often in the index itself, because it was built by a process running with administrative access and therefore contains everything. The retrieval layer can then see documents the asking user cannot, and the model summarises them without knowing that an access limit existed.

Should you filter before or after retrieval?

Before, always, using the identity of the person asking. Filtering after generation is unreliable because by then nobody can tell which source each part came from, and asking the model to withhold what it has already been shown depends on instruction-following rather than on an enforced rule.

What happens when someone's access is revoked but the index is not rebuilt?

The old embedding still contains that content and stays retrievable, so the revocation exists in the permission system and not in the index. Access changes have to be copied into the retrieval layer as an explicit step, otherwise the index is a snapshot of who could see what on the day it was built.

What is the inference problem in AI permissions?

A case where every individual document is permitted but the combination reveals something none of them exposed alone, such as a headcount summary across regions revealing an unannounced reorganisation. Document-level checks cannot catch it, because each check passes correctly, so the practical defence is deciding in advance which aggregations are sensitive and treating those as items with their own permission rules.

What identity should an AI agent act with?

The identity of the person who asked, with the same limits that person has, rather than a shared service account. An agent holding broader access will do whatever a user persuades it to do at a permission level that user was never granted, which turns a prompt into a way to gain access the user should not have.

How do you test permissions in an AI feature?

With at least two real accounts that are supposed to see different things, asking each the same questions and comparing the answers. A permission failure here does not raise an error or crash, it returns a fluent and helpful answer to the wrong person, so only a comparison reveals it.

Why do permission bugs in AI features stay hidden so long?

Because nothing fails visibly. The user receives a plausible answer and has no way to know it drew on material they should not have seen, so unlike a crash or an error page there is no signal that anything went wrong until an audit or a complaint.

Does logging need to change for AI features?

Yes, because the useful record is what was retrieved rather than only what was asked and answered. Without knowing which sources were used for a given response, you cannot answer the one question that matters after an incident, which is whose data was exposed and to whom.

How should permissions affect the design of a new AI feature?

They should be decided at design time rather than added later, because the access model determines the retrieval architecture and the indexing strategy. Adding real permission limits to a working prototype usually means rebuilding the retrieval layer, which is why this belongs in the first design conversation.

What is the minimum safe default for a first AI feature?

Restrict the feature to a single clearly limited data set where every user with access to the feature is entitled to everything in it. That removes the hardest problem while the team learns the rest, and it converts a difficult access design into one simple access check.

Why permissions are the hardest part of an AI feature: Reveneau

Permissions come up last in most AI feature conversations. The retrieval approach gets decided, the model gets chosen, the interface gets designed, and somewhere near the end someone asks how access control works. By then the architecture has already answered the question badly.

The reason this happens is that permissions look like a solved problem. Every serious application has an access model, and it works. The mistake is assuming that model still applies once a feature stops returning records and starts returning summaries.

The structural difference

A conventional feature answers a narrow question. Show this user this order. One user, one record, one check, and thirty years of tooling exist to make that check reliable.

An AI feature reads across everything it can reach and writes a paragraph. The paragraph is new. It did not exist in your database, no row-level rule covers it, and once material from twelve sources has been combined into three sentences, there is nothing left to attach a permission to. The check has to happen before the content is combined, or it cannot happen at all.

That single fact drives everything else.

Where access control actually fails

Four places, and none of them is unusual. We look for all four in any design review.

The index that can see everything

Someone builds the retrieval index with a process running as an administrator, because that is the straightforward way to read all the documents. Now the index holds everything, and the retrieval layer can reach content the asking user cannot.

If the filtering happens after retrieval, or is expressed as an instruction in the prompt, the access limit does nothing. The model has already been shown the material, and asking it not to mention what it has seen depends on instruction-following, which is not an access control mechanism.

The rule: filter at retrieval time, using the identity of the person asking, on every request. Never retrieve broadly and narrow afterwards.

Access removed in one place but still in the index

Someone loses access to a folder on Monday. The permission system records that correctly. The embedding created in March still sits in the index and contains that content, and it is still retrievable.

Removing access has to be copied into the retrieval layer as a deliberate step. Otherwise the index is a fixed record of who could see what on the day it was built, and it becomes more wrong every week. The related problem of copies that remain after permission is removed is covered in retention and deletion across copies.

The combination that reveals what no single document did

This one is genuinely hard and has no simple technical fix.

Every document the feature drew on is permitted. The summary across them exposes something none of them exposed alone. A manager asks a question about staffing across regions, receives a correct aggregate, and can infer a reorganisation that has not been announced. Every access check passed, correctly.

Document-level permissions cannot catch this, because the failure is not at the document level. What helps is deciding in advance which aggregations are sensitive and treating those as items with their own permission rules, rather than trying to reason about every possible inference. That is a product decision more than an engineering one, and it has to be made by someone who knows the business.

The agent acting with its own access

An agent that takes actions with a service account holds whatever that account holds. A user who cannot see finance data can ask the agent a question, and the agent, running with broad access, answers it.

The prompt becomes a way to gain access the user should not have. The fix is that the agent acts with the identity of the person who asked and inherits their limits, which is more work to build and the only version that works. The detailed treatment is in AI agent permissions and guardrails.

Why these stay hidden so long

A permission bug in a normal feature is easy to notice. Someone sees a page they should not, or gets an error, and it gets reported.

A permission bug in an AI feature returns a fluent, helpful, correct-looking answer to the wrong person. Nothing fails. The user cannot tell that part of the response came from material they were never entitled to, because the response does not carry that information. There is no error to notice and nothing to report.

That silence is the whole problem. These defects are found in audits, or in complaints, or not at all.

It also means the usual review process will not catch them. Veracode's Spring 2026 research across more than 150 models found generated code compiling and running correctly over 95 percent of the time while passing security checks in roughly 55 percent of generations, with the weakest types being the ones that cannot be seen at normal reading speed: cross-site scripting at 15 percent and log injection at 13 percent. Access control in a retrieval pipeline has the same property. Every line looks reasonable. The defect is in the relationship between them.

How to test it

Two accounts, minimum. Real ones, belonging to people who are supposed to see different things.

Ask both the same set of questions, including questions deliberately aimed at the access limits, and compare the answers. This is the only reliable way to find the failure, because there is no exception to catch and no status code to check. You are comparing two outputs and looking for content that should only have appeared in one.

Then automate exactly that, as a permanent check that runs on every change. The setup cost is a day. Related method is in how to write your first eval suite, and the trap of checks that cannot fail is in when evals give false confidence.

One more test worth writing: revoke an account's access, do not rebuild the index, and confirm the feature stops returning that content. That is the old-access leak, and it is the one most likely to be silently present right now.

Logging has to change too

Standard request logging records the question and the answer. For an AI feature that is not enough.

The record you need is what was retrieved: which sources fed which response, for which user, at what time. Without it, the question that arrives after an incident, whose data was exposed and to whom, has no answer. You will only be able to guess what the system probably did, which is not something anyone wants to write in an incident report. The broader pattern is in building an audit-ready codebase.

The design-time decision

The reason to settle this first is that the access model determines the architecture. Whether you can use one shared index or need per-tenant separation, whether filtering happens in the vector store or earlier, how often the index rebuilds: those follow from the permission requirements.

Add real access limits to a working prototype later and you usually rebuild the retrieval layer. That is the specific, expensive rework that this conversation, held early, prevents.

If you want a safe starting point: restrict the first version to one clearly limited data set where everyone with access to the feature is entitled to all of it. The public help centre. One team's own documents. That removes the hardest problem entirely while the team learns everything else, and it turns a difficult access design into a single access check at the start.

Thanks to the security reviewers who asked us the awkward question early rather than late. The most useful one was simply: what can the index see, and who built it.

A permission model that fails once content is summarised was only ever a filter on a screen.

Related guide: When a human must stay in the loop.

Sources

Veracode, Spring 2026 GenAI Code Security update: across more than 150 models, over 95 percent syntactic correctness against roughly 55 percent security pass rate, with cross-site scripting at 15 percent and log injection at 13 percent.

Why permissions are the hardest part of an AI feature