Building compliant software in legal technology / Auditing what you have
Compliance gaps in legal software
A legal software product that works well for its users can still have real compliance gaps, because those gaps sit in what nobody thought to design rather than in something built carelessly. This page covers where those gaps concentrate and how to audit for them directly, rather than reviewing the product against its own design.
Key takeaways
- An audit that starts by reviewing the product's existing features can only confirm what was built. It cannot surface an obligation the product never addressed.
- The most common gaps concentrate in reads nobody logged, litigation holds that miss a backup or export copy, and AI features scoped for functionality but not for confidentiality boundaries.
- Enumerating every place client data comes to rest is the highest-value step, and it usually turns up more locations than the team expects.
- Each confirmed gap should close with a check that would catch it recurring, which a one-time fix cannot do.
A legal software product can pass every functional test, delight its users, and still carry real compliance exposure, because compliance gaps in a working application are almost never in the part somebody designed. They sit in the part nobody thought about. This page covers where those gaps concentrate specifically in legal technology, and how to find them.
Start from the obligations, not the code
The natural instinct when auditing an existing product is to walk through its features and check whether each one looks reasonable. That instinct produces a weak audit, because it can only confirm that what was built exists. It cannot surface an obligation the product never addressed at all, since nothing in that process ever mentions the obligation in the first place.
The stronger approach starts from the list of obligations covered across this guide, quoted from the actual rule text, and asks the system to demonstrate each one. Does a query scoped to one firm ever return another firm's records. Does a litigation hold placed through the interface actually reach the backup tier. Does the AI drafting feature's memory ever cross a matter boundary. Each of those is a specific, testable question, and asking it directly finds gaps a feature-by-feature walkthrough structurally cannot.
Where the gaps concentrate
Reads nobody logged. Confidentiality obligations under Rule 1.6(c) cover unauthorized access as well as unauthorized changes, and most applications log writes far more thoroughly than reads. A support engineer or an internal admin who can open client files without that access being recorded is a live gap in exactly the area the rule is most concerned with, and it is invisible to a feature review because nothing about the feature looks wrong when a read succeeds silently.
Holds that miss a copy. A litigation hold implemented as a flag checked by the primary application's own deletion job often does nothing to stop a backup snapshot, a data warehouse replica, or an export sitting in cloud storage from aging out on its own separate schedule. The team believes the hold worked because the record is still visible in the application. It is not visible in the place a court asked about, because nobody enumerated where else a copy of that record lives.
AI features scoped for functionality, not confidentiality. A drafting or summarization feature gets built, tested for whether it produces good output, and shipped, without anyone asking whether its memory or training pipeline crosses a matter boundary. This is a genuinely new category of gap, and it is easy to miss because the team evaluating the feature was thinking about quality, not about the confidentiality question covered in AI drafting features and confidentiality risk.
Metadata left in exports. A document export feature built to satisfy a "download my file" requirement rarely gets revisited for whether the exported file carries prior edits or comments in its metadata, because the export works, technically, and nobody thought to check what else came along with it.
UPL drift as features are added. A product that started as neutral legal information can drift toward giving specific advice as features are added incrementally, each one reasonable on its own, none of them individually flagged for review against the line covered in avoiding unauthorized practice of law in product design. No single commit looks like a problem. The cumulative product does.
Enumerate before you trace
Before checking any control, list every place client or case data comes to rest. This is nearly always longer than the team expects on the first pass: the primary database, backups at whatever interval they run, an analytics or reporting replica, a search index, an export bucket, a third-party AI provider that received the data as part of a prompt, a support ticketing system where a user pasted a document to describe a problem, and a spreadsheet somebody generated for a monthly report. Retention and confidentiality gaps concentrate in the locations nobody listed, not in the primary database everyone already checks.
Run the experiment, do not accept the description
For each control, the audit needs an actual experiment, not a description of intended behavior. Create two client matters with distinct facts and confirm data never crosses between them, including through an AI feature. Place a hold and confirm the record actually survives the deletion job across every location enumerated above. Export a document and inspect the resulting file's metadata directly. Attempt an unauthorized cross-firm query and confirm it fails, rather than trusting the data model's design.
A finding described as "handled in several places" is itself a finding, regardless of whether the current behavior happens to be correct, because a control maintained by convention across multiple call sites will be missed by the next feature added by a team that does not know all the places it needs to touch.
Close every finding with a check
A fix addresses a single instance of a gap. A check addresses the whole class of it going forward. Without the check, the same audit finds the same gap eighteen months later in a system that looked fixed at the time. With it, the closed finding becomes a permanent property, verified on every subsequent change, and it is the difference between a compliance program that improves over time and one that repeats the same review indefinitely. This is the same argument made in more depth on evals as legal compliance evidence.
What an audit does not decide
An audit of this kind finds gaps between the product's behavior and the obligations that were scoped for it. It does not itself decide which obligations apply, which is a legal determination that has to come from counsel before the audit starts, and it does not decide whether a specific finding rises to the level of a reportable incident, which depends on facts specific to the situation. What it produces is a concrete, evidence-based account of where the system's actual behavior diverges from what counsel says it needs to do, which is the starting point for fixing it.
If you want to run this kind of audit on an existing legal software product, get in touch, and why legal software needs a compliance-first build covers the broader argument for building the next version so the same gaps do not reappear.
Common questions
Why can't a compliance audit just review a legal software product's existing features?
Because reviewing existing features can only confirm what was built, and it cannot surface an obligation the product never addressed in the first place. A stronger audit starts from the actual rule text and asks the system to demonstrate each obligation directly, which finds gaps a feature-by-feature walkthrough structurally cannot.
Where do compliance gaps in legal software usually concentrate?
In reads that were never logged, litigation holds that stop at the primary database but miss a backup or export copy, AI features scoped for output quality but never checked for confidentiality boundaries, metadata left in document exports, and gradual drift where a product's features edge toward unauthorized practice of law one reasonable addition at a time.
Why does an audit start by enumerating where client data lives?
Because retention and confidentiality gaps concentrate in locations nobody thought to list, such as backups, analytics replicas, export buckets, and third-party AI providers that received data in a prompt. This enumeration step is nearly always longer than the team expects, and it has to happen before any control can be properly traced.
Should a compliance audit trust a description of how a control works?
No, it should run the actual experiment. Create two client matters with distinct facts and confirm they never cross, place a hold and confirm it survives the deletion job everywhere a copy of the data exists, and export a document to inspect its metadata directly, rather than accepting a description of intended behavior.
What does it mean if a control is described as handled in several places?
It is itself a finding, even if current behavior happens to be correct. A control maintained by convention across multiple call sites will be missed by the next feature a team adds without knowing every place that convention needs to be followed, so the property that matters is that it cannot break, not that it currently works.
Why should every audit finding close with a check rather than just a fix?
Because a fix addresses one instance of a gap while a check addresses the whole class of it going forward. Without a check, the same audit finds the same gap eighteen months later in a system that looks fixed, while a closed finding backed by a check becomes a permanent, continuously verified property.
Can a compliance audit decide which legal obligations apply to a product?
No, that determination has to come from counsel before the audit starts. An audit finds the gap between a product's actual behavior and the obligations counsel has already scoped for it, and it does not itself decide whether a finding rises to the level of a reportable incident, which depends on facts specific to the situation.
How is auditing an AI drafting feature different from auditing a traditional feature?
A traditional feature is usually reviewed for functionality and security, while an AI drafting feature needs an additional, specific check for whether its memory or training pipeline crosses a client matter boundary. This is a newer category of gap, and it is easy to miss because the team building the feature was evaluating output quality, not confidentiality isolation.
Related reading
A practical pre-launch security review for a small team
You do not need perfect security to launch. You need to check the handful of basics that catch most real problems, and to know when the risk is big enough to bring in a specialist.
How to prepare for technical due diligence before a raise or sale
Technical due diligence is where a deal can quietly fall apart. Here is what investors' technical reviewers actually look at, how to get ahead of it, and the red flags that scare them.
When to rewrite vs refactor legacy code (and why the big rewrite is usually wrong)
The full rewrite feels clean and honest, and it is almost always the wrong instinct. Here is when a rewrite is genuinely justified and how to replace old code piece by piece instead.