Legora builds legal AI agents and never reads a single trace

Most teams that build an AI product read what the model did and work backwards from there. A team that cannot read any of it still has to find out why their customer is unhappy. Legora's CTO Jacob Lauritzen told Latent Space how his team does that.

Legora sells software to law firms. The data in the system belongs to each firm's clients. Lauritzen told Latent Space the team cannot see a single trace of what its agent does for a lawyer.

How they test what they cannot see

Legora builds small simulated worlds that copy a customer workflow. Lauritzen said the team treats each world like a training setup for a game-playing agent. The agent runs inside the world, and the team scores the outcome. The real data stays with the law firm.

The evals team is three engineers and three lawyers. They do not read raw production traces. They read paraphrased summaries that a cheap language model produces. The summaries drop any detail that could point back to a client. The team then sorts failures into ten or twenty known failure types, Lauritzen said. Cheap language-model checks sit behind feature flags and watch for each type in new runs.

Lauritzen described one case where the method paid off. Citations were wrong 30 percent of the time. The next model release lifted citation accuracy from 70 percent to 99 percent, he said. Legora had changed nothing on its side. The evals told the team the problem was gone before any lawyer wrote in.

The company behind the method

Legora employs about 100 engineers, Lauritzen said. The number grew from an original plan of 20. The company also has about 100 "legal engineers". These are lawyers with three or four years in large law firms, often with a computer-science degree. The first forward deployed engineer joined in New York nine months ago.

Legora said it serves more than 1,800 customers and has passed 200 million dollars in annual recurring revenue. The company passed 100 million dollars in April, less than 18 months after launch. Legora raised a Series E in October at an 8.8 billion dollar valuation, Lauritzen said.

The company runs on Azure because of data-privacy rules. Lauritzen said the team avoids outside services such as third-party transcription when a client's data would leave the control plane. The product includes Tabular Review, which lets a lawyer open 10,000 contracts in one grid. Each row is a document and each column is a question. The grid holds up to one million cells. The team runs the first ten rows to tune the prompt before the full job runs.

Source

• Latent Space podcast: Building Legal Agents When You Can't See the Data, interview with Jacob Lauritzen, CTO of Legora