AgentCase Lab replays failed agent runs to find where they broke

A Python tool for debugging AI agent runs reached 180 GitHub stars in four days. The project is AgentCase Lab, by GitHub user TngSHoO. It stores every step of an agent's execution. A team can then step through a failed run and find the cause. The licence is MIT, the README states.

What it records

AgentCase Lab captures the whole flow of a run. It stores the user's input, the agent's plan, every tool call, and every model reply. It also stores the memory the agent read and wrote, and the final output. The data is stored in a local SQLite database that the app creates on first launch. The tool itself is a Streamlit web interface. It runs at http://localhost:8501 after one command.

What a user can do with it

The README lists five pages. The dashboard shows success rates, fix rates, root-cause distribution, latency, and cost. The case list filters failed runs by scenario, status and cause. The trace replay walks through a single run step by step. The root-cause view groups failures by category: prompt, retrieval, tool integration, context handling, memory, permissions, or model hallucination. A version comparison page puts two runs of the same task side by side. A team can then test whether a change actually fixed the problem.

How the analysis works

The rules behind the root-cause view are plain pattern matches, the README says. A team that wants richer analysis can plug in an OpenAI-compatible endpoint, with DeepSeek listed as one option. The API key is optional, so a team can run the whole tool offline. The project was built with Python 3.11 and tested with pytest and ruff.

What it does not promise

The local database starts with 30 demo cases. The README states these are made up. The dashboard numbers from the demo set should not be read as a real result. Any real reading of the dashboard comes from putting a team's own runs in. The data model is a small local store, not a logging agent. A team has to feed it from its own runtime.

Where it fits

The project sits next to the newer agent-debugging tools that read trace files and draw them as graphs. AgentCase Lab builds its own database schema. It keeps the cases across runs and tracks a fix rate over time. A team that already writes agent traces into its own store can pull them in through the SQLite file. A team that does not can start with the Streamlit app alone and add the recording later.

The repository was created on 4 November, which puts the 180 stars across four days. It has three forks, no open issues, and one push since launch.

Source

TngSHoO/agentcase-lab, GitHub.