DoorDash cleans up feature flags with multi-agent LLMs, at 13.8 minutes and $4.79 per flag

DoorDash has built a multi-agent LLM workflow that automates the removal of stale feature flags across its codebase. In an evaluation of 50 flags, the system produced usable pull requests for 45 of them, at an average of 13.8 minutes and $4.79 per cleanup. DoorDash puts the manual cost at one to two hours per flag. The work was accepted for the ICSME 2026 industry track.

The size of the pile

DoorDash's experimentation platform holds more than 60,000 feature flags across roughly 623 repositories, and adds about 2,300 new ones each month. A flag is marked stale when it has not been modified for 90 days, is still referenced in code, and has not been archived, retired or excluded. A daily job files Jira tickets for the ones it finds; DoorDash says it has identified more than 1,000 stale flags already.

The cleanup is harder than a text search. DoorDash uses dependency-injected wrappers, so a single Boolean flag can touch five to 20 files, including tests, before it is fully removed. Uber's open-source Piranha tool works on the abstract syntax tree, and DoorDash reports that Piranha's rule-based approach did not cover its dependency-injection patterns, where the link between the flag and the business logic is semantic rather than something a matching rule can see.

The workflow, in two phases

The system runs on Google's Agent Development Kit. In the first phase, an orchestrator running Claude Sonnet reads the Jira ticket, searches the relevant repositories, and pulls metadata from the experimentation platform through Model Context Protocol, including rollout percentage and target value. An engineer reads the generated report and confirms the target value before any code changes start.

The second phase runs Claude Opus cleanup agents in isolated Git worktrees, up to four at once per repository. Each agent locates flag references, picks a cleanup strategy, edits source and tests, and runs builds, tests, JaCoCo patch coverage and Detekt static analysis. A pull request opens only if every check passes. Each agent has a one-hour timeout, and Gradle runs without its daemon so that one worktree cannot see the state of another.

The 45 usable pull requests broke down into 31 first-pass merges, 14 revisions, and five engineer interventions. Simple flags cleaned up on the first pass 100% of the time, medium-complexity flags 94%, and complex flags 85%. The five interventions all involved deep call chains and parameters threaded across interfaces. DoorDash says it found no bugs or regressions in the 50 evaluated changes.

The number worth stopping on is $4.79 per cleanup. Across 1,000 stale flags, that is under $5,000 to remove work that would otherwise take an engineer between 1,000 and 2,000 hours. An engineer still confirms the target value up front and still reviews the pull request at the end, so the change is in what the engineer spends the hour on.

DoorDash plans to add confidence scoring for lower-risk cleanups and a code-quality pass after the removal, to catch things like variable names that no longer make sense once a flag is gone.

Source

Automating Feature Flag Cleanup at Scale with a Multi-Agent LLM System (DoorDash Engineering). Reported by Leela Kumili at InfoQ.