NBER study finds AI coding tools lift commits 240 percent but releases only 30 percent

A software team measures its AI coding tool by lines written, pull requests opened, or model cost per task. An NBER working paper by Mert Demirer, Leon Musolff and Liyuan Yang asks a different question. If an AI tool lifts code-writing activity by a certain amount, how much of that reaches production? Their answer, from a sample of over 500,000 GitHub developers: most of it does not.

The paper is NBER working paper 35275. It was first published in May 2026 and revised in September. Chris Bird is a senior staff engineer at Dryden Technology Group. He folded the paper into a 21-minute talk at a UC Irvine seminar on 30 October 2026. The talk reached the front page of Hacker News this week with 116 points. Its title is "AI, activity, and outcomes in software engineering." Bird calls the finding the activity-outcome gap.

The number, and how it was built

The NBER authors matched over 500,000 GitHub developers against their AI usage telemetry. They tracked three generations of AI tools: autocomplete, interactive agents, and autonomous agents. They then measured what happened at three points in the production chain. The result, in their own words: "The 240% cumulative effect falls to 80% for the number of projects, and to 30% for actual releases."

Code commits per developer were lifted 240 percent across the three generations. Projects started per developer were lifted 80 percent. Actual software releases were lifted only 30 percent. The paper reports an elasticity of substitution of 0.23 between AI effort and human effort. The authors say that indicates "strong complementarities" rather than substitution. Writing code is where the lift concentrates. Putting code out is not.

What Bird added

Bird's talk links the NBER result to a second paper his group has in submission. It is called "The Activity-Outcome Gap: Why AI Does Not Solve Software Engineering." It is a qualitative study of 30 interviews. The gap the NBER numbers find at scale shows up inside a team too. The bottleneck has moved. Code review, continuous integration, testing, and product decisions are where the extra activity gets held.

Four more arXiv papers from Bird's group this year cover pieces of the same pattern. One asks when a software team should run with the lights off. One asks where developers draw the line on AI autonomy. One asks what AI actually lets developers unlock. One asks what drives a developer and a team to adopt generative AI tools. The talk and the handout list every paper by name on cabird.com.

What a team can do with the number

A team paying for a coding agent now has a specific question to put to its own data. The vendor sold a lift in code output. What fraction of that reached production in a time a user can measure? A team that writes 240 percent more code and puts out 30 percent more releases is matching the NBER sample. A team that writes much more code and releases the same amount has the activity-outcome gap as a local problem.

Bird's own suggestion in the talk is that the fix is not more AI at the keyboard. It is the review step, the deploy step, and the product decision. Those are the places the extra code waits. A team speeding those up gets more of the 240 percent to production. A team that only speeds up the keyboard keeps the gap.

Source

NBER working paper 35275 by Mert Demirer, Leon Musolff and Liyuan Yang, and Chris Bird's talk Why Aren't We Shipping Faster Now?.