Harvard says AI agents raise code 30 percent, finished work only 12

A reader whose team just added coding agents to measure the gain should know that the gain in finished work is roughly a third of the gain in code written. The gap is paid in code review, and the Harvard paper that reports it also reports how much slower review got.

The study was published by researchers at Harvard Business School using data from Jellyfish, a software company that connects to GitHub, Jira and calendars to measure engineering activity. The authors looked at roughly 300 million work events from 700,000 workers at more than 700 firms, from January 2021 to March 2026, alongside survey responses from 100 engineers on how they spend their time. GitHub Copilot, Cursor and Claude Code were among the tools in the dataset. InfoQ carried the write-up.

What the paper measures

The authors report that AI coding assistants raised lines of code by 12 percent, commits by 9 percent and pull requests by 5 percent, with only commits reaching statistical significance. AI agents raised all three measures by much more: 30 percent for lines of code, 20 percent for commits and 23 percent for pull requests.

They then measured output as Jira issues and epics resolved per worker per month. Assistants did not raise resolved issues at all. Agents raised them by 12 percent. That is well below the 30 percent increase in lines of code, so the extra code is not turning into finished work at the same rate.

Where the gap sits

To explain the gap, the paper looked at the steps that follow code writing. Review time per pull request rose by 49 percent. Comments per pull request rose by 53 percent. The share of pull requests that needed changes doubled. The share of workers performing code reviews rose by 14 percent, and AI review tools did not close that gap.

The authors note that engineers spend 30 to 40 percent of their time writing code, measured against the 100-engineer survey, and the rest on planning, review, testing and deployment. When one of those steps gets faster, another one becomes the bottleneck.

For a team adopting agents this quarter, the useful move is to measure the review queue before the rollout and again a month in. If review time per pull request moves the way the paper's numbers moved, the bottleneck has moved there too, and the pace of release will depend on that queue rather than on the lines-of-code chart on the agent dashboard.

Jellyfish has a commercial interest in a finding that says engineering output is complicated to measure, which is the service it sells. The paper's dataset is Jellyfish's, and the figures are presented as the authors' own.

Source

Harvard Business School working paper, covered on InfoQ by Sergio De Simone.