Measuring what AI returns: what the studies found, and how to measure your own

Public bodies set up to help companies adopt digital technology, including artificial intelligence (AI), "frequently highlight uncertainty over the return on investment as a critical obstacle for firms considering adopting AI", according to a report by the OECD (the Organisation for Economic Co-operation and Development), Boston Consulting Group (BCG) and INSEAD [13]. Return on investment means what a project gives back compared with what it costs.

Studies that measured AI at work have reached different results. This page sets out what five studies found, what firms report in two official surveys, why time saved is a different figure from money saved, and how a large company measures its own results. It is part of the guide on enterprise AI adoption.

What the studies found

Study	Who was studied	What changed with AI	Limits the authors or sources state
Brynjolfsson, Li and Raymond, Quarterly Journal of Economics, 2025 [1]	5,172 customer support agents at one software firm	Issues resolved per hour up 15% on average; up 30% for less experienced and lower-skilled agents	One firm, medium-run effects
Dell'Acqua and others, Harvard Business School working paper, 2023 [3]	758 BCG consultants	12.2% more tasks, 25.1% faster, more than 40% higher quality on tasks inside AI's abilities; 19 points less likely to be correct on one task outside them	Working paper; BCG staff are co-authors
Noy and Zhang, Science, 2023 [4]	453 college-educated professionals on writing tasks	Time taken down 40%, output quality up 18%	Writing tasks in an online experiment
Bick, Blandin and Deming, Federal Reserve Bank of St. Louis blog, 2025 [5]	US workers in a nationally representative survey	Users saved 5.4% of work hours; 1.4% of all workers' hours	Self-reported time savings; the 2024 results were later revised
Humlum and Vestergaard, National Bureau of Economic Research, revised 2026 [6][7]	25,000 workers in 7,000 Danish workplaces	No measurable effect on earnings or recorded hours; effects above 2% ruled out	Two years after ChatGPT; 11 exposed occupations

Customer support: 15% on average

Erik Brynjolfsson, Danielle Li and Lindsey Raymond studied "5,172 customer-support agents working for a Fortune 500 firm that sells business-process software" [1]. The agents' AI assistant was built on GPT-3, a large language model from OpenAI. Their paper in the Quarterly Journal of Economics, published on 4 February 2025, says: "Access to AI assistance increases worker productivity, as measured by issues resolved per hour, by 15% on average" [1].

The gains were uneven. Less skilled and less experienced agents saw a 30% increase in issues resolved per hour [1]. The most experienced and highest-skilled agents "see small gains in speed and small declines in quality" [1]. New agents learned faster: "treated agents with two months of tenure perform just as well as untreated agents with more than six months of tenure" [1].

The authors state the limit: "We emphasize that these findings capture medium-run effects in a single firm" [1]. They add that the paper was not designed to show the effect of generative AI on total employment or wages [1].

Other websites often give 14% for this study. That figure comes from an earlier working paper, which reported a 14% average gain and a 34% gain for novice and low-skilled workers [2]. The published paper reports 15% and 30% with 5,172 agents, and this guide uses the published figures.

Consulting work: gains inside AI's abilities, losses outside

Fabrizio Dell'Acqua and co-authors ran a pre-registered experiment with 758 BCG consultants. On 18 tasks inside AI's abilities, consultants using GPT-4 completed 12.2% more tasks on average, finished them 25.1% faster, and produced results more than 40% higher in quality compared with a control group [3]. On a task chosen to be outside AI's abilities, "consultants using AI were 19 percentage points less likely to produce correct solutions compared to those without AI" [3]. The same tool helped on one set of tasks and hurt on another.

Writing: 40% less time

Shakked Noy and Whitney Zhang gave writing tasks to 453 college-educated professionals in an online experiment, published in Science on 13 July 2023. With ChatGPT, "The average time taken decreased by 40% and output quality rose by 18%" [4]. They also found that "Inequality between workers decreased" [4]. The journal's editor's summary puts it this way: participants with weaker skills benefited most [4].

Across the workforce: smaller numbers

Alexander Bick, Adam Blandin and David Deming, writing on the Federal Reserve Bank of St. Louis blog, asked workers how much time generative AI saved them. "We found an average time savings of 5.4% of work hours in the November 2024 survey" among users, and across all workers, including those who did not use it, "workers saved 1.4% of total hours because of generative AI" [5]. They translate that into a 1.1% increase in aggregate productivity, from a model they describe as giving "a rough estimate" [5]. The post now carries a note that its 2024 survey results were revised in a later post, after a change in the order of the survey questions [5]. This page gives the figures as the February 2025 post states them.

Anders Humlum and Emilie Vestergaard linked survey answers to official labour market records for 25,000 workers in 7,000 workplaces in Denmark [7]. The workers were in 11 occupations exposed to AI chatbots, and the latest survey round ran in late 2024 [7]. They "estimate precise null effects on earnings and recorded hours at both the worker and workplace levels, ruling out effects larger than 2% two years after the launch of ChatGPT" [6]. A null effect means no measurable change. Their paper is a working paper of the National Bureau of Economic Research (NBER), revised in March 2026.

The 2026 AI Index from Stanford University sums up other studies: "Studies report gains of 14% to 15% in customer support, 26% in software development, and 50% in marketing output" [15]. Only the customer support figure is traced to its paper on this page.

What firms report

Official surveys ask firms about jobs. In the US Census Bureau's AI supplement, collected 17 November 2025 to 8 February 2026, 95.7% of firms that used AI said AI did not change their total employment; 2.3% said it went up and 2.0% said it went down [8]. Because of Executive Order 14434, signed on 29 September 2026, the Census Bureau now calls AI "Super Intelligence" or "SI" in its files [9][10]. Its questions asked about "Artificial Intelligence (AI)" until the collection that began on 19 October 2026, and this supplement closed on 8 February 2026, so this page says AI [10].

In the United Kingdom, the Office for National Statistics (ONS) reports: "In September 2026, of businesses with 10 or more employees that were currently using AI, 6% reported reduced headcount as a result" [11]. Among businesses using AI to improve business operations, "63% reported no change in worker headcount, while 1% reported increased headcount and 6% reported decreased headcount" [12].

Survey	Base	No change in staff numbers	Fewer staff	More staff
US Census Bureau AI supplement [8]	Firms that used AI, November 2025 to February 2026	95.7%	2.0%	2.3%
UK ONS [12]	Firms using AI to improve business operations, June 2026	63%	6%	1%

Time saved is a different number from money saved

The Danish study offers one explanation for the gap between task studies and the wider figures. It reports that "most chatbot users (85%) report reallocating time savings from AI chatbots to other job tasks" [7]. When staff save an hour, most of them spend it on other work. Payroll does not fall on its own, and the company gets more of some other work, which may or may not be worth more.

Reveneau's view is that a company should decide in advance what saved time is for: more cases handled with the same team, faster answers to customers, fewer errors, or new work that was waiting. Each of those can be measured. "Time saved" on its own cannot be checked against the accounts.

How to measure your own return

The US National Institute of Standards and Technology (NIST) AI Risk Management Framework asks a company to write down the costs of AI errors. Item MAP 3.2 covers "Potential costs, including non-monetary costs, which result from expected or realized AI errors or system functionality and trustworthiness" [14]. The framework also says organisations "are encouraged to periodically evaluate whether the AI RMF has improved their ability to manage AI risks" [14]. Reveneau's view is that the same idea fits returns: measure before release, measure after it, and repeat the measure on a schedule.

The table below is Reveneau's measurement plan. It uses the measures the studies above used, so a company's own figures can be read next to theirs.

Measure	How to count it	Which study used it
Output per hour	Cases, tasks or documents finished per hour of staff time	Brynjolfsson and others [1]
Quality	Error rate in a sample checked by a person, or the pass rate on a written set of test cases	Dell'Acqua and others [3]; Noy and Zhang [4]
Time to competence	Weeks until a new hire reaches the team's normal output	Brynjolfsson and others [1]
Hours saved, and where they went	Survey of staff, checked against output	Bick and others [5]; Humlum and Vestergaard [7]
Cost of errors	Money and non-money costs of mistakes the AI makes	NIST MAP 3.2 [14]
Cost per task	Staff time plus model fees, vendor fees and support, divided by tasks finished	Reveneau's addition

In Reveneau's view, three rules make the numbers believable. First, take the baseline before the tool goes live. Second, use a comparison group where possible, as the consulting experiment did with its control group: a team without the tool doing the same work [3]. Third, check quality as well as speed, because the support study found small declines in quality for the most skilled agents [1].

An eval suite helps with the quality line. An eval is a written, repeatable test of an AI system's answers against cases the company chooses, and the pass rate on the same suite, measured before and after a change, is a quality figure that can be repeated. The guides on why AI evals matter and LLM evals explain how to build one. The page choosing your first AI use cases shows how to pick work that can be measured, and build, buy or partner covers the costs of each route.

Where Reveneau fits

Reveneau is an AI software development consultancy with offices in San Francisco and Vancouver. Reveneau says AI writes all of its code and every change must pass an eval suite written from the specification. Reveneau says Jev, TypeSafe AI's decision model, grades that suite, and that the suite runs ten times faster than it did with its previous grader. Reveneau says it uses AI instead of hiring more engineers and puts that saving into clients' prices. To talk about measuring a project, see Reveneau's services or contact Reveneau.