Berkeley study finds 10 minutes of AI use erodes task persistence

A coding assistant that writes most of your code is still a tool you have to stop using when it is wrong, and a new peer-reviewed study measures what that moment looks like.

Researchers at the UC Berkeley Center for Human-Compatible AI, Carnegie Mellon, MIT, Oxford and UCLA published results from three randomised controlled trials with 1,222 participants. Grace Liu, Brian Christian, Tsvetomira Dumbalska, Michiel A. Bakker and Rachit Dubey report that 10 minutes of using ChatGPT on a task leaves people worse at that same task once the tool is taken away, and more willing to skip hard questions. The paper, "AI Assistance Reduces Persistence and Hurts Independent Performance", is now in its fifth arXiv version and was presented at the Conference on Language Modeling.

What the three trials ran

The first trial gave 354 participants a set of fraction problems. One group had ChatGPT access for the first 12 questions and then lost it; the other group never had access. Berkeley's own write-up says the AI group was accurate while the tool was on, and that "almost immediately" after the tool was withdrawn, that group stopped solving questions accurately and started skipping. The chart the researchers published shows a sharp fall in the solve rate and a sharp rise in the skip rate at the moment of withdrawal.

The second trial repeated the fractions test with a larger group of 667 participants. The third trial used SAT reading comprehension prompts with 201 participants. The paper's abstract reports the same pattern across all three: AI help improves performance in the short term, people perform worse without AI afterwards, and they are more likely to give up on difficult problems. The authors state the effects appear within the first 10 minutes of interaction.

What the authors say the mechanism is

The paper puts the cause as a change in expectations. The authors write that AI "conditions people to anticipate instant answers", which deprives them of the struggle the paper argues is necessary for learning. Their recommendation to AI developers is that products should "scaffold long-term competence alongside immediate task completion" rather than optimise for the single-session answer.

Commenters on the Hacker News thread question whether the controls go far enough. The study compares an AI-access group against a no-access group on the same task, and the thread asks what the same test would look like against a calculator, a textbook, or a human tutor. The paper is open on arXiv for a reader who wants to work through the method and judge the controls for themselves.

For a team that uses a coding assistant every day, the direct reading is that the practice of debugging, reviewing and reasoning through a problem does not maintain itself in the background. The study used fraction arithmetic and SAT reading passages, so a step from those tasks to programming is the reader's own. The pattern the authors measure is about any task where an assistant hands over a complete answer and then disappears. The question for a team is whether the review that catches an assistant's mistake also starts to erode when most of the writing happens elsewhere.

Source

The paper "AI Assistance Reduces Persistence and Hurts Independent Performance" by Grace Liu, Brian Christian, Tsvetomira Dumbalska, Michiel A. Bakker and Rachit Dubey, with a plain-English write-up from UC Berkeley and the launch thread on Hacker News.