GitHub found shorter tool output made Copilot tasks cost more

GitHub published a post on 2 September, written by Erik Kristensen and Napalys Klicius, describing how it cut the token cost of Copilot's agent loop. The most useful part is the change that did not work.

The experiment that raised costs

The team built a utility called RTK, short for Rust Token Killer, to shorten the output that command-line tools hand back to the model. It did shorten the output. GitHub writes that "RTK shortened some responses, but when the omitted text mattered, the model sometimes reopened the original output or reran the command to recover what it needed."

Their summary of the result is one sentence: "We saved tokens locally and spent more globally."

GitHub states the general form of the problem as well. "A concise tool response can sometimes require additional calls or work if it leaves out information the agent needs, ultimately making the task slower and more expensive."

What did work

The savings that remained came from removing things the model never needed rather than from compressing things it did. GitHub reports 3.1% from removing view prefixes, 5.5% from selective output compaction, 2.9% from a more compact task-tool prompt, and 2.3% from cutting notification round trips. The task-tool prompt change alone saved roughly 1,300 tokens per turn and about 1.8% of total prompt tokens per session.

Removing line-number prefixes and selectively compressing output each cut average prompt tokens per review by roughly 5%, by GitHub's measurement. An earlier migration to a different set of file tools reduced code review cost by about 20%. In an online A/B test with Copilot CLI users, GitHub reports roughly a 3% reduction in average daily model-inference cost per user.

All of these figures are GitHub's own, measured on its own test setup and its own benchmarks, and the post does not name the models used. Nobody outside GitHub has reproduced them.

The finished task is the unit of measurement

Token spend is now a real cost for anyone running agents at scale, and the obvious way to reduce it is to send the model less. This is a documented case where doing that raised the cost, from a team with enough traffic to run a controlled experiment on it.

The distinction to use in your own work is between removing noise and removing information. Line-number prefixes on every row of a file are noise: the model does not use them, and deleting them cannot cause a retry. The tail of a failing test run is information, and cutting it makes the agent likely to run the test again. The first kind of cut is safe and the second has to be measured.

That also means the unit of measurement has to be the finished task, not the individual call. A dashboard showing average tokens per tool response would have scored RTK as a success. GitHub says it released these changes across the Copilot experiences that share the same underlying harness.

Source

GitHub, "How we make AI coding more cost efficient without sacrificing task quality"