The effort level experiment: low, medium and high on Sonnet 4.6

This is the third experiment in Reveneau's Claude Code token benchmark. It asks what the effort level costs on a task that does not need much reasoning. Reveneau is an AI software development consultancy whose code is written by AI, so a setting that changes the cost of every request is worth measuring before it is set as a default across builds. Reveneau is independent of Anthropic. The full method is on the method page.

The words used on this page

A token is a piece of text the model reads or writes. The context window is all the text the model reads in one request. Claude Code sends the whole conversation again on every request; the service stores the unchanged start, and tokens stored this way are cache write tokens, while tokens read back from the store on a later request are cache read tokens [3]. CLAUDE.md is a file of instructions loaded at the start of every session; this experiment used none. The effort level is a setting that tells the model how much reasoning to do before each answer: Anthropic's model configuration page, read on 4 October 2026, says effort levels control adaptive reasoning, which lets the model decide whether and how much to think on each step, and that lower effort is faster and costs less for straightforward tasks while higher effort gives deeper reasoning for complex problems [1]. A hook is a shell command Claude Code runs on its own at a fixed point; none was used. A non-interactive run is claude -p with a prompt: Claude Code does the task and exits. A turn is one request to the model and its reply. A median is the middle value of five sorted runs.

What was compared

Three settings, each run five times on claude-sonnet-4-6: --effort low, --effort medium and --effort high. Anthropic's CLI reference, read on 4 October 2026, says the --effort flag sets the effort level for the current session, overrides the saved settings for that session and does not persist [4]. The model configuration page lists the levels each model offers: low, medium, high and max for Opus 4.6 and Sonnet 4.6, and xhigh in addition on Opus 4.7, Opus 4.8, Opus 5, Sonnet 5, Opus 5.5 and Sonnet 5.5 [1]. The benchmark did not run max.

The fourth row of the table is the no-CLAUDE.md Sonnet setting from the CLAUDE.md experiment, which passed no --effort flag. It is reported separately, and the reason is explained below.

Everything else was constant: the task of one failing test in a 30-line Node.js project, the same prompt, no CLAUDE.md, the flags on the method page, Claude Code 2.1.118 on a Claude subscription, and a fresh copy of the folder for every run. All runs were on 4 October 2026.

The result

Every figure is Reveneau's own measurement from that run, a median of five, with the dollar figure being Claude Code's own list-price figure on a subscription account. The rows are in runs.json.

Effort	Pass	Cost median (min to max)	Cache read median	Cache write median	Output median	Turns median	Seconds median
low	5 of 5	$0.150 ($0.149 to $0.208)	150,161	25,515	644	6	16
medium	5 of 5	$0.162 ($0.149 to $0.201)	184,771	25,600	677	7	16
high	5 of 5	$0.170 ($0.161 to $0.174)	187,716	25,636	655	7	17
default (no flag), from experiment 1	5 of 5	$0.161 ($0.161 to $0.163)	184,772	25,579	659	7	21

All 15 flagged runs passed, and so did the five default runs.

Reveneau's reading

The effort level moved the median cost by $0.020 between low and high, from $0.150 to $0.170, which is 13 percent of the low median by Reveneau's arithmetic. The ranges overlap: one low run cost $0.208, which is above every high run, and the lowest medium run and the lowest low run both cost $0.149. Five runs can order the medians on this task; they cannot separate the settings run by run.

The difference in medians came from turns. Low effort had a median of 6 turns and medium and high had 7. Each turn re-sends the whole conversation and reads the unchanged start from the cache, so one more turn is one more read of the growing unchanged start; the median cache read count was 150,161 at low, 184,771 at medium and 187,716 at high. The cache write medians were within 121 tokens of each other, 25,515 to 25,636, by Reveneau's arithmetic, which fits a run that builds the same unchanged start at every level.

The output token medians were 644, 677 and 655, within 33 tokens of each other. Anthropic's costs page, read on 4 October 2026, says thinking tokens are billed as output tokens and that the default thinking budget can be tens of thousands of tokens per request depending on the model [2]. If the model had reasoned at length at high effort, that would have shown in the output column. It did not, because the task is one line and a test. The experiment therefore measures what the flag costs when the task does not use the reasoning it allows: on this task, one turn's worth of cache reads.

Why the default row is separate

Anthropic's model configuration page, read on 4 October 2026, says Claude Code resolves the effort level in order: an explicit choice such as --effort, then the saved settings, then the model's default, which is high on every model that supports effort except Opus 5.5 and Sonnet 5.5 at medium and Opus 4.7 at xhigh [1]. That page describes the current version. Reveneau did not verify which level Claude Code 2.1.118 applied to claude-sonnet-4-6 when no flag was passed, and the default row's median of $0.161 is reported beside the three flagged rows without labelling it as any of them. The default row also differs from the flagged rows in its median wall-clock time (the time from start to finish on an ordinary clock), 21 seconds against 16 or 17, and the benchmark did not investigate that.

What the task cannot show about effort

The task is too small to need extended reasoning. The fix is one line, the test confirms it, and the median output was 644 to 677 tokens at every level. Anthropic's model configuration page describes high as the level for work where verification matters or edge cases are likely, and max as the level for hard problems the model works through on its own, with a warning that it may show diminishing returns [1]. None of that applies to this task, so the experiment cannot show what effort does on a task that does use it. On a task where the model reasons at length, the output column would move, and the cost with it.

The experiment also did not run max, and did not run any level on Haiku or Opus. The model choice experiment ran those two models at their defaults only.

Changing effort mid-task was not measured

Every run set its level at launch and kept it. Anthropic's prompt caching page, read on 4 October 2026, says that on most models each effort level has its own cache, so changing effort mid-session recomputes the entire request, and that on Opus 5.5, Sonnet 5.5 and Fable 5.1 with an API key or a Claude subscription the cache stays intact [3]. No run in the benchmark paid or avoided that cost; the prompt caching guide covers it.

What Reveneau recommends

Reveneau's recommendation, drawn from Anthropic's documentation and consistent with this measurement: pick the effort level for the task at the start of the session, and use a lower level for a small, well-defined change where you review each result. Anthropic's costs page says that for simpler tasks where deep reasoning is not needed, lowering the effort level reduces cost [2], and on this task low effort passed every run in a median of 6 turns for $0.150 by Claude Code's figure. The saving on this task was $0.020 per run between low and high, and single runs varied by more than that, so on a task this small the effort level is a smaller cost decision than the model, where the model choice experiment measured a difference of 3.0 times between Sonnet and Opus. The sister page on which model and effort level to choose covers Anthropic's guidance for each level, and the token usage guide explains why the turn count set the cost on this task.

What this experiment cannot show

One invented task of one failing test in a 30-line project is one task. The medians are medians of five, and nothing here is a statistic about Claude Code in general. The model was claude-sonnet-4-6 as resolved by Claude Code 2.1.118 on 4 October 2026; the same version resolved opus to claude-opus-4-7 and haiku to claude-haiku-4-5-20251001, and no effort level was run on either. Anthropic's current documentation names Sonnet 5.5 with a different default effort and a different cache behaviour on an effort change, and that model was not measured. The dollar figures are Claude Code's own list-price estimates on a subscription account, and they do not show how much of a plan's allowance a run used, which no Claude Code output reports. The limits page lists everything the run cannot show, and a later dated run is planned, with no schedule.