The model choice experiment: Haiku, Sonnet and Opus on one task

This is the second experiment in Reveneau's Claude Code token benchmark. It asks what the same small task costs on each of the three models that the aliases haiku, sonnet and opus resolved to on 4 October 2026. Reveneau is an AI software development consultancy whose code is written by AI, so the choice of model is a cost decision in every build, and Reveneau is independent of Anthropic. The full method is on the method page.

The words used on this page

A token is a piece of text the model reads or writes; Anthropic's pricing page, read on 4 October 2026, estimates one token at 4 characters of English and prices every model per million tokens [3]. The context window is all the text the model reads in one request. Claude Code sends the whole conversation again on every request; the service stores the unchanged start, and tokens stored this way are cache write tokens, while tokens read back from the store on a later request are cache read tokens [4]. CLAUDE.md is a file of instructions loaded at the start of every session; this experiment used none. The effort level is a setting for how much reasoning the model does before each answer; this experiment passed no flag for it. A hook is a shell command Claude Code runs on its own at a fixed point; none was used. A non-interactive run is claude -p with a prompt: Claude Code does the task and exits. A turn is one request to the model and its reply. A median is the middle value of five sorted runs.

What was compared

Three settings, each run five times: --model haiku, --model sonnet and --model opus. Anthropic's CLI reference, read on 4 October 2026, says the --model flag takes an alias such as these or a model's full name [6]. On that day, in Claude Code 2.1.118, the aliases resolved to claude-haiku-4-5-20251001, claude-sonnet-4-6 and claude-opus-4-7, read from the modelUsage field of each json result, which Anthropic's cost tracking page describes as a map from model name to that model's token counts and cost [5].

Everything else was held constant: the task of one failing test in a 30-line Node.js project, the same prompt, no CLAUDE.md, no --effort flag, the flags on the method page, a Claude subscription, and a fresh copy of the folder for every run. The Sonnet setting is the same five runs as setting A of the CLAUDE.md experiment.

Because no --effort flag was passed, each model ran at whatever default effort level Claude Code 2.1.118 applied to it. Anthropic's model configuration page, read on 4 October 2026, states a default per model for the current version [2]; Reveneau did not verify that those defaults applied in 2.1.118, so this page does not state the level each model ran at.

The result

Every figure is Reveneau's own measurement from the run of 4 October 2026, a median of five, with the dollar figure being Claude Code's own list-price figure on a subscription account. The rows are in runs.json.

Model (as resolved)	Pass	Cost median (min to max)	Cache read median	Cache write median	Output median	Turns median	Seconds median
claude-haiku-4-5-20251001	5 of 5	$0.060 ($0.060 to $0.071)	222,836	26,543	903	9	15
claude-sonnet-4-6	5 of 5	$0.161 ($0.161 to $0.163)	184,772	25,579	659	7	21
claude-opus-4-7	5 of 5	$0.489 ($0.488 to $0.612)	367,005	45,364	873	7	21

All 15 runs passed.

Reveneau's arithmetic

Haiku's median cost of $0.060 is 37 percent of Sonnet's $0.161. Opus's median of $0.489 is 3.0 times Sonnet's. Both are Reveneau's arithmetic on medians of five runs.

Haiku took more turns, a median of 9 against 7 for the other two, and wrote the most output tokens, a median of 903. It still cost the least per run and passed every run. On a task this small, two extra turns on the lowest-priced model cost less than the same work on a higher-priced one.

Opus read more cache tokens per run than Sonnet, a median of 367,005 against 184,772, with the same median of 7 turns, and wrote more to the cache, 45,364 against 25,579. Reveneau's reading goes no further than the counts: the unchanged start of each request, which the Opus runs re-read every time, was larger. Reveneau did not verify why, and this page does not guess. Finding out would need a comparison of the Opus and Sonnet transcripts, which was not done in this run.

One Opus run took 159 seconds of wall-clock time (the time from start to finish on an ordinary clock) and 8 turns, against 18 to 22 seconds and 7 turns for the other four, and cost $0.612 against $0.488 to $0.490. The cause was not recorded. That run is the maximum in the table and the most costly of all 45 runs; the median of $0.489 reports the usual Opus run.

Anthropic's list prices for the three models

Anthropic's pricing page, read on 4 October 2026, gives these list prices per million tokens [3]. Claude Code's total_cost_usd is computed from a bundled price table at list price, by Anthropic's cost tracking page [5], so these are the rates behind the dollar figures above. Every cache write in the benchmark was a one-hour write, as the method page explains, so the one-hour column is the one that applied.

Model	New input	One-hour cache write	Cache read	Output
Claude Haiku 4.5	$1	$2	$0.10	$5
Claude Sonnet 4.6	$3	$6	$0.30	$15
Claude Opus 4.7	$5	$10	$0.50	$25

These are Anthropic's own prices for its own models. The measured cost ratio between Opus and Sonnet, 3.0, is larger than the ratio of any one pair of prices in the table by Reveneau's arithmetic, and the result table shows that the Opus runs also used more tokens of every kind. Both things are true at once; which part of the difference comes from price and which from token counts was not separated in this run.

The pricing page also says that Claude 4.7 and later models use a newer tokenizer (the rule a model uses to split text into tokens), and the limits page notes what that means for carrying these token counts to other models.

What Anthropic advises on model choice

Anthropic's costs page, read on 4 October 2026, says that Sonnet handles most coding tasks well and costs less than Opus, that Opus should be reserved for complex architectural decisions or multi-step reasoning, and that for simple subagent tasks (a subagent is a separate conversation the main one starts for part of the work) model: haiku can be set in the subagent configuration [1]. The same page names Opus left as the default model as one of the two usual causes of unexpectedly high spend on an API or cloud-provider plan [1].

Reveneau's measurement agrees with the direction of that advice on this task: the task is a one-line fix with a test to confirm it, Sonnet's median was $0.161 against Opus's $0.489, and Haiku's was $0.060. Reveneau's recommendation, drawn from Anthropic's documentation and consistent with this measurement, is to start a task on Sonnet and switch to a larger model only when the task needs it. The sister page on which model and effort level to choose covers Anthropic's guidance in full, and the agent costs guide covers giving a smaller model to a subagent.

That recommendation is for this task and tasks like it. A harder task could change the number of turns each model needs, and a model that takes more turns reads its growing conversation from the cache more times, as the token usage guide explains. Nothing in five runs of a one-line fix says what the three models cost on a task that needs reasoning.

Switching model mid-task was not measured

Anthropic's prompt caching page, read on 4 October 2026, says each model has its own cache, so switching models recomputes the entire request even when the content is identical [4]. Every benchmark run set its model at launch and never changed it, so no run paid that cost. The cost of a mid-task switch is covered in the prompt caching guide.

What this experiment cannot show

One invented task of one failing test in a 30-line project is one task. The medians are medians of five, and nothing here is a statistic about Claude Code in general.

The models were the ones Claude Code 2.1.118 resolved to on 4 October 2026. Anthropic's current documentation names Opus 5.5, Sonnet 5.5 and Fable 5.1, with different prices and defaults, and prices and behaviour for those models were not measured. The ratios on this page are ratios between claude-haiku-4-5-20251001, claude-sonnet-4-6 and claude-opus-4-7 and apply to no other pair.

The dollar figures are Claude Code's own list-price estimates on a subscription account. They do not show how much of a plan's allowance each run used, which no Claude Code output reports, and a plan may count models differently from the list prices; the usage limits guide covers what counts against a plan.

The task took a median of 6 to 10 turns per setting. A long session, a task that needs reasoning, or a task where one model fails and another passes would each give a different comparison, and the run measured none of them. A later dated run is planned, with no schedule.