Nine coding harnesses on a MacBook, and Opencode read 18,046 tokens of system prompt before the model wrote anything

N. Sutton, the author of a local-first coding harness called chad, has published a bench that runs nine coding harnesses through the same eight Exercism exercises with the same local model. The run reached the front page of Hacker News with 163 points. The author states up front that they build one of the harnesses in the test, so read it with that in mind.

Every harness ran on the same M4 MacBook Pro with 24 GB of memory, macOS 26.6.2, and a 3-bit quantisation of Qwen 3.8 27B served through llama.cpp build 10470. A single llama-server instance was shared across all harnesses through a common proxy that pinned the same sampling settings for Qwen. Every number in the tables came from llama-server's own accounting through that proxy, not from the harnesses' self-reports, with two exceptions the author calls out for chad's in-process MLX engine.

Prefill is what you feel

Sutton frames the problem as three ways a harness works against a local model. The first is system prompts and tool schemas the harness must send at the start of every session. On this hardware the laptop read about 90 tokens per second at prefill, so every 1,000 tokens of prompt turned into about 11 seconds of watching the cursor. The pi harness sent 2,008 tokens of combined system prompt and tool schema for Qwen 3.8 27B. Opencode sent 18,046. On a datacenter GPU with prefill in the 10,000 tokens-per-second range the gap is 0.2 seconds against 1.8 seconds. On this laptop it was 22 seconds against 226.

The second problem is context that is already spent before the model does any real work. With 32,000 tokens of usable context on this setup, pi's system prompt uses about 6 percent and leaves 94 percent for the task. Opencode's 18,046 tokens leaves 44 percent.

Three groups on this hardware

Sutton sorts the nine harnesses into three groups by how they behave on his laptop.

Lean and stable: pi, mini-swe-agent, and chad, all with trim system prompts and 96 to 99 percent cache reuse across turns. mini-swe-agent passed 11 of 24 Exercism tasks and had the most timeouts, so lean does not mean best at the task.

Heavy but disciplined: dsh, cline, codex, and goose. Their system prompts and tool schemas are long, but the prefix is byte-stable, so once the first turn finishes the cache carries the rest. The author flags that goose only joined this group at version 1.50.0, since earlier releases put a minute-resolution timestamp in the first user message and dropped cache reuse to 78 percent.

Heavy to start: crush and opencode. You wait 3 to 4 minutes before any activity.

What chad reports for itself

The two rows the author separates are chad's in-process MLX engine, where there is no server to observe, so the timings come from chad's own prefill trace. On the same tasks and prompts the in-process engine went from 7.9 to 17.4 experienced tokens per second, or 12.4 with the speculative-decoding drafter off. Against its own control, the DFlash2 drafter read 23.3 tokens per second versus 15.9, a pooled speedup of 1.47 times, and won on all eight tasks.

The read to take from this is that a harness with a small system prompt and a byte-stable prefix is a bigger lever than the model choice for anyone routing a coding agent to a local endpoint. The number a team should ask a harness vendor for is not "tokens per second on our stack" but the token count of the base prompt plus tool schemas the harness sends every turn.

Source

• Nine coding harnesses vs. your laptop, N. Sutton
• Hacker News discussion