ARC Prize found GPT-6 Astra scores 17.5 to 99.9 percent on ARC-AGI-3

ARC Prize on September 3 published a run of OpenAI's new GPT-6 Astra model on ARC-AGI-3, its agentic-intelligence benchmark. The main result is that Astra sets a new best score on the benchmark. A second result is that the same model produces widely different scores depending on the harness and reasoning level it runs inside.

The numbers

Two harnesses were tested. ARC Prize calls its Standard harness a setup that lets the model carry forward notes it chooses to keep as it moves through the environment. Its Provider Adapter harness preserves opaque reasoning state between requests and uses compaction for longer conversations, so the model can reuse prior work.

On the Semi-Private set, GPT-6 Astra at maximum reasoning effort scored 62.7 percent for $26,098 with the Standard harness. The same model on the Provider Adapter harness at "high" reasoning scored 99.9 percent for $18,817. Across the six reasoning-effort levels ARC Prize tested, Standard-harness scores ranged from 17.5 percent (low, at $38,166) to 62.7 percent (max, at $26,098). Provider Adapter scores ranged from 96.7 percent (none, at $23,457) to 99.9 percent (high, at $18,817). Higher reasoning effort tended to cost less at the top, because Astra solved games in fewer actions and used fewer model calls and tokens.

Action efficiency was the other measured claim. ARC Prize reports Astra used fewer actions than the median tested human on 96 percent of levels. In the replays, the team observed the model turning unfamiliar game mechanics into a compact symbolic world model, representing rules as logic and developing its own shorthand to track state and plan actions.

The human cost is useful for comparison. In controlled testing, human participants were paid $115 per 90-minute session plus $5 per game completed, and attempted roughly nine games per session, working out to about $12.78 per attempted game before bonuses. ARC Prize notes that most of this pays for the participant's time rather than the energy their brain uses; priced as electricity, the estimate drops to about 0.067 cents per game.

Keep the harness the same before comparing models

The useful finding is the range across harnesses. On this benchmark, the same model at the same reasoning effort produces very different numbers depending on the harness around it, and the harness that let it reach 99.9 percent also ran for less money than the one that stopped at 62.7 percent. Anyone reading a single-number benchmark headline is reading one point out of a range that the surrounding code decides.

For a team choosing an agent stack, the practical point is that model comparisons are only comparable when the harness is held constant, and that harness engineering (how state is carried, how long conversations are compacted, how reasoning effort is set) matters as much as the model choice in evaluation. If an internal benchmark compares two models on two different harnesses, the result tells you almost nothing. Keep the harness the same first, then compare, then decide.

Source

• OpenAI's GPT-6 Astra on ARC-AGI-3, Greg Kamradt, ARC Prize, September 3, 2026.