CodeRabbit measured GPT-6 Astra catching 61.3 percent of labelled bugs in code review, at 2.5 times the token price of Sol

CodeRabbit published an evaluation of OpenAI's GPT-6 Astra for code review on 4 September, written by Erik Thorelli and Erfan Al-Hossami. The company sells code review tooling, so this is a vendor measuring a model it may resell, and the numbers below are its own.

The measure CodeRabbit uses is actionable bug coverage: how many bugs from a labelled set the model catches through findings a developer can act on. On the full set, it reports Astra at 61.3 percent, GPT-5.6 Sol at 59.0 percent and Opus 5 at 50.2 percent. The company describes the overall gain over Sol as modest, about 4 percent in relative terms.

The gap is in the cross-file subset

The more interesting split is the harder one. On reviews where the bug spans more than a single file, CodeRabbit reports Astra at 57.1 percent, Sol at 47.6 percent and Opus 5 at 42.9 percent. In relative terms that is 20 percent over Sol and 33 percent over Opus 5.

CodeRabbit's own reading is that the advance is in connecting information that sits in different places, rather than in having room to hold more of it. The post is careful about what this does not show: it says the results do not establish an overall ranking of review quality, do not predict any team's defect rate, and do not promise the same gain on every pull request. It also notes the full set includes simpler reviews where a stronger model has less room to pull ahead, which is part of why the headline gap looks small.

What the coverage costs

The pricing section is where the trade-off gets concrete. CodeRabbit lists Astra at $10 per million input tokens and $50 per million output, checked against published rates on 4 September, and puts Claude Fable 5.1 at the same base rates with different caching prices. GPT-5.6 Sol is $4 and $20, Terra $2 and $12, Luna $0.20 and $1.20.

Holding usage fixed at 100,000 uncached input tokens and 10,000 billable output tokens, the company works those rates into an illustrative task cost: $1.50 for Astra, $1.50 for Fable 5.1, $0.60 for Sol, $0.32 for Terra and $0.032 for Luna. That puts Astra at 2.5 times Sol and roughly 47 times Luna on the same token count. The example deliberately excludes caching, cache writes, tools, retries and regional uplifts, and CodeRabbit says Sol's promotional pricing runs at least through 21 November 2026.

The company then argues against reading its own table too literally. A model that needs fewer tokens or fewer attempts narrows the gap, and CodeRabbit notes OpenAI reports lower estimated task costs for Astra in some of its own evaluations despite the higher per-token price. That is OpenAI's claim about OpenAI's model, and it has not been independently checked here.

On data handling, the post says Astra supports zero data retention for eligible API customers, pointing at OpenAI's data controls documentation for the eligibility rules.

The shape of the result is a routing question rather than a switching one. A 9.5 point spread on cross-file reviews and roughly one point overall, at 2.5 times the token price, argues for sending the model the changes where evidence is scattered and leaving the routine ones with something cheaper. Measuring cost per bug actually caught on your own pull requests will settle that faster than either the coverage number or the price list will.

Source

CodeRabbit: GPT-6 Astra in code review: Gains, privacy, and cost.