CodeRabbit measured GPT-6 Astra on code review and found a small accuracy gain at 2.5 times the cost

CodeRabbit published an evaluation on 4 September comparing GPT-6 Astra against five other models on code review. The headline result is a gain that is real and small, next to a cost difference that is neither.

The write-up is by Erik Thorelli and Erfan Al-Hossami. CodeRabbit sells an AI code review product, so this is a vendor measuring models it might route to, and the numbers below are theirs.

The results

On overall actionable bug coverage, they report Astra at 61.3 percent, GPT-5.6 Sol at 59.0 percent, and Opus 5 at 50.2 percent. They also compared against GPT-5.6 Terra, GPT-5.6 Luna and Claude Fable 5.1.

The wider gap is on cross-file review, where a bug only shows up if the model connects code in one file to code in another. There Astra reached 57.1 percent, Sol 47.6 percent, and Opus 5 42.9 percent. CodeRabbit describes that as 20 percent better than Sol and 33 percent better than Opus 5 in relative terms.

They do not say how many pull requests the evaluation ran on, which limits how much weight the two-point overall gap can carry.

The price

Astra is listed at 10 dollars per million input tokens and 50 dollars per million output tokens. CodeRabbit works a sample task of 100,000 input and 10,000 output tokens: 1.50 dollars on Astra against 0.60 dollars on Sol.

Across the set they put Astra at about 2.5 times the cost of Sol, about 4.7 times Terra, and about 47 times Luna.

Data retention

The post states that OpenAI supports zero data retention on Astra for eligible API customers, and that Anthropic's Fable defaults to 30 days of retention with zero-retention available to eligible customers. For any team whose code cannot sit on a third-party system, that eligibility question comes before the accuracy comparison.

Where the gap is worth paying for

The authors state their own caveat plainly: the results describe one part of review performance, do not establish an overall ranking of review quality, do not predict a team's defect rate, and do not promise the same gain on every pull request. They recommend testing against your own workflow rather than reading capability scores as return on investment.

Taken at face value, the shape of the finding is more useful than the winner. The overall gap is 2.3 points. The cross-file gap is 9.5 points. Cross-file bugs are also the ones human reviewers miss most often, because seeing them requires holding two files in your head at once.

That suggests a split rather than a single choice: the cheaper model on small, contained diffs, the expensive one where a change crosses module boundaries. A team already running automated review can make that split on diff shape, which is information it has before the review starts. Paying 2.5 times on every pull request to gain two points overall is a worse trade than paying it on the tenth of pull requests where the gain is four times larger.

Source

• GPT-6 Astra in code review: gains, privacy, and cost, CodeRabbit, 4 September 2026