AI NewsProductivityAnnouncement
Anthropic grades Claude's own code with a 39-item checklist and a grader
Anthropic's Claude Code team published its verification playbook: a 39-item checklist, three test types, and a separate grader that lifted task success by up to 10 points.

Why it mattersA team writing code with Claude now has a worked example to copy. Checking the work is the slow part, not writing it.
Writing the code is the fast part of AI-native work. Checking what the agent wrote is where teams get stuck. Anthropic has published how its own Claude Code team runs that check. The post is by Addy Osmani on claude.dev, Anthropic's developer blog.
The headline figure is a point of calibration. Anthropic says about 90 percent of the Claude Code team's own code is written and reviewed by Claude. Anthropic also ran a separate test on structured file generation. A second grader lifted task success by up to 10 points over a standard prompt. Both numbers are the team's own.
The three tests it leans on
End-to-end tests come first. The post argues they fail with a screenshot and a description of the broken step. A coding agent can act on that. A unit test failure often lands as a short trace, which the agent then guesses at.
Property tests cover inputs a human reviewer would not write out. The team uses fast-check for JavaScript, Hypothesis for Python, and proptest for Rust. These check invariants like round-trip encoding and idempotency. An agent breaks that kind of rule silently.
Differential tests are for migrations. You run the old system and the new one on the same inputs. You diff the outputs. Anything that moves is a change to account for. Anthropic says this is how its team catches breaking changes a line-by-line review would miss.
The checklist, the skill and the review file
The reliability checklist has 39 items. Each one has a diff-readable trigger, so the checker runs only the items a given change could break. Anthropic says a typical change fires three to ten of them. After three rounds of a coding agent failing to fix the same item, the change goes to a person. Commits cannot change their own checks.
A verify skill lives in .claude/skills/ and lets Claude test the change end-to-end before opening a pull request. Code review reads a REVIEW.md from the base branch, never from the pull request itself. A change cannot rewrite the rules it will be judged by. Review findings are tagged as Important, Nit or Pre-existing. When the same bug pattern turns up in two or more pull requests, Anthropic says it writes a new rule. That saves the reviewer from catching it next time.
The Hacker News thread on the post was at 65 points when checked. Readers are split on cost. One commenter said Playwright would run the same end-to-end checks for the price of CI minutes. A supporter put a number on it: 500 dollars of grading tokens is 6,000 dollars a month. That is a small share of what a team is already spending on coding tokens. Anthropic does not price the approach on the page. The sceptics have a fair point, because Anthropic sells the tokens the grader burns.
For a team building anything with AI, the piece gives numbers a reader can measure their own setup against. Three rounds of agent retries before a human reviews the change. 39 checklist items, each with a rule for when it runs. Three test types with named libraries for each language.
Source
How the Claude Code team thinks about verification, Addy Osmani, Anthropic.
This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.
Get AI News in your inbox
New developer tools, model and agent releases, and how teams are actually using them to release software. Short, and only when there is something worth reading.

