METR says Anthropic is improving its models at twice the historical rate

An independent group called METR says Anthropic is now improving its own models faster than before. The new pace is about twice the old one. METR published the finding on 5 November. It ran an 11-week assessment inside Anthropic from 12 August to 29 October 2026. METR is a research non-profit that studies AI model capabilities. The Verge covered the report and called the findings "alarming" to researchers.

The headline figure. Anthropic is improving its models on clear tasks at about 2x the historical rate as of October 2026. The measurement is on an internal benchmark called AECI. The range METR reports is 1.5x to 2.8x at 95 percent confidence. The old trend was about 15 AECI points a year. The new trend is about 29 points a year. About a year of past progress now happens in six months.

How METR measured it

METR describes its method in the report. Four staff spent an 11-week window inside Anthropic. They had access to internal benchmark scores for unreleased models. They had training compute metrics across different areas. They ran around 15 staff interviews across the R&D pipeline. They used piecewise linear regression with breakpoint detection and bootstrap confidence intervals.

METR also used a self-reported survey. Anthropic employees reported their own speedup from AI. The figure grew from about 1.2x in January 2025 to about 12x in October 2026. METR says the interview consensus centres on a 2x speedup from AI. METR estimates about 80 percent of the current acceleration is due to AI itself, with substantial uncertainty.

What METR could not say

METR states its limits in the report. Most of the evidence most informative for its conclusions is redacted in the public version. Anthropic kept the right to remove confidential information that could help other AI developers. Detailed per-pipeline acceleration estimates are redacted. Bottleneck analyses are partly redacted.

METR also lists things that could be wrong. Redacted evidence is sometimes hard to interpret. Its understanding of bottlenecks changed several times during the assessment. The AECI benchmark may overstate real-world impact if models hit capability thresholds unevenly. A compute-allocation check was cancelled mid-assessment.

What this means for a buyer of model work

Any team planning a product on a specific Anthropic model now has a published estimate of model pace. METR says it found no clear limits in the next 12 months. The four checked limits were algorithmic progress, training compute, experiment compute and data.

A plan that assumes a 12-month model cycle is working against this estimate. A plan that assumes a six-month one is working against the uncertainty range. The 95 percent range covers both.

The eval story on the page is also useful. METR chose to publish the uncertainty range, the methodology and the redacted pieces together. That is a working template for an outside audit of a lab's claim. Teams that want their own eval reviewed by an outside group now have one example to copy.

Source

An independent assessment of AI R&D acceleration at Anthropic, by Tom Cunningham, Nate Rush, Parker Whitfill and Hjalmar Wijk at METR. Reported by The Verge.