AI NewsModels & agentsReported
Claude Fable 5.2 predicts obscure protein sequences at 54 percent
Xinming Tu tested 22 language models on little-studied bacterial proteins. Claude Fable 5.2 predicted the next amino acid 54 percent of the time. The best open protein model got 43.7 percent. Earlier Claude versions scored under 10 percent.

Why it mattersA model that quietly learned a new biology skill is a model that quietly learned something else too. The method here works for any field a buyer wants to check.
Independent researcher Xinming Tu, with a collaborator Leo Tianlai Chen, published a test on 22 language models over the weekend. Tu's post was first posted on 17 November and updated 18 November. The test asks a model to predict the next amino acid in a protein that almost nobody has studied. It is a direct way to check whether a model saw protein sequences during training. The data, the code and the test set are on GitHub under the author's name.
The headline result is that Claude Fable 5.2 was right 54 percent of the time on real proteins. The same model was right 9 percent on the same proteins after the letters were shuffled. The 45 point gap is what Tu reads as sequence learning. A model that only knew letter frequency would score the same on both. Claude Opus 5.5 scored 34.9 percent on real sequences and 7.5 percent shuffled.
How the test is built
The test set has 668 proteins from 167 Pfam families of unknown function. All are from bacteria or archaea. None of the families has a solved structure or more than three papers in the literature. Each protein was cut at five random points. Tu scored only the first amino acid the model produced after the cut. Each protein was also shuffled with the same letters in random order. The shuffled score is the baseline. The gap between real and shuffled is the signal.
Tu then ran the same test on two specialised protein language models. ProGen3-3B scored 43.7 percent with a 32.5 point gap. ProtGPT3-10B scored 38.4 percent with a 27.7 point gap. Claude Fable 5.2 beat both. Opus 5.5 was level with the larger ProtGPT3-10B. GPT and Gemini models in the test showed near zero gap.
Earlier Claude versions did not do this
Older Claude models had small gaps on the same test. Opus 4.5 was plus 1.9 points. Opus 4.6 was plus 0.9 points. Opus 5 was plus 9.6 points. Fable 5.1 was plus 14.9 points. Only Fable 5.2 (plus 45.1) and Opus 5.5 (plus 27.4) show a large gap. Tu notes that Sonnet 4.5 refused 97 percent of the test prompts and was dropped from the comparison.
Two tests rule out the obvious alternatives. First, performance scaled with Pfam family size, in the way a protein language model's performance does. Second, the result held in 128 families with no published papers at all. So the model has learned sequence patterns past what any review paper would carry. Tu also tested 56 proteins first published in August, after any plausible training cut-off, and the gap stayed. Tu ends with: "The latest Claude models know protein families that almost nobody has studied. That takes a lot of sequence data, not a few examples: protein sequences were a sizeable part of their training." Anthropic has not commented.
Why this matters past biology
Anthropic has published its own work using Claude for protein design, including a paper on scorpion toxin binders in October. Tu's piece is not that paper. It is an outside test that measures how much sequence knowledge a general-purpose Claude model carries. The result says more than any press release.
The method is the useful part for a buyer or a team. Find a corpus the model should not know. Make a hole in each example. Score the first token. Shuffle the input and score again. The gap is the signal. The same shape works on code in an unusual language. It works on a private API, or on a corpus the team owns.
Source
This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.
Get AI News in your inbox
New developer tools, model and agent releases, and how teams are actually using them to release software. Short, and only when there is something worth reading.


