Meng Zhang moved Qwen 3.8's answers 18 points toward GPT-5.5 Pro: Reveneau

Why it mattersA team picking an open model for reasoning work now has an 18.18-point measurement that Qwen 3.8's chain of thought follows GPT-5.5 Pro's, which changes which teacher a distillation pipeline should assume.

Meng Zhang, the founder of Tabby, published a rerun of his reasoning-prefill experiment on 2026-09-09 in a public gist. This version uses GPT-5.5 Pro as the teacher model in place of Anthropic Opus 4.8. The rerun changes the answer for one open model: Qwen 3.8 A95B moves +18.18 percentage points toward GPT-5.5 Pro when it is fed the first one percent of GPT-5.5 Pro's reasoning tokens.

Adding one percent of GPT's reasoning

For each of 45 problems, Zhang generated two responses from each target model. One was an ordinary response with no prefill. The other started with the first one percent of GPT-5.5 Pro's reasoning tokens, inserted into the target model's own reasoning channel. The visible answer was then left to the model. Zhang measured how much of GPT-5.5 Pro's visible answer showed up in the first 100 tokens of the target model's answer, using the mean of unigram, bigram, and trigram source recall. Each delta below is an absolute percentage-point change between the unprefilled and prefilled runs.

Qwen moves 18 points, three other models barely move

Across all 45 problems, Zhang reports: DeepSeek V4 Flash moved −1.17 points, Inkling +0.46, Kimi K3 +4.54, and Qwen 3.8 A95B +18.18. The Qwen result is broken down by category: STEM +26.99, non-STEM +12.80, and synthetic puzzles +14.75. The puzzle set matters because it was private and synthetic, so Zhang says none of the tested models could have seen those problems during training.

Kimi K3 had the highest overlap with GPT-5.5 Pro to begin with, at 31.11 percent unprefilled and 35.65 percent prefilled, but the prefill added only 4.54 points. Kimi already matched GPT closely before any prefill. Qwen 3.8 was at 16.79 percent unprefilled, and only rose once the reasoning channel was prefilled.

Zhang's reading: a GPT model in Qwen's training

Zhang writes that in the earlier experiment Qwen barely moved toward Opus 4.8. In this rerun it moved by +18.18 points toward GPT-5.5 Pro, including a large effect on the private puzzles the models had never seen. He concludes that the data suggest Qwen may have learned from GPT-5.5 Pro, or from a closely related GPT model. He frames this as a suggestion the data will support rather than a proof, and he did not name a specific training corpus.

The experiment is small. Forty-five problems is one person's evaluation with 15 private puzzles inside it, and the two other conditions are 15 STEM and 15 non-STEM problems. The signal on Qwen is large enough that a false positive is unlikely at that scale, and the same test on more open models with more problems would settle the question. Zhang's earlier run and the linked Stolen Thoughts site describe the same technique in more detail.

For a team deciding whether Qwen 3.8 is a direct replacement for reasoning work already tuned around GPT, this measurement reduces the list of likely teachers behind the open weights. If your prompt structure and evaluation harness were built around GPT-style chains of thought, Qwen 3.8's own chain looks closer to that style than to Anthropic's, which is the reverse of what the earlier rerun against Opus 4.8 suggested. The switch matters most in a distillation pipeline where the assumed teacher decides which trajectories the student is asked to imitate.

Source

This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.