Simon Willison's laptop model reads weights to name a game

A 27B open model running on a laptop can tell what task a small neural network was trained to do. It only has to read the network's weights. Simon Willison showed this in a blog post. He was testing a claim from a new research paper. The paper is Sparks of Weight-Space Programming. The authors are Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. They ran their test on GPT-6 Astra.

Simon ran the same four prompts on Qwen 3.8 27B. He used LM Studio and his own LLM command-line tool. Each prompt was the plain-text weights of a reinforcement learning policy. The policy had been trained to play one of four classic grid games. The four were Breakout, Pong, Pac-Man and Freeway.

What the laptop model answered

The model had no tools, no code execution, nothing to run. It had to work only from the floating point numbers in the prompt. Qwen identified Breakout in 52 minutes 25 seconds. Simon says the model gave 88 percent self-reported confidence on that answer. Pong came back in 1 hour 4 minutes 54 seconds at 75 percent. Pac-Man came back in 46 minutes 12 seconds at 72 percent.

Freeway failed. Simon said the model called it a Pac-Man-like grid game. Freeway is a 1981 Atari 2600 game where a chicken crosses a motorway. Simon writes that this is more obscure than the other three and the model probably had less to draw on.

How the test was set up

Each prompt file lives in the GitHub repository that accompanies the paper. Simon used the LLM tool's fragment flag to pull each prompt from a URL and feed it to a model. He ran the Qwen pass on a long context window, because the default 8192 tokens failed. He also had to raise the LM Studio read timeout past the default to let the model finish.

He compared the same test against Kimi K3.1 Pro through OpenRouter. Kimi guessed Breakout in 9 minutes 41 seconds with 19,493 reasoning tokens. Qwen on the laptop used 43,506 reasoning tokens to reach its Pac-Man answer. The traces are long and dense. One sample Simon quoted is almost unreadable without a notepad.

The point Simon makes about the paper

Matthew Honnibal, the author of spaCy, called the paper misleading. He said it mixes two different things. One is what a model can tell from raw weights. The other is what a model with code tools can do to change other models. Sean Goedecke argued the Qwen weight-editing example reproduces steps a human ML researcher would use.

Simon focuses on the first half, the no-tools question, and concludes the paper's wider framing is overblown. He writes that the result is cool but says this is not yet the end of source code. The useful fact for a working engineer is narrower. A 17 GB open model, running at home, can do a hard interpretability task on a small network's weights. It takes about an hour per guess.

Source

Qwen 3.8 27B on my laptop can interpret model weights, too, Simon Willison.

Underlying paper: Sparks of Weight-Space Programming, by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab.