MIT paper says an LLM can read and rewrite neural network weights

Four researchers at MIT CSAIL say a frontier model can read and edit neural network weights. They say it does so the way a programmer reads code. Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab call this Weight-Space Programming. They posted their results on 2 November. The post reached 665 points on Hacker News within a few hours.

The authors ran a frontier model, which they call Astra, over the raw weights of several small networks. They did not let it run the network or train it. They asked it to describe the network, fix a broken one, or write a new one from scratch.

What Astra could read

In the first test, Astra named which of four grid games each of four policies was trained to play. All four networks had the same shape and inputs. The only signal was the 17,253 weights. Astra got all four right.

In the next test, Astra named what each of three diffusion models drew: circles, squares or triangles. The authors say it ignored forward passes. It read the shape straight from how the output layer's weights lined up.

In a third test, the authors stripped the first and last layers from three small language models. The remaining four transformer blocks held no tokeniser. Astra named each language, Python code, English or Japanese romaji, in both of its attempts.

What Astra could change

In the next test, Astra wrote weights from scratch for a game-playing network. The authors say it produced policies that beat networks trained with PPO for 100 million steps. No training was done. The weights were edited by text.

The strongest result is on an open-weights model. Astra was asked to improve Qwen3.5-0.8B at maths without a gradient. The authors say it found an issue in the weights that caused reasoning loops. Qwen's own model card warns about this.

Astra then edited 0.016 percent of the weights. The authors say MATH-500 climbed from 7.9 percent to 48.1 percent at the same sampling settings. The same edit improved instruction following too. IFEval tripled.

Why to read the limits

For a team fine-tuning a small model, this is one lab's result and a short one. The networks are small. The language-model edit is on a 0.8-billion-parameter open-weights model. The authors call it "sparks of" Weight-Space Programming, meaning early signs rather than a settled method.

For an interpretability team, the point to notice is that Astra reports its reasoning in natural language. The authors say it writes something like a program that explains what each layer does. Then it changes a specific set of numbers.

The result stays scoped: one carefully guided session edited a smaller model with a known flaw. The authors flag that scale is open. A team that cares about this should read the full post before copying the method.

Source

Rikiya Takehi and co-authors at MIT CSAIL on Weight-Space Programming. Hacker News discussion.