Nathan Barry solves Astra's weight edits with a linear program

An AI model that reads and rewrites the weights of a small network has been a strong result this year. The method cost eight hours of a frontier model's time per fix. Nathan Barry says most of that time was wasted.

Barry posted his result on 9 November. The post reached 237 points on Hacker News. He builds on the Weight-Space Programming paper from MIT CSAIL. In that paper, a frontier model called Astra edited 0.016 percent of Qwen3.5-0.8B's weights. The edit lifted MATH-500 from 7.0 percent to 50.9 percent. Barry asks a direct question. Was Astra solving a problem a linear program could solve?

What the Astra fix was doing

The Qwen3.5-0.8B base model loops on hard questions. Barry says the token stream keeps pushing the model's internal state in one fixed direction. The model never reaches an answer. Astra's fix shifts the weights of certain columns so that direction gets cancelled out. Astra searched for the right columns by trial and error. It changed 130,188 weights across the network. It ran for about eight hours.

Barry rewrote the search as a linear program. The constraints are plain. Push a looping token's state back along the loop direction by at least some amount. Keep each productive token's state within a small bound. Among all edits that meet the constraints, pick the one with the smallest total change. A linear program picks few columns to edit. The smallest total change puts zero weight on columns it does not need.

The numbers

Barry's LP ran on a GPU. It found a solution within 0.2 percent of optimal in about one minute. The reference open-source solver took over 20 minutes. He ran the edit against the same benchmarks the MIT post used. He kept Qwen3.5-0.8B and the same sampling settings. The LP edit changed 16,291 weights, eight times fewer than Astra, and matched the result.

Benchmark	Base	Astra edit	LP edit
MATH-500	7.0 percent	50.9 percent	51.7 percent
GSM8K	14.1 percent	68.2 percent	69.6 percent
IFEval	22.9 percent	61.9 percent	59.7 percent

Barry also tested a simpler steering vector. He added the LP edit's average push along the loop direction as one fixed shift. He recovered almost the same scores. He reads that as evidence the fix is nearly token-uniform. The specific columns Astra chose carried less information than the direction they pointed in.

What this says for small-model tuning

A team running a small open-weights model can now test one thing. Does a frustrating failure mode come from one residual direction? Barry's code finds the edit in a minute on a single GPU, using open tools. Astra needed a frontier model and eight hours per run. The LP version has limits he states on the page. It only covers edits of a single rank-one shape. A human still has to identify the loop direction to cancel.

Source

Can a Linear Program Do Weight-Space Programming? by Nathan Barry. The underlying MIT CSAIL work is Weight-Space Programming by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab.