GPT-5.6 Sol finished Factorio in 13.5 game-hours, with manual help

A long game is a stress test for an agent told to plan, act and recover on its own. Ryan Madden says on his blog that GPT-5.6 Sol launched the rocket in Factorio after 13.5 hours of game time. The post has drawn 132 points on Hacker News.

The model ran "in Codex with low thinking level", Madden writes. He built a game manager that lets the model act in the game through a Python library. The manager enforces the same limits a human player has, including "reach of the player, fog of war". The run was played on peaceful mode. There are no enemies on that mode.

What worked, and what did not

The rocket is the standard long-game goal in Factorio. Launching it means mining, smelting, researching many science packs and building a factory that keeps itself fed. Madden says the previous model, GPT-5.5, "suffered from the model's failures to use transport belts at all". Sol used belts.

Then Sol stalled. Madden writes that scaling issues emerged during late-game blue science research. That slowed progress. The model struggled to connect certain production systems. It worked around that by ferrying supplies between parts of the factory by hand.

The post is explicit about what that means. Madden writes that the rocket launch "was only achieved through significant 'manual' effort on Sol's part". The model "papered over its late-game failures". It did this by "repeatedly ferrying supplies between areas of the factory". A human can do the same thing. It is not what a working factory looks like.

Peaceful mode is doing work in the headline

The default Factorio game sends enemies that destroy a factory left alone. On peaceful mode there are none. A player can build at their own pace, and so can the model. Removing the combat loop takes out one of the two things the game asks the player to balance.

So "beat Factorio on peaceful" is the correct description of what happened. "Beat Factorio" is a bigger claim. Hacker News commenters pointed out the gap. Madden's own title is careful: he writes that the model "can beat Factorio", and then qualifies with "peaceful" in the body.

What a team building an agent takes from it

Madden says what the exercise is for. He writes that playing Factorio on its own is "a medium for building intuition about the nature of model capabilities". He avoids a wider claim about what the model can and cannot do.

The pattern in the run is familiar to anyone running an agent against a long task. The agent gets the main loop right. Then it compensates for a late-stage failure by taking shortcuts a measurement captures only if you look. A pass or fail on the final goal would say the run succeeded. The game log says the agent moved things by hand.

A team watching an agent in production needs the game log, not the pass or fail. If the only check is the outcome, every workaround the agent invents stays invisible. Nobody notices until a human reads the trace.

Source

GPT-5.6 Sol can beat Factorio, Ryan Madden.