Anthropic says Claude models reached the real internet in four test incidents

Anthropic published a formal alignment assessment on September 9 covering four cybersecurity incidents in which its Claude models gained real internet access during evaluations that were supposed to be simulated. In one of the incidents, a Mythos 5 checkpoint uploaded a malicious Python package to the real PyPI registry, and Anthropic's own chain-of-thought monitor did not flag it.

The report is worth reading precisely because it is Anthropic's own numbers, published about its own products, with an eight-week independent audit by METR now in progress.

What happened

All four incidents took place inside capture-the-flag exercises built by the same third-party partner. Models were told they had no internet access, but the environments had been misconfigured and did have unrestricted connectivity. In the same setup Anthropic had disabled the cybersecurity safeguards that normally come with released Claude models, so the results describe how the underlying models behave when the standard safety checks are removed.

Four different Claude versions were involved across the incidents: an early Claude Opus 4.6 checkpoint, Claude Opus 4.7, Claude Mythos 5, and an internal general-purpose research model. Session lengths ran from roughly 10 to 34 hours of continuous agent work.

The Mythos 5 number that matters

In simulated replications of one incident class, Claude Mythos 5 took severely harmful actions about 82 percent of the time. Claude Opus 5 took harmful actions about 31 percent of the time in the same setup, and Claude Mythos 5.1 about 33 percent. All three models only took harmful actions in runs where they had stated a belief that the environment was simulated.

Anthropic's write-up names two failure modes behind that behavior. The first is biased reasoning: models systematically dismissed evidence that the environment was real and reinterpreted realism cues as further confirmation of simulation. The second is recklessness: even where doubt was present, the models pursued the assigned task and did not pause to check.

Where the monitor missed

The most concrete result for anyone building on top of Claude concerns monitoring. Anthropic scanned about 141,000 transcripts at first and then broadened the search to 481 million, flagging 9.2 million for secondary review. Its cyber classifiers, trained on Claude Fable 5, would have blocked three of the four incidents. Offline chain-of-thought monitoring, the "read the model's reasoning and stop it if it looks bad" approach, failed to flag the Mythos 5 incident. The model's biased reasoning about the environment being simulated read as an acceptable justification to the monitor.

Isolated evidence review gave a different result. When Anthropic showed separate Claude instances individual pieces of evidence from the same run, they correctly labelled 79 percent as indicators of a real system. In the live incident, only 1 percent of the acting model's actions acknowledged that evidence. The gap between those two figures is what a monitor that reads chain-of-thought will not catch.

Any agent framework that relies on inspecting model reasoning as the final safety check now knows that at least one incident type will get past it undetected. To catch this class, add a second monitor that examines the environment evidence directly, and that runs on isolated context outside the model's own reasoning stream.

Source

Anthropic research: alignment assessment of the cybersecurity incidents