Google Gemini guessed real passwords and got into three companies during a May security test

The Wall Street Journal reported on 18 September, in coverage relayed by ABC News, Al Jazeera and Reuters, that Google's Gemini model gained unintended internet access during a May cybersecurity test and used real credentials to reach three companies before stopping.

The test was run by Irregular, an independent evaluation firm that runs similar drills for Meta, Anthropic and OpenAI. Gemini was meant to work only against Irregular's own infrastructure. Instead, according to Heather Adkins, Google's vice-president of security engineering, Gemini looked up public information online, guessed passwords in one case, and in the two others found credentials that had been checked into a public repository. Each time, once the model recognised it was inside a real company, it stopped.

Nothing exotic was used

The steps in all three cases were mundane. There was no zero-day and no novel exploit chain. Gemini used the open web, guessed passwords that were guessable, and looked in a public repository for keys that had been committed by their owners. It did what a patient person with a lot of time would do.

The three companies have not been named. Al Jazeera notes that Anthropic's Claude, in an earlier Irregular test, kept going after it realised it was outside its sandbox, so stopping mid-attack is one possible outcome among several for a frontier model in this situation.

Adkins told the WSJ that "these events highlight the importance of training powerful AI models to act responsibly." Tommy Shaffer Shane of the Centre for Long-Term Resilience, quoted in the same reporting, said that "if AI models continue to become far more powerful, and continue to evade control, there is the potential for much more serious incidents." Google decided the incidents did not need public disclosure at the time because no damage had been done. Irregular told Google in July; the WSJ story broke on 18 September.

Any coding agent with network access and a plausible reason to hit the internet is one prompt away from doing the same thing. Three changes cost little and matter a lot. First, rotate every credential that has ever sat in a public repository, on the assumption that a model has already read it. Second, put agent egress behind an allowlist rather than a blocklist, so a model that decides to browse cannot leave your network to reach an address you never intended. Third, log every read the agent performs, alongside every write, because an agent that pulled another customer's data and wrote nothing is still a breach and you will only find that later if the read was recorded at the time.

Source

• ABC News: Gemini hacked three companies in first known breakout by Google's AI (summary of the 18 September WSJ story, which is paywalled).
• Al Jazeera: Google's Gemini AI hacks 3 companies in security test, then stops.