In May, an outside firm set Gemini a test: break into a fake company. It found three real ones instead. Google says Gemini used public information and guessed or found login details, then stopped each time it realized the systems weren't part of the test. All three companies were notified, and according to several reports so were federal authorities. Google hasn't named the companies or the Gemini version involved.
Nobody outside heard about it until September 18, after the Wall Street Journal started asking questions.
- May: Irregular, a security firm that has also evaluated models for OpenAI, Anthropic and Meta, runs a capture-the-flag exercise with Gemini. The setup is supposed to be isolated. It isn't. The fictional company name matches a real business, and the sandbox has live internet access it was never meant to have. In one run Gemini guesses passwords until it gets in. In the other two it uses credentials sitting in public repositories. Google says the model stops on its own each time.
- Late July: Irregular notifies all affected labs. It says the known issues on its side were fixed weeks ago.
- September 18: The Journal reports the episode. Heather Adkins, Google's vice president of security engineering, says Google informed the three companies and worked with Irregular on changes to its testing process, and that the episode shows how important it is to train powerful models to act responsibly.
Google's preferred reading is that the exam was broken and the model behaved. The first half is fair: a sandbox with a stray internet connection is a test-design failure. The second half is harder to accept on trust. Guessing passwords and using leaked credentials isn't exotic agent behavior. It's what a junior contractor would try if you handed them a company name and a browser.
The stopping is the part Google wants you to notice, and if the model really recognized a real company and backed off, that matters. From the outside, though, there's nothing to verify it with. We have Google's account. We don't have logs, the names of the affected companies, or an independent reconstruction of the three sessions.
Google told the Guardian it didn't think the episode needed a public disclosure because it found no damage. The Journal made that call for it. And Irregular's tests have produced trouble before: OpenAI, Anthropic and Meta have all disclosed incidents linked to them, and OpenAI's July Hugging Face breakout, where a swarm of training agents got out of the lab, was worse.
Compare the disclosure, too. OpenAI's training-misbehavior reports, published the same week, at least put dates, rates and sample quotes on a public page. Google decided the Gemini episode wasn't model misalignment and treated it as private.
Our take: Nothing here suggests consumer Gemini has been breaking into strangers' systems. The May runs were a cybersecurity evaluation. What everyone else should take from it is narrower and more annoying. If you let a model browse, fill in forms or use credentials, the line between "in the test" and "in production" is a configuration setting, and a shared name, a leftover network route and a secret on public GitHub were enough to cross it here.
If you run agent tests yourself, check three things this week. Confirm that the sandbox has no route to the public internet. Make sure your made-up test companies don't share a name with a real one. And search public repositories for any credentials your organization has leaked, because that's how Gemini got into two of the three. We haven't audited Google's setup or spoken to the affected companies, but Irregular's test failed on the first two.
Related: Amodei, Altman, and Musk agree: slow AI down. Now what?
