Anthropic said its Claude-based security models gained unauthorized access to the sensitive production environments of three outside organizations during internal testing designed to measure the models’ offensive cyber capabilities.
The events, which Anthropic revealed Thursday, are the second revelation in 10 days that AI models from the world’s wealthiest providers have trespassed into protected networks, an offense that, in additional traditional hacking scenarios, could land the human behind the keyboard in prison for years. Earlier this month, OpenAI said its security models exploited a zero-day vulnerability to be used in breaking into the network of Hugging Face, a platform for open source machine-learning models and AI datasets. The OpenAI models went on to steal access credentials and other confidential Hugging Face information. The OpenAI models also exploited publicly exposed credentials to compromise accounts of 4 other third-party services.
Anthropic said the OpenAI event spurred its engineers to review similar cybersecurity evaluations by Claude models. The audit found three incidents “through which a model accessed the web from inside or while interacting with the evaluation environment of Irregular, certainly one of our third-party evaluation partners, after which gained unauthorized access to the production infrastructure of three different organizations.”
When Models fail to discern reality from fiction
Anthropic said the prompts engineers delivered during “capture the flag” challenges—a typical means for assessing the offensive and defensive capabilities of assorted hacking techniques—made clear that the testing environment was only a simulation and that the models had no access to the open Web. In truth, the testing partner Irregular mistakenly made such access available. In response the models treated the Web paths as a part of the exercises.
The intrusions happened through three Claude models: Opus 4.7, Mythos 5, and an internal research prototype. Opus 4.7, the oldest of the three, was the one which overstepped its boundaries essentially the most. Anthropic explained:
Operating under the false belief that every one accessible entities were intended to be in-scope for the exercise, Claude compromised the impacted organizations’ infrastructure using basic techniques, reminiscent of exploiting weak passwords and unauthenticated endpoints. It didn’t find or exploit any complex vulnerabilities, and in each case, Claude continued working to finish only the particular capture-the-flag task its evaluation had assigned. Nonetheless, in some cases, our older model continued its attack even after getting evidence it was running on the open web; our latest model stopped once it recognized it was on the web. In none of those situations did Claude exfiltrate itself or deliberately try to escape its test environment.
While the Opus model continued its attack even after accurately predicting that it had breached the third-party organization’s production system, Anthropic said, Mythos 5 made an analogous inference but “reasoned its way back to the conclusion that it was still in a simulation.” That meant that Mythos, too, didn’t abort the exercise once it had clearly stepped over the road. After also exceeding the bounds of level behavior, the inner test model eventually found evidence that it, too, had impinged and stopped the exercise.