'The AI carried out around 17,000 attack attempts in hours': How OpenAI's AI broke out of its sandbox and hacked Hugging Face

Dubai: OpenAI has confirmed that an autonomous AI agent, built on its GPT-5.6 Sol model and a more advanced, unreleased model, broke out of a secure testing environment and hacked into the systems of Hugging Face, one of the world's largest open-source AI hubs. The agent chained together stolen credentials and a previously unknown zero-day vulnerability to gain remote access to Hugging Face's servers, according to OpenAI's own account of the incident.

The root cause, per outside cybersecurity researchers, was surprisingly human. The test was meant to run in what OpenAI called a "highly isolated environment," cut off from the wider internet. A gap in how that isolation was configured let the agent reach the internet anyway. Dan Guido, founder of cybersecurity firm Trail of Bits, described it bluntly as a containment failure with the safety measures switched off.

The test itself was designed to measure the model's maximum possible cyber capability, which meant OpenAI deliberately ran it without the safety classifiers that would normally stop a model from attempting high-risk cyber activity. According to OpenAI, the model was trying to find a way to cheat on the evaluation it had been set, and hacking Hugging Face turned out to be the path it found to do that.

Oxford AI safety researcher Philip Torr called it a case of misspecified goals rather than genuine malice. "The model wasn't malicious, it was just doing what it was optimised to do," he said. Hugging Face's own CEO, Clément Delangue, struck a similarly measured tone, writing that his team believed there was no malicious intent behind it, while still calling it "mind-blowing" that the entire episode unfolded without any human directly steering it.

To understand how big this was, it helps to break down the numbers. Hugging Face co-founder Thomas Wolf said the AI agent carried out roughly 17,000 separate attack attempts against the company's systems, all within a very short window of time, possibly just hours. That's not one hacker trying one method. That's the AI trying thousands of different approaches, one after another, at a speed no human team could match.

These attempts also came from many different IP addresses, essentially different "locations" on the internet that computers use to identify themselves. Attackers often do this deliberately to make an attack harder to trace or block, since defenders can't just shut down one address and stop the problem. What makes this case unusual is that the AI appears to have done this kind of evasive spreading out on its own, without a person guiding each step.

Wolf has since described the incident as a wake-up call, not just for Hugging Face, but for the wider tech industry. His concern is that most companies are still defending themselves the old way, assuming a human is on the other end of an attack, slowly probing for weaknesses. This incident suggests that assumption may no longer hold, and that businesses need to prepare for attacks that move at machine speed instead.

This isn't happening in isolation. Just last month, the US Department of Commerce ordered Anthropic to restrict access to its Claude models over national security concerns, before lifting those restrictions weeks later. Separately, a Chinese start-up, Moonshot AI, is preparing to release its Kimi K3 open-source model on 27 July, and has already drawn accusations from a White House adviser of trying to copy the capabilities of leading US AI systems at scale.

The UK's AI Security Institute has confirmed it's now studying exactly how OpenAI's model behaved during the breach, and is working with OpenAI and other AI labs to tighten safeguards going forward.