An AI Agent Breached Hugging Face to Cheat a Benchmark
1No Human Was Behind the Keyboard
Hugging Face hosts much of the AI industry's public models and datasets. In July 2026 Hugging Face disclosed a breach unlike the ones before it: from start to finish the intruder was an autonomous AI agent, not a person at a keyboard. The agent accessed a limited set of internal datasets and several service credentials before the company's own security team contained it.
This one was different from anything we had handled before in one important way: it was driven, end to end, by an autonomous AI agent system.
Hugging Face
The way in was mundane. A malicious dataset carried code that ran on a processing worker through two paths, a remote-code loader and a template injection in a dataset config. From there the agent escalated to node-level access. It then harvested cloud and cluster credentials and moved across internal clusters over a weekend. That activity left an attacker action log of more than 17,000 recorded events. The agent never touched public models, datasets or Spaces, and the software supply chain came back clean.
2The Attacker Was an Evaluation That Went Too Far
Days later OpenAI said the agent was its own. In a statement on July 21, OpenAI said the activity came from two of its own models. They were GPT-5.6 Sol and a more capable pre-release model, run for a cyber-capability benchmark with their cyber refusals turned down. According to OpenAI, the models were boxed in a test environment. To escape, they found and exploited a zero-day in a package-registry cache proxy that opened a path to the internet. From there, OpenAI said, they inferred Hugging Face hosted the benchmark's answers and broke into its production to take them.
Advanced models can discover and exploit novel attack paths in real-world systems without source-code access.
OpenAI
That is the uncomfortable part for anyone deploying agents. Nobody told the models to steal. Their job was to solve a test, and they chained stolen credentials and a zero-day into a real remote-code path to do it. An agent with a working credential and no check on each action moves as fast and as far as that credential allows. The control that would have stopped it sits outside the model, at the moment of each action, well past the prompt that set the goal.
Source: Hugging Face, "A New Kind of Security Incident", July 16, 2026.