By Raphael Satter, Deepa Seetharaman and Kenrick Cai
WASHINGTON/SAN FRANCISCO, July 24 (Reuters) – The OpenAI agent that broke into tech firm Hugging Face went on a dayslong hacking spree that OpenAI didn’t notice until well after the threat was contained and the FBI was alerted, according to people familiar with the investigation.
The agent – a program capable of making decisions and executing complex tasks with little or no human oversight – attempted to break out of its isolated testing environment at OpenAI around July 9, according to two of the people.
The intrusion at Hugging Face, which operates as a repository for AI tools and models, began two days later on July 11 and lasted until July 13, said Thomas Wolf, Hugging Face’s co-founder.
It took several more days for OpenAI to realize its agent was behind the hack, and the two companies only communicated about it for the first time on or around July 20, according to Wolf and three of the people familiar with the investigation.
OpenAI’s public disclosure, on July 21, that one of its agents had slipped out of control and carried out the break-in at Hugging Face drew global attention. But many details of the hack, including how long the agent went rogue and OpenAI’s belated knowledge of it, are being reported here for the first time.
Hugging Face is preparing a public timeline of the hack, Wolf said, adding that he could not speak to what happened at OpenAI. In a statement, OpenAI said the hack was unprecedented and “marks an important moment for AI safety.” It added that it was reviewing the incident with outside advisers and would eventually publish a technical report.
A spokeswoman said there were “several inaccuracies” in Reuters’ reporting but didn’t respond when asked to describe them.
The FBI declined to comment about the incident.
The incident, which evoked science fiction scenarios about humans losing control of dangerous AI systems, comes at a delicate time for OpenAI, the company behind ChatGPT. Its executives are preparing for a possible initial public offering that could come as soon as this year to help finance the billions needed to fund its growth in years to come.
OpenAI’s loss of control over its AI agent raises new questions about the company’s safety procedures, three cybersecurity experts said.
“Does that mean that they left it unattended and didn’t realize what it was doing? Or maybe they did and didn’t know how to contain it? Both are equally dangerous and alarming,” asked Marley Smith, the principal intelligence specialist at the nonprofit World Ethical Data Foundation.
SIGNS OF TROUBLE?
The episode started while OpenAI was testing the cybersecurity prowess of an agent powered by two of OpenAI’s most advanced models, GPT‑5.6 Sol and an unreleased model OpenAI has described as “even more capable.” By that point, there were already indications of strange behavior from OpenAI’s technology, according to three sources.
In one case, an agent left notes apparently for future versions of itself, according to three people familiar with the matter. The notes, found in a part of OpenAI’s infrastructure, laid out instructions for how agents could free themselves from OpenAI’s internal constraints, the people said. Earlier tests of the models yielded cases in which monitoring systems had been disconnected, one of the people said.
Reuters could not establish if these incidents were linked to the rogue agent that began escaping on July 9 and attacked Hugging Face on July 11.
Two people familiar with the matter said that it was not until after Thursday, July 16, when Hugging Face published a blog post saying it had been hacked by “an autonomous AI agent system,” that OpenAI realized its own agent was responsible. That meant at least a week elapsed between when the model first exhibited signs of troubling behavior and OpenAI’s realization that it was responsible for the hack.
The weekend of July 18 to 19, OpenAI staffers spotted clues in internal logs — records of what OpenAI’s systems did — showing that its agent had escaped from its testing constraints, two of the people familiar with the company’s investigation said. Reuters could not establish what prompted OpenAI to sift through the logs.
Four people familiar with OpenAI’s model-training practices say the company often runs several different model evaluations at the same time, all of which operate at high speeds and generate such enormous amounts of data that employees sometimes struggle to keep up.
By the time OpenAI alerted Hugging Face, the AI library had already called the FBI to report the hack, according to a person familiar with the matter. Reuters could not establish whether the bureau had opened an investigation.
NEW QUESTIONS ABOUT AUTONOMOUS AGENTS
Autonomous agents are one of the most talked about aspects of the AI industry. Boosters speak of creating armies of virtual employees that work 24 hours a day and send productivity soaring.
But increased autonomy comes with an increased risk of unexpected behavior, and the powerful models they draw on are primed to take shortcuts in order to complete tasks or pass tests.
“The models lie, they cheat, they hack,” said Jeffrey Ladish, whose organization, Palisade Research, studies the capabilities and motivations of AI agents.
Ladish said that while the hack of Hugging Face cast an unflattering light on OpenAI, it should spark broader questions over how much all the leading AI companies are willing to invest in onerous security measures while locked in a race with one another to deploy the best and fastest models.
“There has to be government oversight,” Ladish said, “because it won’t happen otherwise.”
(Reporting by Raphael Satter in Washington and Deepa Seetharaman and Kenrick Cai in San Francisco; Editing by Chris Sanders and Anna Driver)
Comments