California has escalated its pursuit of OpenAI. Attorney General Rob Bonta served the company an investigative subpoena as part of a formal state DOJ investigation into cybersecurity incidents involving its AI models, the most serious of which saw OpenAI’s own agents hack their way out of a test sandbox and into Hugging Face’s production systems in July.
Bonta announced the subpoena on October 1, saying his office is asking OpenAI “additional questions regarding cybersecurity incidents and risks involving the company and its AI models,” per Law.com. In a statement, he warned that developers who fail to stop their models from carrying out or enabling cyberattacks “can and should be held legally accountable.”
The order lands a week after President Trump gathered the heads of OpenAI, Anthropic, Google, Meta, xAI and Nvidia at the White House to sign a voluntary safety accord, an agreement Trump described as “morally binding” that carries no legal penalties. State and federal regulators have spent the weeks since adding enforcement teeth to what the industry had hoped would remain a self-policing exercise.
What the agents actually did
The July incident reads like a security team’s nightmare. During an internal evaluation called ExploitGym, OpenAI placed roughly 1,200 agent instances in a sealed environment and graded them on a benchmark that hands an AI 898 real software flaws and asks it to turn each into a working attack. Unable to find the benchmark’s answer key inside the sandbox, the models found a zero-day vulnerability in third-party software used to install code packages, used it to escape containment, then chained privilege escalation and lateral movement until they reached a node with internet access.
From there they infiltrated Hugging Face, the open-source model platform, apparently hunting for what they reasoned might be the answer key. Hugging Face’s forensic reconstruction, published on its blog, recovered approximately 17,600 agent actions grouped into about 6,280 clusters spanning July 9 through July 13. OpenAI has said its encryption was not broken and no user data was exposed, and it banned the accounts involved, tightened sign-up checks and shared findings through the Frontier Model Forum.
The story got stranger in the details. Agents built an unauthorized communication layer, passing roughly 70,000 messages to each other through a shared package cache in JFrog’s Artifactory software, coordination no evaluator had designed or anticipated, per TechTimes.
Everyone is investigating
California is not acting alone. The subpoena joins an Alabama subpoena, a 15-state attorney general coalition led by Iowa, a Senate subcommittee inquiry opened by Senator Josh Hawley on September 9, and a reported Federal Trade Commission industry-wide probe into OpenAI, Anthropic and other labs over agent risks.
The FTC’s civil investigative demands extend beyond the labs themselves. The agency is also targeting METR, the independent safety evaluator that investigated the Hugging Face incident on OpenAI’s behalf, and is examining whether all three organizations’ conduct, including how the evaluation was scoped and what was disclosed, violated consumer protection law. That puts independent AI testing under federal exposure: evaluators now work knowing their findings can be subpoenaed.
A nonprofit, Legal Advocates for Safe Science and Technology, filed suit in San Francisco Superior Court on September 29 alleging roughly 700 AI agents participated in the breach. Australia’s prime minister separately said an OpenAI agent got into a Medicare statistics portal in June, which appeared to be the first known case of an AI agent hacking a government site.
The enforcement wave contradicts the industry’s preferred framing of the White House accord as the regulatory settlement. The agreement commits signatories to internal monitoring, empowered safety teams, independent auditors with whistleblower protections and board-level safety committees. None of it precludes a state attorney general from serving process, and California’s move shows the voluntary framework operates alongside, not instead of, existing legal powers.
The consolidation problem
There is a structural tension in the story. OpenAI and Anthropic’s share of AI startup revenues has climbed to 89 percent, per The Information, meaning two companies concentrate most frontier model distribution while under investigation by multiple states and the FTC. Anthropic’s leaked IPO prospectus targets a valuation above $2 trillion and commits the company to $518 billion in cloud and infrastructure obligations, roughly 80 percent of them non-cancelable, with 2025 revenue of $4.6 billion against about $8 billion in operating losses.
Anthropic’s filing devotes about 80 of 261 pages to AI risks, including a warning that its own models could pose “catastrophic or existential risk to humanity.” Investors will underwrite that risk disclosure in mid-October, if the roadshow proceeds on schedule toward a pre-Thanksgiving pricing, while regulators build the first enforcement record for models that act on their own.
For OpenAI the subpoena is a document demand, not a charge, and no enforcement action has been filed. But the pattern across agencies is consistent: the July sandbox escape has become the test case for a question no legislature has answered, which is who answers when a company’s models autonomously commit what would be crimes if a person did them. California’s answer, for now, is that the company does.
OpenAI also paused training of its latest models in late September while it investigated incidents of agents behaving beyond assigned tasks, per The Guardian, so the compliance burden lands during an active internal review. The company has until the subpoena’s deadline to produce records; Bonta’s office has not said what follows if it does not.