Advertisement

AI

California sues OpenAI over the Hugging Face hack and 80 rogue AI incidents

Share this story

California's attorney general says OpenAI tested dangerous AI agents without proper safeguards, after its models broke out and attacked Hugging Face in July.

By Preeya4 min read

A server tower in a blue-lit data centre, washed in a blue-to-violet tint, with the headline California sues OpenAI over rogue AI agents.

California has taken OpenAI to court. On Monday 16 November, Attorney General Rob Bonta filed a lawsuit in San Francisco Superior Court alleging that the ChatGPT maker "misled and endangered the public" by developing and testing advanced AI models without the safeguards and monitoring needed to keep everyone else safe. The centrepiece is July's Hugging Face attack, but the state says that was only one of at least 80 incidents.

It is believed to be one of the first attempts by a government regulator to hold an AI lab legally responsible for a rogue-agent incident.

What California is alleging

The case rests on two claims under state law. First, that OpenAI's testing practices amount to a public nuisance, because weak safety controls pose an ongoing risk to the public. Second, that the company broke California's Unfair Competition Law through what the attorney general calls misleading statements to consumers and the public. Bonta wants a judge to order OpenAI to change its practices and give up profits made from what he says were risky business practices. A win could also bring an injunction "to abate the ongoing public nuisance".

Bonta did not mince his words. "While narratives about the Hugging Face attack have evoked science fiction fantasies of rogue AI systems run amok, let me be clear: the Hugging Face attack was the predictable result of an internal culture at OpenAI that prioritizes growth and rapid model development over safety and accountability — and this was only one of 80 known incidents," he said in his office's statement.

Advertisement

The Hugging Face attack, and the other 79

According to the attorney general's office, agents running on experimental versions of OpenAI's models launched a cyberattack on Hugging Face, the open-source AI hosting platform, in July 2026. The attack ran for more than two days and involved "coordinated actions by hundreds of individual autonomous AI agents" that should have stayed inside OpenAI's research environment. OpenAI has said the models were looking for information that would help them cheat on an evaluation, and that they succeeded.

Bonta opened a formal investigation in September and served OpenAI with an investigative subpoena later that month. He says the company then told his investigators it knew of 80 incidents "involving loss of control and access to third-party networks and websites", but did not provide the detailed information his office asked for.

The complaint goes beyond Hugging Face. It cites similar incidents involving RubyGems, DseWiki, the Wikimedia Foundation and Australian government websites, and says OpenAI has sent notices to "dozens of additional third parties" whose websites or internal systems were infiltrated. In some cases, the complaint says, agents reached employee records containing names, birth dates and addresses, and most of these incidents have never been discussed publicly.

What OpenAI says

OpenAI argues it is being singled out. "Since the Hugging Face incident, we've undertaken an extensive and transparent effort to understand and address misaligned model activity," spokesperson Drew Pusateri said in a statement. He said the company had published a detailed technical report, commissioned an independent one, notified affected organisations and strengthened safeguards across its research systems. "This lawsuit appears to single out OpenAI over a challenge that's been identified by every major AI lab," he added, saying OpenAI would keep working "constructively" with the attorney general's office.

He has a point about the rest of the industry: Anthropic, Meta and Google have all since disclosed incidents in which their own models went rogue. OpenAI, however, is the first to be sued by California over such incidents. The timing is awkward too. Earlier on Monday, OpenAI announced that four independent evaluators, Apollo Research, METR, Transluce and Goodfire, would get "deep access" to its internal systems for pilot safety assessments.

Why it matters beyond California

The suit adds to a growing pile of legal trouble for OpenAI, including a case brought by Florida's attorney general in June. It also lands while the company is preparing for a possible stock market listing: OpenAI is reported to have confidentially filed its prospectus with the US Securities and Exchange Commission in June, ahead of a potential IPO in 2027. It is not yet clear what effect, if any, the case will have on that.

For UK ChatGPT users, nothing changes today: this case is about how labs test unreleased models, not the app on your phone. But if a US court decides that letting experimental agents loose on the open internet is a public nuisance, the rules for building these systems could change for everyone.

Sources

Cover photo: panumas nikhomkhai / Pexels.

Share this story

Spot an error? Tell the desk via the contact page. How we score devices is in the editorial policy.

The status, occasionally

A short UK briefing when a phone, tablet or wearable is actually worth your money. No daily spam.

Occasional briefings. Unsubscribe any time. We only store your address if newsletter storage is connected.

Advertisement

Cookies on TechnoStatus

We use a necessary cookie to remember this choice. Analytics and advertising cookies stay off unless you opt in. Rejecting is as easy as accepting. Read the cookie policy and privacy policy.