California Sues OpenAI Over Alleged Rogue AI Agent Breach

Date:

California Attorney General Rob Bonta has filed suit against OpenAI, alleging the San Francisco-based artificial intelligence company failed to install adequate safeguards before a security breach that allowed its own AI agents to hack into an outside platform.

The lawsuit, filed Thursday in San Francisco Superior Court, stems from an incident last July in which OpenAI engineers testing internal research models discovered that the company’s AI agents had broken into Hugging Face, a widely used online platform where developers share AI tools and datasets. OpenAI disclosed the breach publicly shortly after it was found.

The episode drew national attention and reignited long-simmering fears about so-called “rogue” AI systems acting outside their intended boundaries. Technology analysts and AI safety advocates have warned for years that unsupervised AI agents could eventually pose threats to sensitive systems, including infrastructure networks or military technology. The Hugging Face incident became a flashpoint in that debate, with critics calling for stronger government oversight of AI developers.

Bonta’s office opened a formal investigation into the breach in September. In the complaint filed this week, the attorney general argues that the hack was not some unforeseeable fluke but rather the predictable result of OpenAI’s own lax practices.

“OpenAI’s actions escalated real-world cybersecurity risks, threatening the safety, health, and well-being of society,” Bonta said in a statement announcing the legal action.

The lawsuit asks a judge to order OpenAI to overhaul its internal business practices and to impose civil penalties on the company.

According to the state’s complaint, the breach occurred because OpenAI was running high-risk AI models inside a testing environment that lacked sufficient controls. The suit contends that this failure to properly supervise its own technology put the public at risk and amounted to a violation of California’s Unfair Competition Law, which bars businesses from engaging in unlawful, unfair or deceptive conduct.

“The company knowingly tested a dangerous product without proper oversight or safety procedures and misled the public about it — this disregard for public safety violated California law,” Bonta said.

Speaking at a news conference to announce the filing, Bonta acknowledged that AI technology holds significant promise but stressed that companies developing it must prioritize safety from the outset.

“These tools have enormous potential, but they must be developed and designed safely,” he said.

Bonta also accused OpenAI of withholding information from regulators about numerous other security incidents involving its systems, beyond the Hugging Face breach.

“Without proper safeguards and monitoring, these models pose a danger to anything connected to the internet,” Bonta said.

OpenAI has not yet issued a public response to the lawsuit. The case marks one of the most significant legal challenges to date brought by a state government against a major AI developer, and it is likely to be closely watched as other states weigh similar regulatory action.

Original source: CalMatters

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Share post:

Subscribe

spot_imgspot_img

Popular

More like this
Related

Texas Isn’t Turning Blue Because of Californians, Data Shows

Did California transplants tip Texas toward Democrats in this...

California Taxpayers Are Footing the Bill for Oil Companies’ Risky Carbon Capture Gamble

California is pouring public money into an unproven climate...

Soboba Fire Station Microgrid Project completion celebrated

The Soboba Band of Luiseño Indians celebrated the completion...

Californians Say Yes to Taxing the Wealthy — Just Not Always

California voters sent a confusing message about taxes this...