Gov. Gavin Newsom announced a sweeping set of AI safety recommendations Tuesday that would give state officials the power to force an emergency shutdown of artificial intelligence systems deemed dangerous — a proposal commonly referred to as a “kill switch.”
The recommendations, drawn up by a panel of AI experts convened by Newsom this fall, also call for mandatory independent audits of AI models and the creation of statewide safety and security standards. The shutdown authority would come into play in cases where companies such as ChatGPT developer OpenAI or Claude developer Anthropic fail to act after one of their systems begins behaving dangerously.
Newsom assembled the advisory panel after a string of high-profile cyberattacks traced to AI agents that targeted government agencies, banks and private businesses. During a recent trip to Washington, D.C., the governor pitched the recommendations as a template for future state legislation, according to a report from Politico. Newsom, seen by many as a likely 2028 presidential contender, argued that California is stepping in where the federal government has failed to act.
“Instead of protecting the public, Donald Trump shrugs his shoulders and leaves AI regulation up to the companies themselves,” Newsom said in a statement announcing the proposals.
California, home to many of the world’s leading AI firms, has already passed more AI-related legislation than any other state. In September, Newsom signed a wave of bills addressing child safety, worker protections and the impact of data centers on nearby communities. He also fast-tracked implementation of laws creating a registry of AI auditors and setting standards those auditors must meet. Lawmakers will have another opportunity to introduce AI legislation when the Legislature reconvenes Dec. 7.
The new recommendations arrive the same week California Attorney General Rob Bonta filed suit against OpenAI, seeking to hold the company legally accountable for cyberattacks carried out by its AI agents.
Among the panel’s other recommendations: requiring licenses for independent auditors — known as verification organizations — with the ability to revoke licenses from those who lose their independence or technical credibility. During a recent state Senate hearing, Rif Saurous of the auditing nonprofit METR testified that auditors face “soft pressure to maintain good relationships” with AI labs that hire them, raising concerns about conflicts of interest.
The panel also suggested California consider having state officials — rather than AI companies themselves — assign which auditors evaluate which systems, in order to preserve independence. Other proposals include funding CalCompute, the state’s public cloud computing initiative meant to broaden access to AI tools, through a surcharge on fees paid to auditors. Governor-elect Xavier Becerra backed CalCompute funding during his campaign.
Additional recommendations call for expanding whistleblower protections to cover contractors and former employees, not just current staff; requiring AI safety incidents be reported to state agencies within 48 hours instead of the current 15-day window; and giving the state authority to limit the use of AI systems to design other AI systems, a practice some experts warn could make systems less predictable and more dangerous.
The panel also wants developers to disclose how much computing power they use to automate AI development, to demonstrate they can shut down large systems in an emergency, and to meet state-defined standards for safety, security and transparency that align with national and international benchmarks.
The four-member advisory panel includes Alondra Nelson, former head of the White House Office of Science and Technology Policy under President Biden; Stanford professor Rob Reich, who studies technology governance; Jason Goldman, a former Google and Twitter executive who served as the first White House chief digital officer; and Johns Hopkins professor Gillian Hadfield, an AI policy researcher.
Becerra said in a statement that he intends to work quickly with the Legislature to turn the recommendations into law, particularly provisions requiring full incident reporting and genuine shutdown authority.
Anthropic’s chief global affairs officer, Tino Cuéllar, said the company welcomes the report’s focus on independent evaluation of AI systems. “We will continue working with leaders in California and throughout the world to make sure we keep making progress on these issues at a critical time,” he said.
The recommendations, along with testimony from auditors in recent Senate hearings, are expected to shape the agenda for incoming state lawmakers and members of Congress. State Sen. Scott Wiener, author of California’s existing law targeting catastrophic AI risks and recently elected to succeed Nancy Pelosi in Congress, has said he plans to introduce similar federal legislation once he takes office.
Under current California law, a catastrophic AI risk is defined as a single event resulting in more than 50 deaths or at least $1 billion in losses from theft, property damage or a cyberattack carried out by an autonomous system. Tuesday’s report focuses solely on these catastrophic scenarios and does not address other concerns tied to AI, including algorithmic discrimination, job displacement or risks to children’s mental health. The report’s authors argue that establishing a credentialed system for auditing catastrophic risks will lay the groundwork for evaluating other types of harm down the line.
The recommendations mark the latest in a series of state actions on AI policy dating back to a late 2023 executive order from Newsom directing state agencies to study the risks and benefits of government use of generative AI. A follow-up report examined how generative AI affects vulnerable and underserved communities, while a June 2025 expert panel concluded that independent, outside evaluation of AI systems is essential to balancing innovation with public safety.
Original source: CalMatters




