In this illustration taken on June 23, 2023, the word AI (artificial intelligence) and a robotic hand are placed on a computer motherboard. REUTERS/Dado Ruvic/Illustration/File Photo
[1/2]In this illustration taken on June 23, 2023, the word AI (artificial intelligence) and a robotic hand are placed on a computer motherboard. REUTERS/Dado Ruvic/Illustration/File Photo Purchase License rightopen a new tab
Bill would require safety testing of advanced AI models
Tech companies are largely opposed to this.
The bill now awaits the California Governor’s signature.
SAN FRANCISCO (Reuters) – California’s legislature on Wednesday passed a hotly contested artificial intelligence (AI) safety bill, which now needs one more vote before it goes to Governor Gavin Newsom, who has until Sept. 30 to decide whether to sign or veto the bill.
Technology companies that are developing generative AI that can respond to prompts with full text, images and voice and perform repetitive tasks with minimal intervention have largely pushed back against the bill, known as SB 1047, saying it could drive AI companies out of the state and stifle innovation.
Some Democrats in the U.S. Congress, including Representative Nancy Pelosi, also opposed the move, while supporters included Tesla Inc (TSLA.O).open a new tab CEO Elon Musk, who also runs AI company xAI, has said he supports the bill.
The bill would require safety testing of many cutting-edge AI models that cost more than $100 million to develop or require a certain amount of computing power. It would also require AI software developers operating in the state to outline how to stop their AI models if they malfunction — essentially a kill switch.
The bill also gives state attorneys general the power to sue if developers don’t comply, including in cases of ongoing threats such as AI taking over government systems like the power grid.
The bill also requires developers to hire third-party auditors to evaluate their safeguards and provides additional protections for whistleblowers who report misuse of AI.
The bill’s author, Democratic state Sen. Scott Wiener, represents San Francisco, home to OpenAI and many of the startups developing the powerful software. Wiener said the law is needed to protect the public before AI advances get out of hand or out of control.
Martin Casado, a general partner at venture capital firm Andreessen Horowitz, said before the vote that he hopes Newsom will veto it. “This is the most bipartisan, broad-based opposition I’ve ever seen,” he said.
Alphabet (GOOGL.O)open a new tab OpenAI, Meta Platforms (META.O), backed by Google and Microsoftopen a new tab Amazon (AMZN.O) expressed its concerns in a letter to Weiner.open a new tabAntropic, which is backed by the group, said the bill’s benefits likely outweigh its costs, but added that there were still concerns and ambiguities.
sign up here.
Reporting by Anna Tong in San Francisco; Editing by Matthew Lewis
Our standards: Thomson Reuters Trust Principles.open a new tab
Purchasing License Rights
Anna Tong is a San Francisco-based correspondent for Reuters covering the technology industry. She joined Reuters in 2023 after working as a data editor at the San Francisco Standard. She previously worked as a product manager at a technology startup, in user insights at Google and in call centre operations. She graduated from Harvard University.