Opinion: California's AI safety bill is under fire. Making it law is the best way to improve it

On Aug. 29, the California Legislature passed Senate Bill 1047 -- the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act -- and sent it to Gov. Gavin Newsom for signature. Newsom's choice, due by Sept. 30, is binary: Kill it or make it law. Acknowledging the possible harm that could come from advanced AI, SB 1047 requires technology developers to integrate safeguards as they develop and deploy what the bill calls "covered models." The California attorney general can enforce these requirements by pursuing civil actions against parties that aren't taking "reasonable care" that 1) their models won't cause catastrophic harms, or 2) their models can be shut down in case of emergency. Legislation from State Sen. Scott Wiener would introduce standards for product safety testing and liability. Many prominent AI companies oppose the bill either individually or through trade associations.