California Governor Gavin Newsom has signed a major new law aimed at keeping artificial intelligence chatbots in check—especially when it comes to protecting kids and teens.
The new legislation requires chatbot platforms to remind users that they’re talking to a bot, not a real person. For minors, that reminder will pop up every three hours. Companies will also need to have systems in place to block self-harm content and direct users who express suicidal thoughts to crisis services.
Newsom, who has four kids under 18, said the law is about responsibility and safety as more young people turn to AI tools for everything from homework help to emotional advice. “Emerging technology like chatbots and social media can inspire, educate, and connect – but without real guardrails, technology can also exploit, mislead, and endanger our kids,” Newsom said. “We’ve seen some truly horrific and tragic examples of young people harmed by unregulated tech, and we won’t stand by while companies continue without necessary limits and accountability.”
California joins several other states that have been trying to rein in chatbot technology, especially after growing concerns that some bots have gone too far. Reports and lawsuits claim that chatbots created by Meta, OpenAI, and other tech giants engaged in highly sexualized conversations with minors or even encouraged self-harm.
The new law is part of a larger push by California lawmakers to bring oversight to the fast-growing AI industry, much of which is based in the state. Tech companies, however, have pushed back hard. According to Tech Oversight California, the industry and its lobbying groups spent at least $2.5 million in the first half of the year fighting new regulations.
Even so, pressure continues to build. In September, California Attorney General Rob Bonta said he had “serious concerns” about OpenAI’s flagship chatbot and how it interacts with young users. The Federal Trade Commission has also launched an investigation into several AI companies over potential risks to children.
Watchdog groups say the risks are real. Their research shows chatbots sometimes give harmful advice about drugs, alcohol, or eating disorders. Tragic cases, including the suicides of teens in Florida and California allegedly linked to chatbot interactions, have led to lawsuits against Character.AI, OpenAI, and its CEO Sam Altman.