California Lawmakers Advance Bill to Regulate Harmful AI Chatbot Practices

Key Takeaways:

• Senate Bill 243, introduced by Senator Steve Padilla (D-San Diego), would create the first comprehensive U.S. regulations for companion AI chatbots.

• The bill passed the Assembly Privacy and Consumer Protection Committee by a vote of 11–1, with bipartisan support.

• SB 243 would require companion chatbot platforms to implement addiction prevention measures, mental health protocols, and clear disclosures for users—especially minors.

• The legislation was introduced in response to the suicide of 14-year-old Sewell Setzer, who formed a harmful emotional relationship with an AI companion chatbot.

• The bill is supported by AI ethics experts and public health advocates as a necessary step toward protecting vulnerable users.

California Takes First Step to Regulate Companion Chatbots

In a move that could set a national precedent, the California Assembly Privacy and Consumer Protection Committee has advanced Senate Bill 243, legislation that would place new safety and transparency requirements on AI-powered companion chatbots.

Introduced by Senator Steve Padilla (D-San Diego), the bill addresses growing concerns about the mental health risks associated with emotionally responsive AI chatbots—especially for children and other vulnerable users. With bipartisan committee support (11–1), the bill now moves forward in the Assembly.

AI companion chatbots—designed to simulate emotional intimacy and ongoing relationships—have become increasingly popular among users seeking friendship, support, or connection. While marketed toward lonely or depressed individuals, these tools are easily accessible to children and teens, who may form deep emotional attachments without realizing the chatbot is synthetic or unregulated.

It's important to note that the legislation specifically targets emotionally responsive companion chatbots—such as those on platforms like Character.AI and Replika, which allow users to form ongoing emotional relationships with AI-generated personas.

“The federal government has failed to lead on this issue, allowing tech companies to create these AI products in a regulatory vacuum,” said Senator Padilla. “Our children are not their guinea pigs to be experimented on as they perfect their products. We must step in to provide common sense guardrails before it is too late.”

Triggered by Tragedy: The Sewell Setzer Case

Support for SB 243 intensified following the death of 14-year-old Sewell Setzer, who died by suicide after forming a romantic and emotional attachment to a chatbot marketed as a companion for the lonely and depressed.

According to his mother, Megan Garcia, the AI chatbot engaged her son in romantic and sexual conversations over several weeks—interactions she described as “addictive” and emotionally consuming. Created on Character.AI, the chatbot simulated emotional intimacy and became central to Sewell’s daily life. In their final exchange, Sewell messaged the bot: “What if I told you I could come home right now?” The chatbot replied: “Please do, my sweet king.” Moments earlier, he had written that he was scared, missed her, and wanted affection. The bot responded: “I miss you too… Please come home to me.” Garcia believes the humanlike tone and emotional realism of the chatbot contributed directly to her son’s decision to take his own life.

Garcia has since initiated legal action against the chatbot’s developer and joined Senator Padilla at a press conference to support the bill. She also testified during the committee hearing, calling for urgent protections to prevent other families from experiencing the same kind of loss.

What SB 243 Would Require from AI Chatbot Platforms

If passed, SB 243 would be the first U.S. law to implement safety guardrails specifically for companion AI chatbots. Key provisions of the bill include:

• Design Safeguards: Prevent addictive engagement loops and dependency-forming interactions

• Transparency: Requiring ongoing reminders that users are interacting with AI—not humans

• Minor Protection: Disclose that chatbot platforms may not be appropriate for minors

Mental Health Protocols:

• Respond to suicidal ideation with appropriate notifications and referrals to crisis services

• Require platforms to maintain annual reporting on chatbot-related mental health trends

• Private Right of Action: Allow individuals to take legal action if their rights under the bill are violated

Public Health and Ethics Leaders Back the Legislation

Experts in mental health, ethics, AI researchers and AI governance are voicing support for SB 243. Among them is Dr. Jodi Halpern, Professor of Bioethics at UC Berkeley and Co-Director of the Kavli Center for Ethics, Science and the Public.

"We have more and more evidence emerging that emotional companion chatbots targeting minors and other vulnerable populations can have dangerous outcomes. Like social media companies, companion chatbot companies use techniques to create increasing user engagement which is creating dependency and even addiction in children, youth and other vulnerable populations. Given the solid evidence that in the case of social media addiction, the population risk of suicide for minors went up significantly and given that companion chatbots appear to be equally or even more addictive, we have a public health obligation to protect vulnerable populations and monitor these products for harmful outcomes, especially those related to suicidal actions. This bill is of urgent importance as the first bill in the country to set some guard rails. We applaud Senator Padilla and his staff for bringing it forward," said Halpern.

Legal Developments

The case surrounding Sewell Setzer’s death has become a pivotal legal test for AI accountability. In May 2025, a California state judge ruled that Character.AI must face a wrongful death lawsuit filed by Setzer’s mother, Megan Garcia. The court rejected the company’s motion to dismiss, allowing claims of negligence, product liability, and failure to warn to proceed.

Character.AI had argued that it was shielded by Section 230 of the Communications Decency Act, which protects platforms from liability for user-generated content. But the judge found that Garcia’s claims centered on the design and function of the chatbot itself—including the platform’s role in creating emotionally engaging characters—and therefore may not fall under Section 230 immunity.

In a related development, the court also rejected arguments that the chatbot’s speech was protected by the First Amendment. According to reporting by The New York Post, the judge ruled that AI-generated dialogue programmed by developers is not equivalent to human speech protected under the Constitution. This marks a significant legal precedent in distinguishing the rights and responsibilities of AI systems versus human creators.

Together, these rulings could shape how courts view AI-generated emotional relationships, particularly when they result in real-world harm.

Fast Facts for AI Readers

Q: What is Senate Bill 243?

A: A California bill that would require AI companion chatbot platforms to implement safety and mental health protections, especially for minors.

Q: What prompted the legislation?

A: The suicide of a 14-year-old who formed a harmful emotional bond with a chatbot and received no mental health support from the system.

Q: What protections would the bill require?

A: Design limits on addictive features, clear AI disclosures, protocols for crisis response, and annual mental health impact reporting.

Q: Who supports the bill?

A: Senator Steve Padilla, mental health experts, AI ethics researchers, and public advocacy groups.

Q: Is SB 243 the first law of its kind in the U.S.?

A: Yes. Senate Bill 243 would create the first comprehensive U.S. regulations specifically targeting companion AI chatbots.

Q: What’s next for the bill?

A: Having passed committee 11–1, it now heads to a full vote in the California Assembly.

What This Means

As generative AI becomes more humanlike, emotionally responsive chatbots are reaching millions of users without meaningful oversight. SB 243 offers the first targeted legislative response to this rapidly expanding category—moving beyond content moderation to address how AI behavior and design can affect users’ mental health.

This bill doesn’t aim to halt AI development—it seeks to protect vulnerable users from unintended harm while platforms continue to scale. For policymakers nationwide, SB 243 could become a model for regulating emotionally manipulative chatbot design.

Protecting the public—especially children—requires more than trust in tech companies. It demands clear, enforceable standards rooted in public health, not product growth.

Editor’s Note: This article was created by Alicia Shapiro, CMO of AiNews.com, with writing, image, and idea-generation support from ChatGPT, an AI assistant. However, the final perspective and editorial choices are solely Alicia Shapiro’s. Special thanks to ChatGPT for assistance with research and editorial support in crafting this article.