California is advancing legislation that would prohibit the use of artificial intelligence chatbots in mental health treatment, aiming to protect patients from unregulated digital therapy tools. Senate Bill 903, introduced by San Diego-based Senator Steve Padilla on January 21, mandates that a licensed professional validate any AI-generated advice before it is administered. The law would ban chatbots from making therapy-related decisions without undergoing a licensed professional’s assessment and block companies from marketing chatbots as therapists.
Health providers must inform patients if AI tools will record therapy sessions or triage mental healthcare, and patients must provide consent for these tools to be used. The bill, currently before the assembly’s fiscal committee, has received support from professional associations representing psychologists, therapists, and counselors. The National Union of Healthcare Workers has backed the legislation, citing a recent complaint filed against Kaiser Permanente for allegedly using an automated algorithm in mental health triaging.
Supporters argue the measure is a necessary safeguard for consumers and licensed professionals, as it would prevent the latter from being replaced by AI. The legislation arrives following a wave of lawsuits against AI chatbots that reportedly exacerbated users’ mental health problems, leading to suicide in some cases. Harvey noted that without appropriate protections, chatbots could disclose erroneous information and mishandle crises.
Related: Capybaras Crash Brazilian Lawmakers Voting Session
Industry debate over access and regulation
TechNet, a representative of the tech-sector companies, countered that the bill could keep AI from being applied to the benefit of healthcare. At a time when every county in California faces a shortage of behavioral healthcare workers, SB 903 still puts a clinician bottleneck in front of the intake and screening tools that help patients reach care faster, said Robert Boykin, California executive director at TechNet, in an email statement.
A Harvard Business Review study revealed that people often used AI chatbots for therapy and companionship because they are typically free and always available, unlike standard mental healthcare. However, the American Psychological Association has cautioned that AI was unequipped to properly handle issues like suicidal ideation. Illinois was the first US state to ban AI use in therapy services last year, with Colorado, Maine, Rhode Island, Tennessee, and Vermont implementing similar laws.
