The California New Act requires AI to remind children: Chat robots are not humans

Recently, California has proposed a bill called SB243, which aims to protect children from the impact of potential risks of artificial intelligence chat robots. The bill, introduced by California Senator Steve Padilla, mainly requires AI companies to regularly remind minors that chatbots are actually artificial intelligence, not humans.

The central purpose of the bill is to prevent children from encountering addiction, isolation and misleading when using chatbots. In addition to requiring AI companies to issue regular reminders, the bill restricts companies from using “addiction interaction models” and requires them to submit annual reports to the California Department of Health Care Services. These reports need to include the number of detection times about the idea of ​​suicide by young users, and the number of times the chat robot mentioned this topic. In addition, AI companies need to inform users that their chat robots may not be suitable for some children.

The background of this bill is closely related to a parent's wrong death lawsuit against CHARACTER.AI. The lawsuit claims the company's customized AI chatbots are "extremely dangerous" because their children eventually chose to commit suicide after long interactions with the chatbots. Another lawsuit accused the company of sending "harmful materials" to young people. To this end, Character.AI announced that it is developing parental control features and launching a new AI model designed to block "sensitive or suggestive" content to ensure the safety of teenagers.

Padira said at a press conference: "Our children are not white mice experimented by technology companies, and they cannot test at the cost of their mental health. We need to provide common sense protection for chat robot users to prevent developers from using them to use them. Known as addictive and predatory approaches.” As states and federal governments increasingly focus on the security of social media platforms, AI chatbots are expected to be the next focus of lawmakers.