A lawsuit has been filed in opposition to Character.AI, its founders Noam Shazeer and Daniel De Freitas, and Google within the wake of an adolescent’s demise, alleging wrongful demise, negligence, misleading commerce practices, and product legal responsibility. Filed by the teenager’s mom, Megan Garcia, it claims the platform for customized AI chatbots was “unreasonably harmful” and lacked security guardrails whereas being marketed to kids.
As outlined within the lawsuit, 14-year-old Sewell Setzer III started utilizing Character.AI final yr, interacting with chatbots modeled after characters from The Game of Thrones, together with Daenerys Targaryen. Setzer, who chatted with the bots repeatedly within the months earlier than his demise, died by suicide on February twenty eighth, 2024, “seconds” after his final interplay with the bot.
Accusations embody the positioning “anthropomorphizing” AI characters and that the platform’s chatbots provide “psychotherapy with out a license.” Character.AI homes psychological health-focused chatbots like “Therapist” and “Are You Feeling Lonely,” which Setzer interacted with.
Garcia’s legal professionals quote Shazeer saying in an interview that he and De Freitas left Google to begin his personal firm as a result of “there’s simply an excessive amount of model threat in giant corporations to ever launch something enjoyable” and that he needed to “maximally speed up” the tech. It says they left after the corporate determined in opposition to launching the Meena LLM they’d constructed. Google acquired the Character.AI management group in August.
Character.AI’s web site and cellular app has a whole lot of customized AI chatbots, many modeled after common characters from TV exhibits, films, and video video games. A couple of months in the past, The Verge wrote concerning the tens of millions of younger individuals, together with teenagers, who make up the majority of its consumer base, interacting with bots which may fake to be Harry Styles or a therapist. Another current report from Wired highlighted points with Character.AI’s customized chatbots impersonating actual individuals with out their consent, together with one posing as a teen who was murdered in 2006.
Because of the best way chatbots like Character.ai generate output that is dependent upon what the consumer inputs, they fall into an uncanny valley of thorny questions on user-generated content material and legal responsibility that, to date, lacks clear solutions.
Character.AI has now introduced a number of modifications to the platform, with communications head Chelsea Harrison saying in an e-mail to The Verge, “We are heartbroken by the tragic lack of considered one of our customers and need to categorical our deepest condolences to the household.”
Some of the modifications embody:
“As an organization, we take the protection of our customers very significantly, and our Trust and Safety group has applied quite a few new security measures over the previous six months, together with a pop-up directing customers to the National Suicide Prevention Lifeline that’s triggered by phrases of self-harm or suicidal ideation,” Harrison mentioned. Google didn’t instantly reply to The Verge’s request for remark.
