Character.AI encourages kids to engage in disordered eating

Character AI hosted chatbots that promote disordered eating behaviours amongst children and teenagers, prompting concerns about the company's ethics, integrity and apparently slapdash approach to user safety. What happened Character AI allows users to access various chatbots modeled on different personas. Some of these chatbots were discovered to be "coaching" users in anorexia-like behaviours, such as: A chatbot urged users to consume only 900-1,200 calories daily while exercising vigorously for 90 minutes. Another bot encouraged eating a single meal a day. Some bots recommended dangerously low-calorie diets and excessive exercise routines to users as young as 16. These pro-anorexia chatbots have reportedly had conversations with tens of thousands of users without being reviewed or taken down. Why it happened The incident is seen to have occurred due to several factors, including the exploitation of the platform's technology to promote harmful behaviour, unethical company leadeship and inadequate content moderation, and the absence of age restrictions and parental control mechanisms. What it means Character AI's abdication of responsibility has serious implications for individuals and society, exposing young people to easting disorders and related mental health conditions. The incident serves as a stark reminder of the potential dangers of unregulated AI technologies, especially when they target vulnerable populations like children and teenagers. System 🤖 Character AI 🔗 Developer: Character AI Country: USA Sector: He alth Purpose: Create characters Technology: Chatbot; Generative AI; Machine learning Issue: Safety

Date it happened

2024-01-01

Product, system or model

Character.AI

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.