Report: AI companion apps "relentlessly" pry and exploit user data
Top "AI companion apps" systemically collect vast amounts of user data without consent and then share it for targeted advertising, according to researchers. Analysing a number of apps - including Replika , Wysa and Chai - that ranked highest on Google's Play store, Mozilla Foundation researchers found they collect enormous amounts of personal information, usually without clear user consent. The researchers also found that almost all examined apps sell user data and share it for targeted advertising, with some using over 24,000 trackers in just one minute of use. Many of the applications allow for weak passwords and provide vague privacy policies, and users often remain unaware of how their data is being used or shared, raising significant security concerns, Mozilla discovered. The report also records how users report forming deep emotional connections with these AI companions, leading to dependency and unrealistic expectations of relationships. This emotional manipulation is particularly concerning for vulnerable individuals seeking companionship. The marketing of these apps often targets lonely individuals, promising emotional support and companionship, while simultaneously encouraging users to share intimate details about their lives. System 🤖 Anima Chai Crushon.AI EVA AI Genesia AI iGirl Mimico Replika AI companion chatbot Romantic AI Talkie Soulful AI Operator: Developer: Anima AI; Chai Research Corp ; Codeway Dijital; Luka Inc; Matrix Mobile; Novi; Peekaboo Tech; Romantic AI; SubSup Country: Global Sector: Media/entertainment/sports/arts Purpose: Provide companionship Technology: Chatbot; NLP/text analysis; Neural network; Deep learning; Machine learning Issue: Privacy; Safety; Security
- Date it happened
- 2024-02-01
- Product, system or model
- Anima; Chai; Crushon.AI; EVA AI; Genesia AI; iGirl; Mimico; Replika; Romantic AI; Talkie Soulful AI
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.