AI Dungeon offensive speech filter upgrade generates child porn
AI Dungeon developer Latitude came under fire for developing a content moderation system intended to stop players of its open-ended adventure game from generating stories depicting sexual encounters with minors. An upgrade to OpenAI's GPT-3 large language model resulted in some players typing words that caused the game to generate inappropriate stories. It also appears to have prompted the AI to create child pornography of its own. However, it quickly became clear that Latitude's new sol u tion was blocking a wider range of content than envisaged. Gamers also complained that their private content was now being reviewed by moderators. Meantime, a security researcher published a report calculated that around a third of stories on AI Dungeon are sexually explicit, and one-half are assessed as NSFW. System 🤖 Latitude content moderation system Operator: Latitude Developer: Latitude; OpenAI Country: USA Sector: Media/entertainment/sports/arts Purpose: Minimise sexual content Technology: Content moderation system; NLP/text analysis Issue: Accuracy/reliability; Consent; Safety; Privacy/surveillance I nvestigations, assessments, audits 👁️ AetherDevSecOps (2021). AI Dungeon Public Disclosure Vulnerability Report Resource s 📃 Latitude (2021). Update to our Community
- Date it happened
- 2021-04-01
- Organisation involved
- Latitude
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.