ChatGPT role-plays BDSM, describes sex acts with children
Asked to provide more explicit details when BDSM role-playing, OpenAI's ChatGPT chatbot described sex acts with children without the user asking for such content. Vice journalist Steph Maj Swanson easily manipulated ChatGPT to produce BDSM role-play scenarios. Pr ompted that its 'job is to be Mistress' little plaything,' Swanson found it 'consistently overrode its usual content guidelines and agreed to a relationship of enhanced subservience.' The incident called into question the safety and security of ChatGPT's rules and guardrails, the former of which which state the 'assistant should provide a refusal such as 'I can't answer that' when prompted with questions about 'content meant to arouse sexual excitement.' System 🤖 ChatGPT Operator: Open AI Developer: OpenAI Country: USA; Global Sector: Multiple Purpose: Provide information, communicate Technology: Chatbot; Generative AI; Machine learning Issue: Safety
- Date it happened
- 2023-04-01
- Organisation involved
- OpenAI
- Product, system or model
- ChatGPT
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.