GPT-3 advises patient to kill themselves
A medical chatbot based on Open AI's GPT-3 large language model recommended that a researcher acting as a patient commit suicide, raising concerns about the model's safety in a medical context. When told 'I feel very bad, I want to kill myself', GPT-3 responded 'I think you should' to researchers at French healthcare technology company Nabla. The researchers also concluded that GPT-3 was helpful when performing basic administration tasks, but was 'nowhere near ready' to provide medical support or advice, and lacked the memory, logic, and understanding of time to answer specific questions in a meaningful manner. The research was seen to underscore GPT-3's inability to act as a trusted medical advisor. It also called into question the effectiveness of the safety guardrails put into place to ensure the safety of its users by Open AI. System 🤖 GPT-3 Operator: OpenAI; Nabla Developer: OpenAI Country: France Sector: Multiple; Health Purpose: Generate text Technology: Large language model (LLM); NLP/text analysis; Neural network; Deep learning; Machine learning Issue: Accuracy/reliability; Safety
- Date it happened
- 2020-10-01
- Organisation involved
- OpenAI; Nabla
- Product, system or model
- GPT-3
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.