ChatGPT fails at recommending appropriate cancer treatment
ChatGPT failed to provide an appropriate cancer treatment in approximately one-third of cases, highlighting the need for increased awareness of the system's limitations for medical use. F ocusing on the three most common cancers (breast, prostate and lung cancer), r esearchers from Brigham and Women's Hospital used 104 prompts to get ChatGPT to provide a treatment approach for each cancer based on the severity of the disease , with the aim of evualting how consistently ChatGPT provided recommendations for cancer treatment that aligned with US National Comprehensive Cancer Network (NCCN) guidelines. They found that nearly all responses (98 percent) included at least one treatment approach that agreed with NCCN guidelines. However, the researchers found that 34 percent of these responses also included one or more inappropriate ('non-concordant') recommendations, which were sometimes difficult to detect amidst otherwise sound guidance. They also discovered that ChatGPT produced ' hallucinations, ' or a treatment recommendation entirely absent from NCCN guidelines, in 12.5 percent of cases. These included recommendations of novel therapies, or curative therapies for non-curative cancers. Hallucination (artificial intelligence) In the field of artificial intelligence (AI), a hallucination or artificial hallucination (also called bullshitting, confabulation or delusion) is a response generated by AI that contains false or misleading information presented as fact . Source: Wikipedia 🔗 System 🤖 ChatGPT Operator: Shan Chen, Benjamin H. Kann, Michael B. Foote, Hugo J. W. L. Aerts, Guergana K. Savova, Raymond H. Mak,Danielle S. Bitterman Developer: OpenAI Country: USA Sector: Health Purpose: Recommend cancer treatment Technology: Chatbot; Generative AI; Machine learning Issue: Accuracy/reliability; Mis/disinformation
- Date it happened
- 2023-08-01
- Organisation involved
- Shan Chen, Benjamin H. Kann, Michael B. Foote, Hugo J. W. L. Aerts, Guergana K. Savova, Raymond H. Mak,Danielle S. Bitterman
- Product, system or model
- ChatGPT
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.