IBM Watson recommends "unsafe and incorrect" cancer treatments
IBM Watson for Oncology, an artificial intelligence system designed to assist doctors in cancer treatment decisions, was in the spotlight after internal documents revealed it had recommended "unsafe and incorrect" cancer treatments. According to documents obtained by STAT News , the system was trained using "synthetic" or hypothetical patient cases rather than real patient data, leading to inaccurate recommendations. In one example, a 65-year-old man diagnosed with lung cancer, who also seemed to have severe bleeding. Watson reportedly suggested the man be administered both chemotherapy and the drug “Bevacizumab.” But the drug can lead to “severe or fatal hemorrhage,” according to a warning on the medication, and therefore shouldn’t be given to people with severe bleeding. IBM reportedly kept these issues secret for over a year, raising concerns about transparency in its product and in AI healthcare applications more generally. Some doctors expressed frustration with the system, stating it was unusable for most cases. While no patients were reportedly harmed as these recommendations were not used on real cancer patients, the revelations raised serious questions about the reliability and safety of AI in healthcare. IBM defended the system, claiming continuous improvements based on feedback and scientific advancements. System 🤖 Watson for Oncology Documents 📃 IBM Watson for Oncology sales manual Operator: Developer: IBM; Memorial Sloan Kettering Hospital Country: USA Sector: Health Purpose: Diagnose cancer; Recommend treatments Technology: Machine learning Issue: Accuracy/reliability; Safety; Effectiveness/value Investigations, assessments, audits 🧐 ST AT News. IBM’s Watson supercomputer recommended ‘unsafe and incorrect’ cancer treatments, internal documents show
- Date it happened
- 2018-07-01
- Product, system or model
- Watson for Oncology
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.