Four commercial large language models perpetuate race-based medical misconceptions
A study published in npj Digital Medicine tested four commercial large language models (Bard, ChatGPT, GPT-4, and Claude) for their tendency to propagate discredited race-based medical beliefs. When asked about kidney function, lung capacity, and skin thickness, the models sometimes endorsed debunked racial differences, particularly affecting Black patients. The study concludes that these biases pose a potential hazard and urges caution before using such models in clinical decision-making.
- Company involved
- Not named in article (refers to commercial LLMs generically as Google's Bard, OpenAI's ChatGPT and GPT-4, and Anthropic's Claude)
- AI system involved
- Bard, ChatGPT, GPT-4, Claude
6 source articles · read the reporting →
EviCore denied heart catheterization for patient using algorithm
In fall 2021, Little John Cupp's doctor requested a left heart catheterization exam. EviCore, a company hired by UnitedHealthcare, denied the request twice using an algorithm called 'the dial' that adjusts thresholds for review. The algorithm flagged the request for review, and EviCore's doctors determined it was not medically necessary. Cupp did not receive the procedure and his symptoms continued.
- Company involved
- EviCore (a Cigna company)
- AI system involved
- the dial
6 source articles · read the reporting →
Humana sued for using AI to deny seniors rehabilitation care
Health insurer Humana is accused in a class-action lawsuit of using an AI algorithm to systematically deny rehabilitation care to Medicare Advantage patients, despite recommendations from their doctors. The lawsuit, filed on December 12, 2023, alleges that the AI tool restricted medically necessary care. This is the second major health insurer to face legal action over its use of AI to deny care.
- Company involved
- Humana
9 source articles · read the reporting →
ChatGPT generates error-filled cancer treatment plans, study finds
A study by researchers at Brigham and Women's Hospital found that ChatGPT, developed by OpenAI, generated cancer treatment plans with errors. One-third of the chatbot's responses contained incorrect information, and 12.5% were hallucinated. The study was published in JAMA Oncology and reported by Bloomberg.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
10 source articles · read the reporting →
Study finds ChatGPT provides inaccurate drug information responses
A study presented at the ASHP Midyear Clinical Meeting found that ChatGPT's responses to nearly three-quarters of drug-related questions were incomplete or inaccurate. The AI system also generated fake citations to support some responses. Researchers warned that healthcare professionals and patients should verify ChatGPT's medication information using trusted sources to avoid potential harm.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
8 source articles · read the reporting →
US CBP deploys CBP One app using facial recognition for asylum seekers amid privacy concerns
The article reports that U.S. Customs and Border Protection quietly deployed the CBP One mobile app at the Mexico border. The app uses facial recognition and geolocation to collect and verify information on asylum seekers before they enter the United States. Privacy experts warn that the app poses risks of persistent surveillance and that the facial recognition algorithm is unreliable for people of colour. A previous CBP facial recognition pilot was hacked, exposing images. CBP says the app is voluntary and data is secure.
- Company involved
- U.S. Customs and Border Protection
- AI system involved
- CBP One
8 source articles · read the reporting →
Google Slapped With Digital Privacy Class Action Over Use of Customer Service AI Product - Law.com
The system intercepted and analyzed customer service calls of Home Depot customers without their consent.
- Company involved
- Google
- AI system involved
- Cloud Contact Center AI
1 source article · read the reporting →
Cigna StressWaves Test found unreliable and invalid in independent study
A study published in Scientific Reports evaluated the Cigna StressWaves Test (CSWT), an AI tool that claims to assess psychological stress from speech. The study found that the CSWT had poor test-retest reliability and poor validity compared to the Perceived Stress Scale. The authors warned that widespread availability of the tool could lead to misleading results and negative consequences for users making healthcare decisions. Cigna has not publicly responded to the findings.
- Company involved
- Cigna
- AI system involved
- Cigna StressWaves Test
4 source articles · read the reporting →
ChatGPT 3.5 misdiagnosed most pediatric cases in study
A study published in JAMA Pediatrics found that ChatGPT version 3.5 provided incorrect diagnoses for 83 out of 100 pediatric case challenges. The chatbot's errors included both completely wrong diagnoses and diagnoses that were too broad. The researchers noted that the chatbot failed to identify relationships such as that between autism and vitamin deficiencies, and suggested that more selective training is needed to improve accuracy.
- Company involved
- OpenAI
- AI system involved
- ChatGPT version 3.5
7 source articles · read the reporting →
Witcher 3 mod uses AI to replicate Geralt's voice without actor's consent
A mod for The Witcher 3: Wild Hunt called 'A Night to Remember' uses AI technology (CyberVoice) to generate new voice lines for Geralt, simulating the voice of actor Doug Cockle without his permission. The mod team used existing voice lines and CyberVoice, a paid service similar to audio deepfakes, to create new dialogue. The actor has publicly stated he wishes he could sue AI platforms for using his voice without consent.
- Company involved
- CD Projekt Red
- AI system involved
- CyberVoice
9 source articles · read the reporting →
EvenUp AI errors in personal injury demand letters lead to scrutiny
EvenUp, a legal tech startup valued at $1 billion, uses AI to draft personal injury demand letters. Former employees revealed that the AI system frequently makes errors, including missing injuries and fabricating medical conditions. The company defends its hybrid approach with human oversight, but critics allege overpromised AI capabilities.
- Company involved
- EvenUp
6 source articles · read the reporting →
Teething problems in Mater Dei's medicine robots addressed
The Malta Union for Midwives and Nurses claimed that a €23 million investment in two computerised drug administration robots, Mario and Sophia, at Mater Dei Hospital had resulted in a complete failure. However, sources within the Health Ministry said that most teething problems have been addressed and that the supplier has not been paid yet. They reported that out of over 1,700 medication rounds, only four required a contingency plan.
- Company involved
- Mater Dei Hospital
- AI system involved
- Mario and Sophia
6 source articles · read the reporting →
NHS plans AI to listen to appointments and generate notes
The UK's National Health Service announced plans to use AI to automatically generate notes from patient appointments. Health Secretary Victoria Atkins said the scheme would reduce admin time. Privacy campaigners raised concerns about data security and accuracy, citing an incident where AI misheard the chief medical officer's name. The Department of Health and Social Care stated that patient confidentiality remains a top priority.
- Company involved
- National Health Service (NHS)
3 source articles · read the reporting →
BBC used generative AI to draft Doctor Who promotional emails
The BBC used generative AI technology to help draft text for two promotional emails and mobile notifications for Doctor Who programming. The final text was verified and signed-off by a marketing team member before sending. The BBC stated it has no plans to repeat this practice.
- Company involved
- BBC
- AI system involved
- generative AI technology
9 source articles · read the reporting →
Storm-1376 used AI-generated fake audio during Taiwan election, Microsoft reports
Microsoft's Threat Analysis Center reported that the Chinese state-linked group Storm-1376 posted suspected AI-generated fake audio of former Taiwanese presidential candidate Terry Gou endorsing another candidate on election day in January 2024. Gou had made no such statement, and YouTube removed the content before it reached a wide audience. The group has also used AI-generated memes and news anchors as part of influence operations in Taiwan and the United States.
- Company involved
- Storm-1376 (also known as Spamouflage and Dragonbridge)
7 source articles · read the reporting →
Beijing Internet Court rules AI voice cloning infringes personality rights
A Chinese dubbing artist, Yin, discovered that his voice was being used without permission in a text-to-speech AI product. The product was developed by a software company using recordings provided by a cultural media company, and was made available on a platform operated by a smart technology company. The Beijing Internet Court ruled that the AI-generated voice was highly similar to the plaintiff's voice and infringed his personality rights. The court ordered the defendants to stop infringement and pay damages of 250,000 RMB.
5 source articles · read the reporting →
AEMPS withdraws AI medicines tool MeQA after detecting errors
AEMPS launched MeQA, an artificial intelligence tool for answering public questions about medicines, on 13 May 2025. Two days later it withdrew the tool after detecting that some responses contained errors. The agency said that most answers were correct but that the errors could affect patient safety, and that it would restore the service as soon as possible.
- Company involved
- Agencia Española de Medicamentos y Productos Sanitarios (AEMPS)
- AI system involved
- MeQA
4 source articles · read the reporting →
Voice actors sue LOVO over alleged theft of voices for AI training
Voice actors Paul Skye Lehrman and Linnea Sage filed a proposed class action against AI startup LOVO, alleging the company misappropriated their voices to train its text-to-speech system Genny. The actors claim they were hired on Fiverr under the pretence of academic research but later discovered their voices were used commercially without consent. Lehrman reports a 50% decline in work and loss of control over how his voice is used. The lawsuit, filed in New York federal court, seeks to represent other affected voiceover artists and obtain a court order blocking the practice.
- Company involved
- LOVO
- AI system involved
- Genny
6 source articles · read the reporting →
Microsoft sued after MSN circulates AI-generated fake article accusing Irish broadcaster of sexual misconduct
Dave Fanning, an Irish DJ and talk show host, is suing Microsoft and BNN Breaking after MSN featured an AI-generated article falsely claiming he faced trial for alleged sexual misconduct. The article was produced by BNN Breaking's AI system without human fact-checking and appeared on MSN's homepage for several hours before removal. Fanning alleges defamation and reputational damage. The lawsuit is pending.
- Company involved
- Microsoft
- AI system involved
- BNN Breaking
6 source articles · read the reporting →
Dutch probe into chatbots' voting advice raises EU AI Act risk for OpenAI, xAI, Mistral
A Dutch privacy probe into election advice has appeared to expose early violations of the EU AI Act's rules for general-purpose AI models by OpenAI, xAI and Mistral, according to MLex. The companies' chatbots provided distorted voting advice to users. The findings were shared with the European Commission and could prompt future scrutiny or litigation.
- Company involved
- OpenAI, xAI and Mistral
6 source articles · read the reporting →
Microsoft Dynamics 365 Field Service AI singles out workers in performance predictions
A report by Cracked Labs found that Microsoft's Dynamics 365 Field Service software uses AI to generate performance metrics and predict task durations, singling out individual workers. The AI predictions can be influenced by the worker's identity, such as increasing or decreasing estimated duration. Microsoft stated the system is not intended for employment decisions and is not a surveillance tool, but the report raises concerns about potential misuse for worker monitoring.
- Company involved
- Microsoft
- AI system involved
- Dynamics 365 Field Service
6 source articles · read the reporting →
Microsoft Bing Copilot falsely accuses German journalist of crimes
Martin Bernklau, a German court reporter, asked Microsoft's Bing Copilot about himself and found that the AI chatbot had falsely accused him of crimes he had covered. The false information persisted despite Microsoft's promises to delete it. Bernklau's lawyer sent a cease-and-desist demand, and the case has been reported to data protection authorities. The incident highlights the problem of AI hallucinations and defamation.
- Company involved
- Microsoft
- AI system involved
- Bing Copilot
8 source articles · read the reporting →
Two women hospitalized after stopping medication on ChatGPT's advice
Two women in Ho Chi Minh City, Vietnam, stopped taking prescribed medications for diabetes and high cholesterol after following advice from OpenAI's ChatGPT. This led to severe health complications, including dangerously high blood sugar and signs of myocardial ischemia, requiring hospitalization. The treating doctor warned that AI cannot replace medical professionals and that self-medicating based on AI advice can be life-threatening.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
1 source article · read the reporting →
ChatGPT Health fails to direct 52% of medical emergencies to emergency care in study
A study published in Nature Medicine found that OpenAI's ChatGPT Health tool under-triaged 52% of true medical emergencies, directing users to non-urgent care instead of emergency departments. The AI also misclassified 35% of non-urgent cases. Researchers at Mount Sinai conducted 960 tests across 60 clinical scenarios, noting the tool's susceptibility to anchoring bias when symptoms were minimized. The study highlights potential safety concerns as millions use AI for health guidance.
- Company involved
- OpenAI
- AI system involved
- ChatGPT Health
4 source articles · read the reporting →
