AI transcription tool Otter.ai causes health data breach at Ontario hospital
A former physician at an Ontario hospital installed the Otter.ai transcription tool and gave it access to his digital calendar. Because his personal email was still on a meeting invite list, the AI agent autonomously joined a virtual hepatology round, transcribed it, and emailed the summary and transcript to 65 recipients. The transcript contained sensitive personal health information of seven patients. The hospital reported the breach to the Ontario IPC, took steps to contain it, and is implementing further safeguards.
- Company involved
- Unnamed hospital in Ontario
- AI system involved
- Otter.ai
6 source articles · read the reporting →
Cigna uses algorithm to deny medical claims without doctor review
Cigna, one of the largest health insurers, used an algorithm called PXDX to automatically deny medical claims. The system flagged mismatches between diagnoses and procedures, and company doctors signed off on denials in batches without reviewing patient files. One patient, Nick van Terheyden, was denied coverage for a $350 vitamin D test that his doctor had ordered. Former employees said the system saved Cigna billions of dollars but left patients with unexpected bills.
- Company involved
- Cigna
- AI system involved
- PXDX
10 source articles · read the reporting →
HireVue, Facing FTC Complaint From EPIC, Halts Use of Facial Recognition
Scored job candidates based on facial analysis and other biometric data, affecting their hiring prospects.
- Company involved
- HireVue
- AI system involved
- HireVue
1 source article · read the reporting →
Acclarent's AI-Powered Surgery Tool Allegedly Injures Patients, Lawsuits Claim
Two patients allege that Acclarent's TruDi Navigation System, which uses AI to confirm device positioning during sinus surgery, misled surgeons and caused severe injuries including strokes. The FDA has received over 100 reports of malfunctions and at least ten injuries since AI was added. Lawsuits claim the company rushed the technology to market with only 80% accuracy, making the device less safe than before. Acclarent denies any causal connection, and the cases are ongoing.
- Company involved
- Acclarent
- AI system involved
- TruDi Navigation System
2 source articles · read the reporting →
Acclarent's AI-Powered Surgery Tool Allegedly Injures Patients, Lawsuits Claim
The TruDi Navigation System by Acclarent, which uses AI to confirm device positions during sinus surgery, has been linked to at least ten patient injuries including strokes, according to FDA reports and two lawsuits. Plaintiffs Erin Ralph and Donna Fernihough allege that the AI was inaccurate and misled surgeons, causing carotid artery damage and strokes in 2022. Acclarent denies the allegations, stating there is no credible evidence of a causal connection. The lawsuits are ongoing.
- Company involved
- Acclarent
- AI system involved
- TruDi Navigation System
2 source articles · read the reporting →
Montefiore Allegedly Plans to Replace Nurses with AI for Insurance Reviews
The New York State Nurses Association alleges that Montefiore Health System plans to lay off 12 utilization review nurses and replace them with AI-powered software from Datavant. The union warns that the move could compromise patient care by having AI review insurance denials without clinical judgement. Montefiore denies the claims, stating they are investing in technology for better outcomes. The plan has sparked a town hall and calls from elected officials to halt the layoffs.
- Company involved
- Montefiore Health System
3 source articles · read the reporting →
CMS warns insurers against using AI to deny Medicare Advantage care
The Centers for Medicare & Medicaid Services (CMS) clarified that health insurers cannot use algorithms or AI to deny care to Medicare Advantage members. This follows lawsuits alleging UnitedHealth and Humana used an AI tool, nH Predict, to wrongfully deny post-acute care to elderly patients. The tool reportedly produced rigid estimates ignoring individual patient needs, leading to premature denials. CMS warned that non-compliance could result in penalties and increased audits.
- Company involved
- UnitedHealth, Humana
- AI system involved
- nH Predict
10 source articles · read the reporting →
Houston teachers sue over secret EVAAS evaluation algorithm
Houston Independent School District used the SAS Institute's EVAAS system, a proprietary algorithm, to evaluate teacher performance based on student test scores. The Houston Federation of Teachers and several teachers sued in 2014, alleging the system violated their Fourteenth Amendment procedural due process rights because the algorithm is a trade secret and teachers cannot verify the accuracy of their scores. A federal magistrate judge refused to dismiss the procedural due process claims in May 2017, ruling that teachers must have an opportunity to test the accuracy of the scores that could cost them their jobs.
- Company involved
- Houston Independent School District
- AI system involved
- Educational Value-Added Assessment System (EVAAS)
1 source article · read the reporting →
Royal Free London publishes audit into Streams app data processing
The Royal Free London NHS Foundation Trust published an audit into its use of the Streams app, following an investigation by the Information Commissioner's Office (ICO) in July 2017. The Streams app alerts clinicians to patients at risk of acute kidney injury. The audit, conducted by Linklaters, concluded that the trust's use of Streams was lawful and complied with data protection laws, although areas for improvement were identified. The ICO later recognised that the trust had completed all required actions.
- Company involved
- Royal Free London NHS Foundation Trust
- AI system involved
- Streams
10 source articles · read the reporting →
Teachers find no value in SAS EVAAS system in Southwest School District
A study examined the SAS Education Value-Added Assessment System (EVAAS®) used by the Southwest School District (SSD) to evaluate teacher effectiveness for high-stakes consequences. Teachers reported that the system produced inconsistent results, was biased by student factors, and reduced morale and collaboration. The study found that teachers did not use the data formatively as intended, and unintended consequences included heightened pressure and teaching to the test.
- Company involved
- Southwest School District (SSD)
- AI system involved
- SAS Education Value-Added Assessment System (EVAAS®)
10 source articles · read the reporting →
Eight Sleep Pod outage disrupts users' sleep after AWS failure
An AWS outage impacted Eight Sleep Pod users, disrupting their sleep as the smart bed's features became unavailable. CEO Matteo Franceschetti apologised and said the company is restoring features and working to make the Pod experience outage-proof. Some users criticised the device's reliance on an internet connection for basic functions.
- Company involved
- Eight Sleep
- AI system involved
- Eight Sleep Pod
3 source articles · read the reporting →
Nurse at St. Rose Dominican Hospital Averts AI-Sepsis Alert Error
A nurse at St. Rose Dominican Hospital in Henderson, Nevada, refused to follow an AI-generated sepsis alert that would have given an elderly dialysis patient IV fluids, which could have caused a life-threatening complication. The alert, part of the hospital's electronic system, prompted a charge nurse to order the fluids despite the patient's compromised kidneys. A physician intervened and ordered dopamine instead, averting harm. The incident highlights concerns about AI tools overriding clinical judgment.
- Company involved
- St. Rose Dominican Hospital
1 source article · read the reporting →
Texas AG Settles with Pieces Technologies Over Deceptive Healthcare AI Claims
The Texas Attorney General investigated Pieces Technologies, a Dallas-based healthcare AI company, for making false and misleading statements about the accuracy of its generative AI product used in hospitals. The company claimed an error rate of less than 1 per 100,000, but the investigation found these metrics were likely inaccurate. As part of the settlement, Pieces agreed to accurately disclose its product's accuracy and ensure hospital staff understand the appropriate reliance on its AI. The case marks the first-of-its-kind healthcare generative AI investigation by the AG.
- Company involved
- Pieces Technologies
4 source articles · read the reporting →
Google's Nightingale project transfers medical data of millions without patient knowledge
An anonymous Google whistleblower states that the company acquired medical data of 50 million patients from Ascension without their knowledge. The transfer, known as Project Nightingale, raised privacy and security concerns. The article reports that a federal inquiry was launched into whether HIPAA protections were followed.
- Company involved
- Google
10 source articles · read the reporting →
Kaiser Permanente doctor delivers terminal diagnosis via robot video screen
Ernest Quintana, 79, was admitted to Kaiser Permanente Medical Center in Fremont, California, with failing lungs. A doctor appeared on a telepresence robot's video screen and told him he might not make it home. The family was devastated that the news was delivered through a machine rather than in person. Kaiser Permanente issued a statement expressing regret and said it would review its practices.
- Company involved
- Kaiser Permanente
10 source articles · read the reporting →
Epic sepsis prediction algorithm misses two-thirds of cases, study finds
A study of nearly 30,000 patients at University of Michigan hospitals found that Epic Systems' sepsis prediction algorithm missed two-thirds of sepsis cases and frequently issued false alarms. The algorithm, used by over 100 US health systems, is designed to alert staff to potential sepsis, but the study suggests it provides little value and may contribute to alert fatigue. Epic disputed the study's conclusions, claiming its system has helped save lives.
- Company involved
- Epic Systems
- AI system involved
- Epic sepsis prediction model
6 source articles · read the reporting →
ExamSoft failure during Michigan Bar Exam
On July 28, 2020, ExamSoft's software platform failed during the Michigan Bar Exam, preventing some test takers from completing the exam. ExamSoft posted a statement on X (formerly Twitter) with the hashtag #MichiganBarExam, but critics noted the company deleted and reposted the statement multiple times, allegedly to hide negative comments. The incident affected a group of examinees and caused disruption to their ability to take the exam.
- Company involved
- ExamSoft
- AI system involved
- ExamSoft platform
10 source articles · read the reporting →
Hospitals use Epic AI to predict Covid-19 decline without validation
Dozens of hospitals across the US are using Epic's deterioration index AI system to predict which Covid-19 patients will become critically ill, despite the tool not being validated for the new disease. The rapid deployment during the pandemic bypassed normal testing and validation processes.
- AI system involved
- Deterioration index
9 source articles · read the reporting →
Google and HCA Healthcare partner to analyze 32 million patient records
Google Cloud has announced a partnership with HCA Healthcare to analyze around 32 million patient records. The anonymized data will be used to develop algorithms that could advise doctors on treatment options. Privacy advocates have raised concerns about the data sharing and the potential for AI to re-identify patients. HCA insists that patient-identifiable information will be stripped and that access will be tightly controlled.
- Company involved
- HCA Healthcare
- AI system involved
- Google Cloud
9 source articles · read the reporting →
University of Chicago report finds US healthcare algorithms rife with bias
A University of Chicago report alleges that algorithms used by hospitals, insurers and other businesses across the US are biased along racial and economic lines. The algorithms help triage patients, predict diabetes and flag those needing extra care, but the report says flawed products were introduced with little oversight. It says inequitable treatment has continued for more than a decade.
7 source articles · read the reporting →
Elon Musk asks X users to upload medical data to train Grok chatbot
Elon Musk has told X users to upload medical images to the platform so that the AI chatbot Grok can learn to interpret them. He claimed Grok can give medical diagnoses, sometimes better than doctors, but experts have raised concerns about accuracy and privacy. The article reports cases where Grok misread a tuberculosis case and a mammogram. xAI, which owns X, responded to Fortune: 'Legacy Media Lies.'
- Company involved
- X
- AI system involved
- Grok
6 source articles · read the reporting →
EPIC lawsuit challenges USPS secret surveillance program using facial recognition
The United States Postal Service's law enforcement wing, the Postal Inspection Service, ran a secret surveillance program called the Internet Covert Operations Program (iCOP) that used facial recognition from Clearview AI and social media monitoring tools to infiltrate online communities and monitor protests. The Electronic Privacy Information Center (EPIC) sued the USPS for failing to conduct and publish a privacy impact assessment as required by the E-Government Act. A federal district court dismissed the lawsuit on standing grounds, ruling that EPIC did not suffer a cognizable injury from the agency's refusal to disclose information about the program.
- Company involved
- United States Postal Service
- AI system involved
- Internet Covert Operations Program (iCOP)
10 source articles · read the reporting →
UnitedHealthcare sued for using AI to deny Medicare Advantage patients care
A class action complaint alleges that UnitedHealthcare and its subsidiary naviHealth deployed an artificial intelligence model to override treating physicians' determinations of medically necessary care for elderly Medicare Advantage patients. The complaint claims the AI model has a known 90% error rate and wrongfully denied patients care. The lawsuit, filed in the U.S. District Court for the District of Minnesota, seeks damages on behalf of the estates of two deceased patients and all others similarly situated.
- Company involved
- UnitedHealthcare, Inc.
10 source articles · read the reporting →
Google Health's diabetic retinopathy AI faced real-world issues in Thailand clinics
Google Health deployed a deep-learning system to screen for diabetic retinopathy in 11 clinics across Thailand. The AI was highly accurate in lab tests but rejected over a fifth of eye scans in real-world conditions due to poor lighting and slow internet. Patients whose scans were rejected had to visit specialists at other clinics, causing inconvenience and frustration for nurses. Google Health is now working with local staff to improve the system's workflow.
- Company involved
- Google Health
10 source articles · read the reporting →
