Google and HCA Healthcare partner to analyze 32 million patient records
Google Cloud has announced a partnership with HCA Healthcare to analyze around 32 million patient records. The anonymized data will be used to develop algorithms that could advise doctors on treatment options. Privacy advocates have raised concerns about the data sharing and the potential for AI to re-identify patients. HCA insists that patient-identifiable information will be stripped and that access will be tightly controlled.
- Company involved
- HCA Healthcare
- AI system involved
- Google Cloud
9 source articles · read the reporting →
Google Health's diabetic retinopathy AI faced real-world issues in Thailand clinics
Google Health deployed a deep-learning system to screen for diabetic retinopathy in 11 clinics across Thailand. The AI was highly accurate in lab tests but rejected over a fifth of eye scans in real-world conditions due to poor lighting and slow internet. Patients whose scans were rejected had to visit specialists at other clinics, causing inconvenience and frustration for nurses. Google Health is now working with local staff to improve the system's workflow.
- Company involved
- Google Health
10 source articles · read the reporting →
UnitedHealth used ALERT algorithm to limit mental health coverage, regulators found
ProPublica reports that UnitedHealth Group's Optum subsidiary used the ALERT algorithm to flag mental health patients and therapists as outliers, leading to therapy coverage denials. Regulators in California, Massachusetts and New York alleged this breached the federal Mental Health Parity and Addiction Equity Act, and UnitedHealth agreed to restrict the system in those jurisdictions. The company denies wrongdoing and says its programmes are compliant. Affected patients, including Medicaid enrollees, were left to pay out-of-pocket or go without care.
- Company involved
- UnitedHealth Group
- AI system involved
- ALERT
5 source articles · read the reporting →
NHS liver transplant algorithm causes longer waits for younger patients
An NHS computer algorithm that prioritises liver transplant patients most likely to die soon has led to younger patients waiting 156 days longer on average than those over 60. The algorithm, launched in 2018, gives a score based on 21 recipient and seven donor parameters. Patients like Sarah Meredith, 31, have been waiting two years for a transplant. NHS Blood and Transplant defends the system as saving lives but acknowledges the stress on younger patients.
- Company involved
- NHS Blood and Transplant
10 source articles · read the reporting →
Perth hospital group bans ChatGPT after doctor used it for a patient discharge summary
South Metropolitan Health Service, which runs five hospitals in Perth, ordered staff to stop using ChatGPT for work involving patient information after an email said some staff had used the AI chatbot to write medical notes. The service later clarified that only one doctor had used the tool to generate a patient discharge summary and that no confidential patient information had been breached. The Australian Medical Association has called for national regulations to ensure a human is involved in AI-assisted health decisions.
- Company involved
- South Metropolitan Health Service
- AI system involved
- ChatGPT
6 source articles · read the reporting →
EviCore denied heart catheterization for patient using algorithm
In fall 2021, Little John Cupp's doctor requested a left heart catheterization exam. EviCore, a company hired by UnitedHealthcare, denied the request twice using an algorithm called 'the dial' that adjusts thresholds for review. The algorithm flagged the request for review, and EviCore's doctors determined it was not medically necessary. Cupp did not receive the procedure and his symptoms continued.
- Company involved
- EviCore (a Cigna company)
- AI system involved
- the dial
6 source articles · read the reporting →
UNOS liver allocation policy reduces transplants in poorer states
The United Network for Organ Sharing (UNOS) implemented a new liver allocation policy in 2020 called the acuity circles system, which prioritizes the sickest patients regardless of location. An analysis by The Markup and The Washington Post found that the policy led to sharp declines in liver transplants in several Southern and Midwestern states and Puerto Rico, while increasing transplants in New York and California. Patients in affected states allege the system disadvantages poorer regions, and hospitals have sued to overturn the policy.
- Company involved
- United Network for Organ Sharing (UNOS)
- AI system involved
- Acuity circles liver allocation policy
10 source articles · read the reporting →
Teething problems in Mater Dei's medicine robots addressed
The Malta Union for Midwives and Nurses claimed that a €23 million investment in two computerised drug administration robots, Mario and Sophia, at Mater Dei Hospital had resulted in a complete failure. However, sources within the Health Ministry said that most teething problems have been addressed and that the supplier has not been paid yet. They reported that out of over 1,700 medication rounds, only four required a contingency plan.
- Company involved
- Mater Dei Hospital
- AI system involved
- Mario and Sophia
6 source articles · read the reporting →
ChatGPT Health fails to direct 52% of medical emergencies to emergency care in study
A study published in Nature Medicine found that OpenAI's ChatGPT Health tool under-triaged 52% of true medical emergencies, directing users to non-urgent care instead of emergency departments. The AI also misclassified 35% of non-urgent cases. Researchers at Mount Sinai conducted 960 tests across 60 clinical scenarios, noting the tool's susceptibility to anchoring bias when symptoms were minimized. The study highlights potential safety concerns as millions use AI for health guidance.
- Company involved
- OpenAI
- AI system involved
- ChatGPT Health
4 source articles · read the reporting →
Security Health Plan used AI to cut off nursing home care for 85-year-old woman
Frances Walter, an 85-year-old woman with a shattered shoulder, had her nursing home care payment cut off by Security Health Plan after an algorithm predicted she would recover in 16.6 days. The algorithm, nH Predict from NaviHealth, did not account for her severe pain and allergy to pain medicine. She was forced to spend her life savings and enroll in Medicaid while fighting the denial. A federal judge later ruled the denial was speculative and she was owed thousands of dollars.
- Company involved
- Security Health Plan
- AI system involved
- nH Predict
1 source article · read the reporting →
Hospitals Use OpenAI’s Whisper Despite Medical Transcription Hallucinations
OpenAI’s Whisper transcription tool is used by over 30,000 medical workers and can insert fabricated text into patient records. Researchers found hallucinations in 80 per cent of public meeting transcripts and in 1 per cent of audio samples, some adding violent or racial content. Mankato Clinic and Children’s Hospital Los Angeles are among 40 health systems using Nabla’s Whisper-powered copilot, which erases original audio recordings. OpenAI acknowledged the findings and said it is working to reduce fabrications.
- Company involved
- Mankato Clinic; Children’s Hospital Los Angeles
- AI system involved
- Whisper
2 source articles · read the reporting →
Brookdale Senior Living algorithm blamed for understaffing at assisted-living facilities
Managers at Brookdale Senior Living, the largest assisted-living chain in the US, allege that an algorithm called 'Service Alignment' set staffing levels so low that facilities were dangerously short-handed. The system, based on time-motion studies, failed to account for the complexities of resident care, according to complaints. Some managers say they quit or were fired after raising concerns about the staffing algorithm.
- Company involved
- Brookdale Senior Living
- AI system involved
- Service Alignment
1 source article · read the reporting →
University of Chicago Medical Center sued over sharing patient data with Google
The University of Chicago Medical Center is accused in a class-action lawsuit of sharing hundreds of thousands of patient medical records with Google without proper de-identification or patient consent. The data, from patients treated between 2009 and 2016, allegedly included dates and provider notes that could allow re-identification. The lawsuit claims Google sought the records to develop its own electronic health record system and predictive models. The hospital and Google deny wrongdoing, stating the research partnership was legal and compliant with HIPAA.
- Company involved
- University of Chicago Medical Center
1 source article · read the reporting →
Penn Medicine study: kidney algorithm reduced Black transplant access
For years, a clinical algorithm used for the national kidney transplant waitlist made Black patients appear healthier than they were, making it harder for them to secure transplant spots. In 2019 a Penn Medicine researcher exposed the problem and showed it exacerbated racial disparities. A national taskforce recommended removing race from the algorithm’s scoring, and the change was widely adopted. Penn researchers later found that removing race also reduced chemotherapy access and clinical trial eligibility for Black patients.
- AI system involved
- kidney function algorithm
1 source article · read the reporting →
U.S. hospitals' race-based eGFR calculation delayed Black kidney patients' transplants
For years, U.S. transplant hospitals used a race-based version of the eGFR equation to estimate Black patients' kidney function. The adjustment inflated Black patients' readings, delaying their referral to the kidney transplant waitlist and leaving some to die or deteriorate on dialysis. After years of advocacy, the United Network for Organ Sharing prohibited the race-based calculation in 2022 and required transplant programmes to revise affected patients' waiting times.
- Company involved
- U.S. transplant hospitals
- AI system involved
- eGFR (estimated glomerular filtration rate)
2 source articles · read the reporting →
VA's Suicide Prevention Algorithm Favours Men, Ignores Sexual Trauma
The Department of Veterans Affairs' REACH VET algorithm, designed to identify veterans at high risk of suicide, reportedly prioritises white men and excludes factors like military sexual trauma that disproportionately affect women. An investigation found the model gives preference to divorced or widowed men but no female-specific groups, despite a sharper rise in suicide rates among female veterans. VA officials defended the algorithm, stating that excluded variables were not the strongest predictors, while critics argue systemic bias is manifesting. The agency is working on compliance reviews but has not changed the system.
- Company involved
- Department of Veterans Affairs
- AI system involved
- REACH VET
2 source articles · read the reporting →
Hospital Denies Pain Medication to Patient Based on Narx Score
Elizabeth Amirault, a 34-year-old chronic pain patient, was denied narcotic pain medication during a hospital visit in Fort Wayne, Indiana, in August 2022. A nurse practitioner told her that her Narx Score, generated by Bamboo Health's NarxCare platform, was too high to prescribe opioids. The algorithm uses prescription drug monitoring data to assess risk of misuse, but critics say it can harm patients by cutting off needed care. Amirault believes her score was unfairly high due to her multiple surgeries and doctors.
- AI system involved
- NarxCare
3 source articles · read the reporting →
Capital Standard, LLC v. U.S. Bank National Association (CA Florida (2d)): AI-hallucinated content in court filing, Monetary Sanction; Adverse Costs…
The AI generated false legal citations that were included in a court filing, misleading the court.
1 source article · read the reporting →
Scottish Prison Service freed eight life prisoners due to risk assessment system error
The Scottish Prison Service's Level of Service and Case Management System failed to update risk assessments when new information was entered, leading to incorrect risk scores for 495 closed cases and 285 open cases. As a result, eight prisoners serving life for murder or rape were granted temporary release without accurate risk assessments. Justice Secretary Keith Brown acknowledged the error and stated that the cases would be reviewed. Opposition politicians called for an apology, claiming the releases put the public at risk.
- Company involved
- Scottish Prison Service
- AI system involved
- Level of Service and Case Management System
10 source articles · read the reporting →
Lawsuit: AI Illegally Recorded Doctor-Patient Encounters - BankInfoSecurity
The system recorded and transcribed patient-doctor conversations without patients' informed consent.
- Company involved
- Sutter Health and MemorialCare
- AI system involved
- Abridge AI platform
1 source article · read the reporting →
