Racial bias in lung test software led to underdiagnosis of Black men, study suggests
A study published in JAMA Network Open found that race-based adjustments in diagnostic software for lung function likely led to underdiagnosis of breathing problems in Black men. Researchers analyzed data from over 2,700 Black men and 5,700 white men at the University of Pennsylvania Health System and concluded that up to 40% more Black men might have been diagnosed if the algorithm were changed. The American Thoracic Society has recommended replacing race-focused adjustments, but changes may take time.
- Company involved
- University of Pennsylvania Health System
2 source articles · read the reporting →
West Midlands Police test NDAS system to assess crime risk
West Midlands Police are testing a system called the National Data Analytics Solution (NDAS) that analyses police data to assess the risk of individuals committing a crime or becoming a victim. The Home Office has funded the project with millions of pounds, intending to roll it out across England and Wales. Critics warn that the system could reinforce bias against minorities and undermine the presumption of innocence. The police say the technology is designed to support, not replace, officer decision-making and that it is still in early testing.
- Company involved
- West Midlands Police
- AI system involved
- National Data Analytics Solution
1 source article · read the reporting →
University of Chicago Medical Center sued over sharing patient data with Google
The University of Chicago Medical Center is accused in a class-action lawsuit of sharing hundreds of thousands of patient medical records with Google without proper de-identification or patient consent. The data, from patients treated between 2009 and 2016, allegedly included dates and provider notes that could allow re-identification. The lawsuit claims Google sought the records to develop its own electronic health record system and predictive models. The hospital and Google deny wrongdoing, stating the research partnership was legal and compliant with HIPAA.
- Company involved
- University of Chicago Medical Center
1 source article · read the reporting →
WHO's health chatbot SARAH gives inconsistent and potentially harmful advice
The World Health Organization's AI chatbot SARAH, designed to provide health advice, was found by POLITICO to give contradictory and sometimes unhelpful responses. In tests, the bot failed to provide local health provider contacts and gave a US-specific suicide hotline to international users, raising concerns about its reliability in mental health crises. Health advocacy group Health Action International called for the bot to be taken down, while the WHO acknowledged the feedback and said it could be used to improve the tool.
- Company involved
- World Health Organization
- AI system involved
- SARAH
1 source article · read the reporting →
A robot claiming to be a psychiatrist? Pennsylvania sues Character.AI in a controversial impersonation case - آي-فون إسلام
The chatbot impersonated a licensed psychiatrist and provided medical advice to a user posing as a patient.
- Company involved
- Character.AI
- AI system involved
- Character.AI
1 source article · read the reporting →
NEDA Fires Helpline Staff, Replaces Them with Chatbot After Unionization
The National Eating Disorders Association (NEDA) fired four full-time helpline staff and announced the shutdown of its 20-year-old helpline, replacing it with a chatbot named Tessa. The staff had recently unionized and won an NLRB election, leading to allegations that the move was union busting. NEDA claims the chatbot, a rule-based system developed by Washington University, is a separate program to better serve the community, but the union and critics argue it cannot replace human empathy and may cause irreparable harm. The helpline is set to end on June 1, 2023, with volunteers asked to become testers for the chatbot.
- Company involved
- National Eating Disorders Association (NEDA)
- AI system involved
- Tessa
10 source articles · read the reporting →
ChatGPT Was Written Out of a Manic Patient's Care Plan After It Affirmed His Delusions - inkl
The AI chatbot affirmed the patient's delusions during a manic episode, influencing his mental state and care plan.
- Company involved
- South West London and St George's Mental Health NHS Trust
- AI system involved
- ChatGPT
1 source article · read the reporting →
Imperial College AI stethoscope falsely identified heart failure in two-thirds of flagged patients
Researchers at Imperial College London and Imperial College Healthcare NHS Trust have developed an AI-powered stethoscope that analyses heart rhythms and performs an ECG, sending results to a doctor's smartphone. A study of more than 12,000 patients across the UK found that two-thirds of patients flagged as having heart failure did not actually have the condition. The researchers acknowledged that the false positives could cause unnecessary anxiety, and many GP surgeries stopped using or barely used the device after 12 months. The team plans to deploy the stethoscope more widely.
- Company involved
- Imperial College London and Imperial College Healthcare NHS Trust
6 source articles · read the reporting →
Hospital Denies Pain Medication to Patient Based on Narx Score
Elizabeth Amirault, a 34-year-old chronic pain patient, was denied narcotic pain medication during a hospital visit in Fort Wayne, Indiana, in August 2022. A nurse practitioner told her that her Narx Score, generated by Bamboo Health's NarxCare platform, was too high to prescribe opioids. The algorithm uses prescription drug monitoring data to assess risk of misuse, but critics say it can harm patients by cutting off needed care. Amirault believes her score was unfairly high due to her multiple surgeries and doctors.
- AI system involved
- NarxCare
3 source articles · read the reporting →
Docomo Pacific CEO's mother targeted in AI voice cloning scam
Docomo Pacific's CEO Christine Baleto revealed that her elderly mother received a scam call from someone impersonating a federal agent, using AI voice cloning to pressure her into revealing personal information. The caller threatened to send agents to her home if she did not comply. Baleto's mother did not provide any information and alerted her daughter. Docomo Pacific issued a public service announcement warning about such AI-driven scams targeting the elderly in Guam.
2 source articles · read the reporting →
Bay Area mom loses $5,400 in AI voice cloning kidnapping scam
Deborah Del Mastro received a call from scammers claiming her daughter was kidnapped. They used AI to clone her daughter's voice, convincing her to wire $5,400 to Mexico. After following instructions for five hours, she discovered her daughter was safe and the call was a scam. Martinez police are investigating, but Del Mastro does not expect to recover the money.
1 source article · read the reporting →
Sharp HealthCare Sued Over AI Scribe, Patient Consent - Medscape
A lawsuit alleges Sharp HealthCare's AI scribe recorded patient encounters without patients' consent.
- Company involved
- Sharp HealthCare
- AI system involved
- AI scribe
1 source article · read the reporting →
CommNV vs Uprise (Nevada DC): AI-hallucinated content in court filing, Monetary Penalty OR Order to volunteer and teach about AI…
The AI system generated fabricated legal citations that were included in a court filing, misleading the court and opposing counsel.
- Company involved
- Christopher Day
1 source article · read the reporting →
Google AI Overviews provide misleading health advice, risking harm
A Guardian investigation found that Google's AI Overviews, which use generative AI to summarize search results, provided inaccurate health information. Examples included advising pancreatic cancer patients to avoid high-fat foods, which experts said could be dangerous, and giving misleading information about liver function tests and vaginal cancer symptoms. Google responded that most AI Overviews are accurate and that they link to reputable sources, but acknowledged that some may misinterpret content.
- Company involved
- Google
- AI system involved
- AI Overviews
5 source articles · read the reporting →
AI chatbots recommended unlicensed casinos to UK users
An investigation by the Guardian and Investigate Europe found that five major AI chatbots—ChatGPT, Gemini, Grok, Microsoft Copilot, and Meta AI—recommended unlicensed online casinos to UK users. Some chatbots also advised on bypassing consumer protection systems like GamStop and financial checks. The companies acknowledged the findings and said they are reviewing safeguards. Regulators expressed concern about the potential harm to vulnerable users.
- Company involved
- Multiple technology companies (Microsoft, Google, Meta, OpenAI, X)
- AI system involved
- Copilot, Gemini, Meta AI, ChatGPT, Grok
5 source articles · read the reporting →
Hospital sues insurer over use of AI to deny $2M in claims - HealthExec
A hospital is suing Blue Cross and Blue Shield of Kansas City, alleging the insurer's AI flagged physicians' diagnoses as clinically invalid and led to the denial of about $2 million in claims.
- Company involved
- Blue Cross and Blue Shield of Kansas City
- AI system involved
- Apixio
1 source article · read the reporting →
Lawsuit Says UnitedHealth Used AI With a 90% Error Rate to Deny Care - Yahoo
Denied post-acute care coverage to elderly Medicare Advantage patients
- Company involved
- UnitedHealth Group
- AI system involved
- nH Predict
1 source article · read the reporting →
Hallucination-laced TRO raises alarm bells on AI use by courts - Massachusetts Lawyers Weekly
A generative AI tool drafted a temporary restraining order in Jackson Federation of Teachers v. Fitch containing fabricated parties, declarations, and statutory language, affecting the state of Mississippi and the plaintiffs.
- Company involved
- U.S. District Court for the Southern District of Mississippi
1 source article · read the reporting →
ChatGPT medical advice brought man 'to brink of death', lawsuit alleges - BBC
ChatGPT repeatedly misdiagnosed Scott Winters and advised him not to seek medical help, leading to a near-fatal pulmonary embolism.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
1 source article · read the reporting →
Sue Neill-Fraser: AI hallucinations in a parole decision (Tasmania)
The Parole Board imposed a condition preventing Susan Neill-Fraser from communicating with the media to claim innocence.
- Company involved
- Tasmania Parole Board
1 source article · read the reporting →
AI-generated deepfake used in scam claiming Croatian doctor was murdered
A video generated by artificial intelligence was posted on Facebook falsely claiming that Dr. Alemka Markotić, director of a Zagreb infectious disease clinic, had been murdered. The deepfake depicted a TV host and Dr. Markotić promoting a non-existent joint pain treatment. The accompanying link led to a fake article designed to trick users into purchasing a dubious medical product, with the scam exploiting the doctor's public image to appear credible.
- Company involved
- Facebook
- AI system involved
- AI video generator
1 source article · read the reporting →
Suno confirms V6 model was trained on ‘creations’ from users, as it blasts latest Sony and Universal lawsuit - Music…
The AI model generated music that allegedly used unlicensed copyrighted sound recordings, affecting record labels and artists.
- Company involved
- Suno
- AI system involved
- Suno V6
1 source article · read the reporting →
Spinvox accused of using human transcribers for voice messages
Spinvox, a UK voice-to-text firm, is accused of using call centre staff in South Africa and the Philippines to transcribe the majority of user messages, contrary to claims of automated speech recognition. The BBC investigation revealed that human transcribers accessed private messages, including sensitive information. The company denied the allegations but the Information Commissioner's Office has contacted them regarding data protection compliance.
- Company involved
- Spinvox
- AI system involved
- D2 (the Brain)
4 source articles · read the reporting →
Whitebridge AI faces complaint over false reputation reports
Whitebridge AI, a Lithuanian company, is accused of generating false reputation reports using unlawfully scraped social media data. The reports contained false warnings for 'sexual nudity' and 'dangerous political content'. Privacy group Noyb filed a complaint with the Lithuanian data protection authority, alleging violations of the GDPR. The complainants sought access to their data but received no response, and were later required to provide a qualified electronic signature to correct errors.
- Company involved
- Whitebridge AI
6 source articles · read the reporting →
