Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.
Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.
92 incidents closest to “Emmie for Patients” · matched on meaning · public reporting
The AI generated fictitious legal citations that were submitted to the court in a legal filing.
1 source article · read the reporting →
AI-generated fake legal citations were submitted to the court, affecting the plaintiff's appeal and the defendant's legal costs.
1 source article · read the reporting →
South Metropolitan Health Service, which runs five hospitals in Perth, ordered staff to stop using ChatGPT for work involving patient information after an email said some staff had used the AI chatbot to write medical notes. The service later clarified that only one doctor had used the tool to generate a patient discharge summary and that no confidential patient information had been breached. The Australian Medical Association has called for national regulations to ensure a human is involved in AI-assisted health decisions.
6 source articles · read the reporting →
The article reports that scientists are questioning the reliability of the QCovid risk prediction algorithm used by the NHS to prioritise COVID-19 vaccinations. Due to missing data on weight or ethnicity, the algorithm automatically assigns a high BMI and high-risk ethnicity, leading to young, healthy patients being incorrectly labelled as high risk and called for early vaccination. GPs have identified hundreds of such cases, causing anxiety and potentially delaying vaccinations for more vulnerable individuals. NHS Digital acknowledged the issue and said clinicians can remove patients from the shielding list.
10 source articles · read the reporting →
The AI-generated legal briefs with hallucinated cases were used to deny Hannah Payne's motion for a new trial.
1 source article · read the reporting →
A study published in npj Digital Medicine tested four commercial large language models (Bard, ChatGPT, GPT-4, and Claude) for their tendency to propagate discredited race-based medical beliefs. When asked about kidney function, lung capacity, and skin thickness, the models sometimes endorsed debunked racial differences, particularly affecting Black patients. The study concludes that these biases pose a potential hazard and urges caution before using such models in clinical decision-making.
6 source articles · read the reporting →
In fall 2021, Little John Cupp's doctor requested a left heart catheterization exam. EviCore, a company hired by UnitedHealthcare, denied the request twice using an algorithm called 'the dial' that adjusts thresholds for review. The algorithm flagged the request for review, and EviCore's doctors determined it was not medically necessary. Cupp did not receive the procedure and his symptoms continued.
6 source articles · read the reporting →
Health insurer Humana is accused in a class-action lawsuit of using an AI algorithm to systematically deny rehabilitation care to Medicare Advantage patients, despite recommendations from their doctors. The lawsuit, filed on December 12, 2023, alleges that the AI tool restricted medically necessary care. This is the second major health insurer to face legal action over its use of AI to deny care.
9 source articles · read the reporting →
A study by researchers at Brigham and Women's Hospital found that ChatGPT, developed by OpenAI, generated cancer treatment plans with errors. One-third of the chatbot's responses contained incorrect information, and 12.5% were hallucinated. The study was published in JAMA Oncology and reported by Bloomberg.
10 source articles · read the reporting →
A study presented at the ASHP Midyear Clinical Meeting found that ChatGPT's responses to nearly three-quarters of drug-related questions were incomplete or inaccurate. The AI system also generated fake citations to support some responses. Researchers warned that healthcare professionals and patients should verify ChatGPT's medication information using trusted sources to avoid potential harm.
8 source articles · read the reporting →
The system denied necessary medical care to Medicare Advantage beneficiaries.
1 source article · read the reporting →
A study published in JAMA Pediatrics found that ChatGPT version 3.5 provided incorrect diagnoses for 83 out of 100 pediatric case challenges. The chatbot's errors included both completely wrong diagnoses and diagnoses that were too broad. The researchers noted that the chatbot failed to identify relationships such as that between autism and vitamin deficiencies, and suggested that more selective training is needed to improve accuracy.
7 source articles · read the reporting →
In January 2021, the California Employment Development Department used Thompson Reuters automated batch review software to flag 1.1 million unemployment claims as potentially fraudulent. EDD stopped payments on those claims without prior notice. Later, over 600,000 were confirmed as legitimate after claimants used ID.me to verify their identity. The incident highlights the trade-off between fraud prevention and timely benefit access.
7 source articles · read the reporting →
EvenUp, a legal tech startup valued at $1 billion, uses AI to draft personal injury demand letters. Former employees revealed that the AI system frequently makes errors, including missing injuries and fabricating medical conditions. The company defends its hybrid approach with human oversight, but critics allege overpromised AI capabilities.
6 source articles · read the reporting →
An African American man has filed a lawsuit in federal court alleging that the algorithm used to determine kidney transplant priority is racially biased. The plaintiff, a Los Angeles barber, claims the algorithm puts Black patients lower on the waitlist, denying them timely access to organs. He seeks damages and a higher position on the list.
7 source articles · read the reporting →
The Malta Union for Midwives and Nurses claimed that a €23 million investment in two computerised drug administration robots, Mario and Sophia, at Mater Dei Hospital had resulted in a complete failure. However, sources within the Health Ministry said that most teething problems have been addressed and that the supplier has not been paid yet. They reported that out of over 1,700 medication rounds, only four required a contingency plan.
6 source articles · read the reporting →
The UK's National Health Service announced plans to use AI to automatically generate notes from patient appointments. Health Secretary Victoria Atkins said the scheme would reduce admin time. Privacy campaigners raised concerns about data security and accuracy, citing an incident where AI misheard the chief medical officer's name. The Department of Health and Social Care stated that patient confidentiality remains a top priority.
3 source articles · read the reporting →
AEMPS launched MeQA, an artificial intelligence tool for answering public questions about medicines, on 13 May 2025. Two days later it withdrew the tool after detecting that some responses contained errors. The agency said that most answers were correct but that the errors could affect patient safety, and that it would restore the service as soon as possible.
4 source articles · read the reporting →
A U.S. District Court judge ruled that Tennessee's TennCare Connect system, built by Deloitte for over $400 million, illegally denied thousands of low-income residents and people with disabilities Medicaid and disability benefits due to programming and data errors. The system automatically terminated coverage without properly considering eligibility for all available programs. A class action lawsuit filed in 2020 resulted in the ruling.
10 source articles · read the reporting →
Advocacy groups filed a complaint with the Federal Trade Commission alleging that Deloitte's eligibility software, TIERS, used by Texas Medicaid, wrongly disenrolled qualified recipients due to glitches. Nearly 1.8 million Texans lost coverage after the pandemic pause ended, with many errors attributed to procedural issues but some linked to system malfunctions. Deloitte denies the claims, while the state says it restored care for at least 90,000 people. The FTC has not yet responded to the complaint.
6 source articles · read the reporting →
An elderly woman sought advice from AskVet's AI chatbot VERA for her dog's diarrhoea. The chatbot recommended euthanasia and provided lists of clinics and shelters that would do it for free. The woman followed the advice and euthanized her dog. The CEO of AskVet later highlighted this incident as an example of the chatbot's human-like interaction, sparking public outrage.
5 source articles · read the reporting →
Michel Janse, a 27-year-old Christian social media influencer, discovered during her honeymoon that her likeness had been used without consent in a YouTube advertisement for erectile dysfunction supplements. The ad was created using AI to clone her face and voice. Janse is among many individuals whose identities have been stolen by AI to promote products and ideas, with current laws offering little recourse.
2 source articles · read the reporting →
Two women in Ho Chi Minh City, Vietnam, stopped taking prescribed medications for diabetes and high cholesterol after following advice from OpenAI's ChatGPT. This led to severe health complications, including dangerously high blood sugar and signs of myocardial ischemia, requiring hospitalization. The treating doctor warned that AI cannot replace medical professionals and that self-medicating based on AI advice can be life-threatening.
1 source article · read the reporting →
PimEyes, a Polish facial recognition search engine, allows users to upload a photo and find matching images from across the internet. The system scrapes billions of photos and creates biometric profiles without consent. A banker named Dylan (pseudonym) discovered that an old party photo of him was found by the system, potentially outing his private life. Privacy advocates and platforms like Instagram and YouTube are taking action against PimEyes, alleging violations of the GDPR.
6 source articles · read the reporting →
