IRCC uses AI triage for Temporary Resident Visa applications
Immigration, Refugees and Citizenship Canada (IRCC) uses an AI system called Advanced Analytics to triage Temporary Resident Visa applications from India and China. The system categorizes applications into tiers, with Tier 1 approved automatically and others sent to human officers. Critics allege the system lacks transparency and may introduce bias, leading to visa refusals without clear rationale. The author, a Canadian immigration lawyer, is filing Federal Court cases on behalf of clients affected by refusals.
- Company involved
- Immigration, Refugees and Citizenship Canada (IRCC)
- AI system involved
- Advanced Analytics Triage of Overseas Temporary Resident Visa Applications
10 source articles · read the reporting →
Humana sued for using AI to deny seniors rehabilitation care
Health insurer Humana is accused in a class-action lawsuit of using an AI algorithm to systematically deny rehabilitation care to Medicare Advantage patients, despite recommendations from their doctors. The lawsuit, filed on December 12, 2023, alleges that the AI tool restricted medically necessary care. This is the second major health insurer to face legal action over its use of AI to deny care.
- Company involved
- Humana
9 source articles · read the reporting →
ChatGPT generates error-filled cancer treatment plans, study finds
A study by researchers at Brigham and Women's Hospital found that ChatGPT, developed by OpenAI, generated cancer treatment plans with errors. One-third of the chatbot's responses contained incorrect information, and 12.5% were hallucinated. The study was published in JAMA Oncology and reported by Bloomberg.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
10 source articles · read the reporting →
Study finds ChatGPT provides inaccurate drug information responses
A study presented at the ASHP Midyear Clinical Meeting found that ChatGPT's responses to nearly three-quarters of drug-related questions were incomplete or inaccurate. The AI system also generated fake citations to support some responses. Researchers warned that healthcare professionals and patients should verify ChatGPT's medication information using trusted sources to avoid potential harm.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
8 source articles · read the reporting →
Cigna StressWaves Test found unreliable and invalid in independent study
A study published in Scientific Reports evaluated the Cigna StressWaves Test (CSWT), an AI tool that claims to assess psychological stress from speech. The study found that the CSWT had poor test-retest reliability and poor validity compared to the Perceived Stress Scale. The authors warned that widespread availability of the tool could lead to misleading results and negative consequences for users making healthcare decisions. Cigna has not publicly responded to the findings.
- Company involved
- Cigna
- AI system involved
- Cigna StressWaves Test
4 source articles · read the reporting →
ChatGPT 3.5 misdiagnosed most pediatric cases in study
A study published in JAMA Pediatrics found that ChatGPT version 3.5 provided incorrect diagnoses for 83 out of 100 pediatric case challenges. The chatbot's errors included both completely wrong diagnoses and diagnoses that were too broad. The researchers noted that the chatbot failed to identify relationships such as that between autism and vitamin deficiencies, and suggested that more selective training is needed to improve accuracy.
- Company involved
- OpenAI
- AI system involved
- ChatGPT version 3.5
7 source articles · read the reporting →
Black man alleges racial bias in kidney transplant algorithm
An African American man has filed a lawsuit in federal court alleging that the algorithm used to determine kidney transplant priority is racially biased. The plaintiff, a Los Angeles barber, claims the algorithm puts Black patients lower on the waitlist, denying them timely access to organs. He seeks damages and a higher position on the list.
7 source articles · read the reporting →
UNOS liver allocation policy reduces transplants in poorer states
The United Network for Organ Sharing (UNOS) implemented a new liver allocation policy in 2020 called the acuity circles system, which prioritizes the sickest patients regardless of location. An analysis by The Markup and The Washington Post found that the policy led to sharp declines in liver transplants in several Southern and Midwestern states and Puerto Rico, while increasing transplants in New York and California. Patients in affected states allege the system disadvantages poorer regions, and hospitals have sued to overturn the policy.
- Company involved
- United Network for Organ Sharing (UNOS)
- AI system involved
- Acuity circles liver allocation policy
10 source articles · read the reporting →
NHS plans AI to listen to appointments and generate notes
The UK's National Health Service announced plans to use AI to automatically generate notes from patient appointments. Health Secretary Victoria Atkins said the scheme would reduce admin time. Privacy campaigners raised concerns about data security and accuracy, citing an incident where AI misheard the chief medical officer's name. The Department of Health and Social Care stated that patient confidentiality remains a top priority.
- Company involved
- National Health Service (NHS)
3 source articles · read the reporting →
Anthropic ordered to respond over alleged AI-hallucinated citation in court filing
A US federal magistrate judge has ordered Anthropic to respond to music publishers' claim that a court filing by Anthropic data scientist Olivia Chen cited a fictitious academic article that may have been generated by Anthropic's AI tool Claude. The publishers' lawyer said he had confirmed with the named author and journal that the article did not exist. Anthropic's lawyer disputed this, saying it was a mis-citation rather than an AI hallucination. The filing was made in an ongoing copyright case brought by Universal Music Group, Concord, and ABKCO against Anthropic over the use of song lyrics to train Claude.
- Company involved
- Anthropic
- AI system involved
- Claude
5 source articles · read the reporting →
AEMPS withdraws AI medicines tool MeQA after detecting errors
AEMPS launched MeQA, an artificial intelligence tool for answering public questions about medicines, on 13 May 2025. Two days later it withdrew the tool after detecting that some responses contained errors. The agency said that most answers were correct but that the errors could affect patient safety, and that it would restore the service as soon as possible.
- Company involved
- Agencia Española de Medicamentos y Productos Sanitarios (AEMPS)
- AI system involved
- MeQA
4 source articles · read the reporting →
Google AI Overviews generate erroneous search summaries
In May 2024, Google launched AI Overviews, a feature in Search that generates AI-powered summaries. Shortly after, users reported odd and erroneous overviews for some queries, including satirical or nonsense results. Google acknowledged the issues in a blog post and stated they made more than a dozen technical improvements to reduce inaccuracies. The company said that less than one in 7 million queries resulted in a content policy violation.
- Company involved
- Google
- AI system involved
- AI Overviews
10 source articles · read the reporting →
Audit of LAION-400M finds sexual violence, racial slurs, and stereotypes in dataset
An audit of the LAION-400M dataset by Abeba Birhane and colleagues at University College Dublin and University of Edinburgh found that its automated curation using CLIP failed to remove sexually explicit images, racial slurs, and stereotypes. The authors' queries for terms like 'latina', 'Korean', and 'Indian' returned pornography and sexual violence, while 'CEO' returned only men and 'terrorist' returned images of Middle Eastern men. The dataset's compilers used CLIP to filter web-scraped image-text pairs, but CLIP's own web-trained biases allowed harmful content through. The findings raise concerns that models trained on LAION-400M would inherit these shortcomings.
- Company involved
- LAION-400M team
- AI system involved
- LAION-400M
7 source articles · read the reporting →
Meta's cross-check program delays removal of violating content for privileged users
The Oversight Board's policy advisory opinion on Meta's cross-check program found that the system grants certain users, such as business partners and celebrities, additional human review before removing violating content, while ordinary users face immediate removal. This unequal treatment allows potentially harmful content to remain on the platform for days, and Meta has failed to track whether the program improves accuracy. The Board made 32 recommendations to address these flaws.
- Company involved
- Meta
- AI system involved
- cross-check program
10 source articles · read the reporting →
Audit of RisCanvi finds biases and reliability issues in criminal justice system
Eticas conducted an adversarial audit of RisCanvi, an AI risk assessment tool used in Catalonia's criminal justice system. The audit uncovered biases in risk classifications against specific demographics and significant reliability issues. The findings call for fairer practices in criminal justice AI.
- Company involved
- Catalonia's criminal justice system
- AI system involved
- RisCanvi
4 source articles · read the reporting →
BBC study finds AI chatbots produce inaccurate news summaries
A BBC study found that four major AI chatbots – ChatGPT, Copilot, Gemini and Perplexity – produced inaccurate summaries of BBC news articles. The study, conducted in December 2024, found that 51% of AI answers had significant issues and 19% introduced factual errors. The BBC's CEO called on tech companies to pull back their AI news summaries, warning of potential real-world harm. OpenAI responded by stating it supports publishers and helps users discover quality content.
- Company involved
- OpenAI, Microsoft, Google, Perplexity
- AI system involved
- ChatGPT, Copilot, Gemini, Perplexity
5 source articles · read the reporting →
OpenAI Estimates Hundreds of Thousands of ChatGPT Users May Experience Mental Health Crises Weekly
OpenAI released estimates that around 0.07% of active ChatGPT users show signs of psychosis or mania weekly, and 0.15% express suicidal ideation. The company updated GPT-5 to better recognise mental distress and guide users to support. This follows reports of users being hospitalised, divorced, or dying after prolonged conversations with the chatbot, with loved ones alleging it fuelled delusions.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
4 source articles · read the reporting →
Two women hospitalized after stopping medication on ChatGPT's advice
Two women in Ho Chi Minh City, Vietnam, stopped taking prescribed medications for diabetes and high cholesterol after following advice from OpenAI's ChatGPT. This led to severe health complications, including dangerously high blood sugar and signs of myocardial ischemia, requiring hospitalization. The treating doctor warned that AI cannot replace medical professionals and that self-medicating based on AI advice can be life-threatening.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
1 source article · read the reporting →
ChatGPT Health fails to direct 52% of medical emergencies to emergency care in study
A study published in Nature Medicine found that OpenAI's ChatGPT Health tool under-triaged 52% of true medical emergencies, directing users to non-urgent care instead of emergency departments. The AI also misclassified 35% of non-urgent cases. Researchers at Mount Sinai conducted 960 tests across 60 clinical scenarios, noting the tool's susceptibility to anchoring bias when symptoms were minimized. The study highlights potential safety concerns as millions use AI for health guidance.
- Company involved
- OpenAI
- AI system involved
- ChatGPT Health
4 source articles · read the reporting →
Instagram urged to remove pro-anorexia content harmful to vulnerable users
The Guardian reported that thousands of pro-anorexia hashtags and accounts were easily searchable on Instagram. Eating disorder charities and clinicians warned this content could perpetuate eating disorders and called for stricter moderation. Instagram had recently banned graphic self-harm images and said it was continuing to review its policies. The article noted that workarounds allowed users to find images despite a 2012 ban on some high-risk search terms.
- Company involved
- Instagram
- AI system involved
- Instagram
1 source article · read the reporting →
Security Health Plan used AI to cut off nursing home care for 85-year-old woman
Frances Walter, an 85-year-old woman with a shattered shoulder, had her nursing home care payment cut off by Security Health Plan after an algorithm predicted she would recover in 16.6 days. The algorithm, nH Predict from NaviHealth, did not account for her severe pain and allergy to pain medicine. She was forced to spend her life savings and enroll in Medicaid while fighting the denial. A federal judge later ruled the denial was speculative and she was owed thousands of dollars.
- Company involved
- Security Health Plan
- AI system involved
- nH Predict
1 source article · read the reporting →
Researchers find bias in chest X-ray AI classifiers against women, Hispanic and Medicaid patients
A study by the University of Toronto, Vector Institute, and MIT found that AI classifiers trained on public chest X-ray datasets exhibited racial, gender, and socioeconomic bias. Female patients, Hispanic patients, and those with Medicaid insurance were disproportionately affected, with the classifiers often providing incorrect diagnoses for Medicaid patients. The researchers attributed the bias to imbalanced training data and called for rigorous fairness analyses before clinical deployment.
2 source articles · read the reporting →
Hospitals Use OpenAI’s Whisper Despite Medical Transcription Hallucinations
OpenAI’s Whisper transcription tool is used by over 30,000 medical workers and can insert fabricated text into patient records. Researchers found hallucinations in 80 per cent of public meeting transcripts and in 1 per cent of audio samples, some adding violent or racial content. Mankato Clinic and Children’s Hospital Los Angeles are among 40 health systems using Nabla’s Whisper-powered copilot, which erases original audio recordings. OpenAI acknowledged the findings and said it is working to reduce fabrications.
- Company involved
- Mankato Clinic; Children’s Hospital Los Angeles
- AI system involved
- Whisper
2 source articles · read the reporting →
Hyderabad doctors warn after ChatGPT advice causes patient harm
In Hyderabad, a 30-year-old kidney transplant patient discontinued antibiotics on ChatGPT's advice, losing her transplanted kidney and returning to dialysis. A 62-year-old diabetic man suffered weight loss and low sodium after following a ChatGPT diet plan. Doctors warn that AI tools lack clinical judgment and should not replace professional medical consultation.
- AI system involved
- ChatGPT
5 source articles · read the reporting →
