Audit of LAION-400M finds sexual violence, racial slurs, and stereotypes in dataset
An audit of the LAION-400M dataset by Abeba Birhane and colleagues at University College Dublin and University of Edinburgh found that its automated curation using CLIP failed to remove sexually explicit images, racial slurs, and stereotypes. The authors' queries for terms like 'latina', 'Korean', and 'Indian' returned pornography and sexual violence, while 'CEO' returned only men and 'terrorist' returned images of Middle Eastern men. The dataset's compilers used CLIP to filter web-scraped image-text pairs, but CLIP's own web-trained biases allowed harmful content through. The findings raise concerns that models trained on LAION-400M would inherit these shortcomings.
- Company involved
- LAION-400M team
- AI system involved
- LAION-400M
7 source articles · read the reporting →
DeepSeek's R1 chatbot failed to block any jailbreak prompts in security tests
Security researchers from Cisco and the University of Pennsylvania tested 50 well-known jailbreak prompts against DeepSeek's R1 reasoning model. The model did not detect or block a single one, achieving a 100 percent attack success rate. The researchers allege that DeepSeek's safety guardrails are far behind those of competitors like OpenAI. DeepSeek did not respond to requests for comment.
- Company involved
- DeepSeek
- AI system involved
- DeepSeek R1
3 source articles · read the reporting →
Audit of RisCanvi finds biases and reliability issues in criminal justice system
Eticas conducted an adversarial audit of RisCanvi, an AI risk assessment tool used in Catalonia's criminal justice system. The audit uncovered biases in risk classifications against specific demographics and significant reliability issues. The findings call for fairer practices in criminal justice AI.
- Company involved
- Catalonia's criminal justice system
- AI system involved
- RisCanvi
4 source articles · read the reporting →
OpenAI Estimates Hundreds of Thousands of ChatGPT Users May Experience Mental Health Crises Weekly
OpenAI released estimates that around 0.07% of active ChatGPT users show signs of psychosis or mania weekly, and 0.15% express suicidal ideation. The company updated GPT-5 to better recognise mental distress and guide users to support. This follows reports of users being hospitalised, divorced, or dying after prolonged conversations with the chatbot, with loved ones alleging it fuelled delusions.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
4 source articles · read the reporting →
Two women hospitalized after stopping medication on ChatGPT's advice
Two women in Ho Chi Minh City, Vietnam, stopped taking prescribed medications for diabetes and high cholesterol after following advice from OpenAI's ChatGPT. This led to severe health complications, including dangerously high blood sugar and signs of myocardial ischemia, requiring hospitalization. The treating doctor warned that AI cannot replace medical professionals and that self-medicating based on AI advice can be life-threatening.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
1 source article · read the reporting →
ChatGPT Health fails to direct 52% of medical emergencies to emergency care in study
A study published in Nature Medicine found that OpenAI's ChatGPT Health tool under-triaged 52% of true medical emergencies, directing users to non-urgent care instead of emergency departments. The AI also misclassified 35% of non-urgent cases. Researchers at Mount Sinai conducted 960 tests across 60 clinical scenarios, noting the tool's susceptibility to anchoring bias when symptoms were minimized. The study highlights potential safety concerns as millions use AI for health guidance.
- Company involved
- OpenAI
- AI system involved
- ChatGPT Health
4 source articles · read the reporting →
Facebook's 'People You May Know' feature reveals psychiatrist's patients to each other
A psychiatrist discovered that Facebook's 'People You May Know' feature was recommending her patients as friends to each other and to her, despite no apparent connection. One patient saw friend suggestions for older and infirm people he did not know, who were later recognized as the psychiatrist's patients. Another patient saw a fellow patient from the office elevator, revealing their full name and profile. Facebook could not explain the specific cause but suggested phone contact syncing may have been responsible.
- Company involved
- Facebook
- AI system involved
- People You May Know
2 source articles · read the reporting →
Greater Noida Student Dies After School Alleges AI Cheating
A 16-year-old student in Greater Noida died by suicide on 23 December 2025 after school authorities questioned her over suspected use of AI tools during a pre-board exam. Her father alleges that teachers and the principal publicly humiliated her, causing severe emotional distress, and has filed a police complaint for abetment of suicide. The school denies harassment, stating they followed CBSE protocols, and has provided CCTV footage to police. The incident is under investigation.
1 source article · read the reporting →
Security Health Plan used AI to cut off nursing home care for 85-year-old woman
Frances Walter, an 85-year-old woman with a shattered shoulder, had her nursing home care payment cut off by Security Health Plan after an algorithm predicted she would recover in 16.6 days. The algorithm, nH Predict from NaviHealth, did not account for her severe pain and allergy to pain medicine. She was forced to spend her life savings and enroll in Medicaid while fighting the denial. A federal judge later ruled the denial was speculative and she was owed thousands of dollars.
- Company involved
- Security Health Plan
- AI system involved
- nH Predict
1 source article · read the reporting →
Replika Users Report Crisis After AI Companion Suddenly Rejects Sexual Advances
Users of the AI companion chatbot Replika reported that the app stopped responding to their sexual advances in early February 2023, causing emotional distress and crisis among some users. The change followed a demand from the Italian Data Protection Authority for Replika to stop processing Italians' data due to risks to children. Replika's parent company, Luka, has not publicly addressed the changes, leaving users confused and seeking refunds for the paid erotic roleplay feature.
- Company involved
- Luka
- AI system involved
- Replika
1 source article · read the reporting →
Delhi High Court orders takedown of AI-doctored content of Sadhguru
The Delhi High Court has directed online platforms to remove AI-generated fake videos, audios, and images of spiritual leader Sadhguru that were being used to promote scams. The interim order came in a personality rights case filed by Sadhguru and Isha Foundation. The court's action aims to prevent individuals from falling prey to these scams.
4 source articles · read the reporting →
Researchers find bias in chest X-ray AI classifiers against women, Hispanic and Medicaid patients
A study by the University of Toronto, Vector Institute, and MIT found that AI classifiers trained on public chest X-ray datasets exhibited racial, gender, and socioeconomic bias. Female patients, Hispanic patients, and those with Medicaid insurance were disproportionately affected, with the classifiers often providing incorrect diagnoses for Medicaid patients. The researchers attributed the bias to imbalanced training data and called for rigorous fairness analyses before clinical deployment.
2 source articles · read the reporting →
TikTok algorithm pushed self-harm content to teen accounts, study finds
A study by the Center for Countering Digital Hate found that TikTok's recommendation algorithm showed self-harm and eating disorder videos to accounts registered as 13-year-olds within minutes of showing interest. The researchers set up standard and vulnerable accounts, and the vulnerable accounts received 12 times more self-harm and suicide-related recommendations. TikTok disputed the findings, stating the study did not reflect real user experience.
- Company involved
- TikTok
- AI system involved
- TikTok recommendation algorithm
3 source articles · read the reporting →
Hospitals Use OpenAI’s Whisper Despite Medical Transcription Hallucinations
OpenAI’s Whisper transcription tool is used by over 30,000 medical workers and can insert fabricated text into patient records. Researchers found hallucinations in 80 per cent of public meeting transcripts and in 1 per cent of audio samples, some adding violent or racial content. Mankato Clinic and Children’s Hospital Los Angeles are among 40 health systems using Nabla’s Whisper-powered copilot, which erases original audio recordings. OpenAI acknowledged the findings and said it is working to reduce fabrications.
- Company involved
- Mankato Clinic; Children’s Hospital Los Angeles
- AI system involved
- Whisper
2 source articles · read the reporting →
Hyderabad doctors warn after ChatGPT advice causes patient harm
In Hyderabad, a 30-year-old kidney transplant patient discontinued antibiotics on ChatGPT's advice, losing her transplanted kidney and returning to dialysis. A 62-year-old diabetic man suffered weight loss and low sodium after following a ChatGPT diet plan. Doctors warn that AI tools lack clinical judgment and should not replace professional medical consultation.
- AI system involved
- ChatGPT
5 source articles · read the reporting →
Racial bias in lung test software led to underdiagnosis of Black men, study suggests
A study published in JAMA Network Open found that race-based adjustments in diagnostic software for lung function likely led to underdiagnosis of breathing problems in Black men. Researchers analyzed data from over 2,700 Black men and 5,700 white men at the University of Pennsylvania Health System and concluded that up to 40% more Black men might have been diagnosed if the algorithm were changed. The American Thoracic Society has recommended replacing race-focused adjustments, but changes may take time.
- Company involved
- University of Pennsylvania Health System
2 source articles · read the reporting →
University of Chicago Medical Center sued over sharing patient data with Google
The University of Chicago Medical Center is accused in a class-action lawsuit of sharing hundreds of thousands of patient medical records with Google without proper de-identification or patient consent. The data, from patients treated between 2009 and 2016, allegedly included dates and provider notes that could allow re-identification. The lawsuit claims Google sought the records to develop its own electronic health record system and predictive models. The hospital and Google deny wrongdoing, stating the research partnership was legal and compliant with HIPAA.
- Company involved
- University of Chicago Medical Center
1 source article · read the reporting →
Ninth Circuit Sanctions Attorneys Over AI-Hallucinated Legal Briefs
The Ninth Circuit sanctioned immigration attorneys from Sethi Law Group after they filed briefs containing AI-generated false cases and quotations. The court imposed monetary sanctions, a six-month suspension from Ninth Circuit practice, and referred the matter to the California State Bar. The court also required future filings to disclose AI use and certify that all citations were verified. The case is Lnu v. Blanche.
- Company involved
- Sethi Law Group
2 source articles · read the reporting →
WHO's health chatbot SARAH gives inconsistent and potentially harmful advice
The World Health Organization's AI chatbot SARAH, designed to provide health advice, was found by POLITICO to give contradictory and sometimes unhelpful responses. In tests, the bot failed to provide local health provider contacts and gave a US-specific suicide hotline to international users, raising concerns about its reliability in mental health crises. Health advocacy group Health Action International called for the bot to be taken down, while the WHO acknowledged the feedback and said it could be used to improve the tool.
- Company involved
- World Health Organization
- AI system involved
- SARAH
1 source article · read the reporting →
MP Maria Miller calls for ban on AI 'nudifying' tool DeepSukebe
MP Maria Miller is calling for a ban on AI-powered 'nudifying' tools that generate nude images of women without consent. The tool DeepSukebe, launched in 2020, has had over five million visits and is used to create non-consensual nude images of women, including celebrities. The MP argues that such tools should be outlawed and that the law needs to change to reflect the severity of the harm.
- AI system involved
- DeepSukebe
10 source articles · read the reporting →
VA's Suicide Prevention Algorithm Favours Men, Ignores Sexual Trauma
The Department of Veterans Affairs' REACH VET algorithm, designed to identify veterans at high risk of suicide, reportedly prioritises white men and excludes factors like military sexual trauma that disproportionately affect women. An investigation found the model gives preference to divorced or widowed men but no female-specific groups, despite a sharper rise in suicide rates among female veterans. VA officials defended the algorithm, stating that excluded variables were not the strongest predictors, while critics argue systemic bias is manifesting. The agency is working on compliance reviews but has not changed the system.
- Company involved
- Department of Veterans Affairs
- AI system involved
- REACH VET
2 source articles · read the reporting →
Perplexity AI Misdiagnosis Leads to Man's Death After He Avoids Cancer Treatment
Benjamin Riley's father, a former neuroscientist, used Perplexity AI to self-diagnose his chronic lymphocytic leukemia and was incorrectly told he had Richter's Transformation, leading him to refuse his oncologist's recommended Ven-Obi treatment for a year. Despite his son's efforts to correct the misinformation by contacting the study authors, the father's condition worsened, and he eventually started treatment but died weeks later. The incident highlights the dangers of relying on AI for medical advice.
- Company involved
- Perplexity AI
- AI system involved
- Perplexity AI
1 source article · read the reporting →
NEDA Fires Helpline Staff, Replaces Them with Chatbot After Unionization
The National Eating Disorders Association (NEDA) fired four full-time helpline staff and announced the shutdown of its 20-year-old helpline, replacing it with a chatbot named Tessa. The staff had recently unionized and won an NLRB election, leading to allegations that the move was union busting. NEDA claims the chatbot, a rule-based system developed by Washington University, is a separate program to better serve the community, but the union and critics argue it cannot replace human empathy and may cause irreparable harm. The helpline is set to end on June 1, 2023, with volunteers asked to become testers for the chatbot.
- Company involved
- National Eating Disorders Association (NEDA)
- AI system involved
- Tessa
10 source articles · read the reporting →
AI image generation triggers psychosis in startup employee
Caitlin Ner, head of user experience at an unnamed AI image generation startup, spent up to nine hours a day generating AI images of herself. Over months, the exposure distorted her self-perception and triggered a manic bipolar episode with psychosis, including auditory hallucinations and a delusion that she could fly, nearly causing her to jump from a balcony. She left the startup, sought therapy, and recovered. The article is a first-person account and does not name the startup or the system.
2 source articles · read the reporting →
