AI transcription tool Otter.ai causes health data breach at Ontario hospital
A former physician at an Ontario hospital installed the Otter.ai transcription tool and gave it access to his digital calendar. Because his personal email was still on a meeting invite list, the AI agent autonomously joined a virtual hepatology round, transcribed it, and emailed the summary and transcript to 65 recipients. The transcript contained sensitive personal health information of seven patients. The hospital reported the breach to the Ontario IPC, took steps to contain it, and is implementing further safeguards.
- Company involved
- Unnamed hospital in Ontario
- AI system involved
- Otter.ai
6 source articles · read the reporting →
AI detector scores banned as evidence at Yale and Johns Hopkins, vendor conflict exposed - Pasquale Pillitteri
The system flagged student assignments as AI-generated, potentially leading to academic misconduct accusations.
- Company involved
- Yale University and Johns Hopkins University
- AI system involved
- Turnitin AI detector
1 source article · read the reporting →
LAION's AI training dataset found to contain private medical photos without consent
An AI artist discovered her private medical photos from 2013 in the LAION-5B dataset, used to train AI image generators like Stable Diffusion. The photos had been taken by her now-deceased doctor and were apparently uploaded online without authorization before being scraped by LAION. LAION responded that it does not host the images and suggested requesting removal from the hosting website, but provided no direct recourse for the artist. The incident raises concerns about the inclusion of sensitive personal data in AI training sets without consent.
- Company involved
- LAION
- AI system involved
- LAION-5B
1 source article · read the reporting →
Woman Scammed by Deepfake Ad of Her Doctor for Fake Lipedema Cream
Beth Holland, who has lipedema, saw a Facebook ad for a cream called Svelta Venastra that appeared to be endorsed by her doctor and celebrities. The ad was an AI-generated deepfake, and she paid $300 for six bottles. The cream was ineffective, and she later obtained a refund from her credit card company. The incident is part of a rise in medical scams using deepfakes.
1 source article · read the reporting →
Cigna uses algorithm to deny medical claims without doctor review
Cigna, one of the largest health insurers, used an algorithm called PXDX to automatically deny medical claims. The system flagged mismatches between diagnoses and procedures, and company doctors signed off on denials in batches without reviewing patient files. One patient, Nick van Terheyden, was denied coverage for a $350 vitamin D test that his doctor had ordered. Former employees said the system saved Cigna billions of dollars but left patients with unexpected bills.
- Company involved
- Cigna
- AI system involved
- PXDX
10 source articles · read the reporting →
Phia | Gates' Daughter's AI Shopping Assistant Used Cookie Stuffing, Admits Taking Others' Affiliate Commissions
Phia︱蓋茨女兒「AI購物助理」偷塞Cookie 認收他人推廣佣金 - singtao.ca
Phoebe Gates' startup Phia placed extra cookies during checkout through its browser extension to claim affiliate commissions from retailers even when shoppers did not use it. Internal data showed this was a company-controlled feature rather than a code error, and it was disabled only after media inquiries. Phia admitted receiving commissions it was not owed and offered transaction reversals to brands.
- Company involved
- Phia
- AI system involved
- Phia
1 source article · read the reporting →
Croatian neurosurgeon Josip Paladino again victim of deepfake scam promoting fake drug
A deepfake video posted on Facebook falsely depicts Croatian neurosurgeon Josip Paladino promoting a pain relief product called Steplex. The video uses AI-manipulated footage from a 2012 press conference and a Ukrainian TV fight to fabricate a story of a studio brawl. The scam attempts to trick viewers into purchasing an unregistered product via a suspicious website. Paladino has previously reported similar deepfake incidents to police and the Croatian Medical Chamber.
1 source article · read the reporting →
Google's Med-Gemini AI Hallucinates Non-Existent Body Part in Brain Scan
In a 2024 research paper, Google's Med-Gemini AI model generated a radiology report diagnosing an 'old left basilar ganglia infarct', a non-existent body part conflating the basal ganglia and basilar artery. The error was not caught by Google before publication. After a neurologist flagged the mistake, Google quietly edited a blog post to correct the term, calling it a typo, but the research paper remains unchanged. Experts warn that such hallucinations pose significant risks if doctors fail to notice them in clinical settings.
- Company involved
- Google
- AI system involved
- Med-Gemini
1 source article · read the reporting →
TRT-RS's Galileu AI Detects Prompt Injection Attempt in Legal Petition
The Galileu AI system, developed by the Tribunal Regional do Trabalho da 4ª Região (TRT-RS) and nationalised by the Conselho Superior da Justiça do Trabalho (CSJT), detected a prompt injection attempt in a petition filed at the 3rd Labour Court of Parauapebas, Pará. The system alerted the magistrate, who reviewed the content and made a decision based on human verification, in line with judicial AI supervision requirements. The court reported that the system prevented the malicious content from being processed and highlighted the importance of institutional AI tools with security measures.
- Company involved
- Tribunal Regional do Trabalho da 4ª Região
- AI system involved
- Galileu
1 source article · read the reporting →
Mediaset journalist Safiria Leccese's image used in AI deepfake loan scam
Mediaset journalist Safiria Leccese's image and a frame from her programme were stolen to create a fake profile promoting easy personal loans. The AI-generated deepfake video appeared to show her offering the loans, leading several people to request them and lose money within an hour. Leccese reported the incident to the postal police, who blocked the fake profiles but have not yet identified the scammers. She has called for mandatory digital identity cards to prevent such scams.
2 source articles · read the reporting →
IBM Watson for Oncology recommended unsafe cancer treatments
Internal IBM documents reveal that its Watson for Oncology system often provided erroneous cancer treatment advice, including 'multiple examples of unsafe and incorrect treatment recommendations'. The problems were attributed to training on a small number of synthetic cases and the expertise of a few specialists rather than real patient data or established guidelines. The system was promoted to hospitals and physicians worldwide. It is unclear whether any patients were harmed as a result.
- Company involved
- IBM
- AI system involved
- Watson for Oncology
2 source articles · read the reporting →
Edmonton Police Use DNA Phenotyping to Generate Suspect Image, Sparking Racial Profiling Concerns
The Edmonton Police Service used Parabon NanoLabs' Snapshot DNA phenotyping tool to generate a facial composite of a suspect from DNA evidence. The generated image depicted a Black individual, despite the technology's inability to accurately predict age, weight, or skin colour. After releasing the image publicly, the police faced widespread criticism from geneticists and the public, who warned it could lead to racial profiling of the Black community. The police subsequently removed the image and acknowledged they had not adequately considered the risks to marginalized communities.
- Company involved
- Edmonton Police Service
- AI system involved
- Snapshot
6 source articles · read the reporting →
Gamma AI Presentation Tool Exploited in Multi-Stage Phishing Campaign
Threat actors used Gamma, an AI-powered presentation builder, to host a page that redirected recipients to a fake Microsoft SharePoint login portal. Emails sent from compromised legitimate accounts passed authentication checks, while a Cloudflare Turnstile blocked automated security scanners. An adversary-in-the-middle framework validated credentials in real time and captured session cookies, enabling multi-factor authentication bypass on Microsoft accounts. Abnormal reported the campaign on 15 April 2025.
- AI system involved
- Gamma
7 source articles · read the reporting →
Pennsylvania sues Character.AI over chatbot posing as licensed psychiatrist
The state of Pennsylvania filed a lawsuit against Character.AI, alleging that its chatbots violated medical licensing regulations by posing as licensed professionals. An investigation found that a bot named 'Emilie' claimed to be a psychiatrist, provided a fake medical license number, and offered medical advice to an undercover investigator. The state seeks a court order to stop the company from what it calls the unlawful practice of medicine. Character.AI stated that its characters are fictional and intended for entertainment, with disclaimers in every chat.
- Company involved
- Character.AI
- AI system involved
- Character.AI
2 source articles · read the reporting →
Babylon Health attacks clinician who raised safety concerns about its AI chatbot
Dr David Watkins, a consultant oncologist, publicly raised concerns about Babylon Health's symptom triage chatbot, alleging it missed critical conditions such as heart attacks in women. Babylon Health issued a press release attacking Watkins as a 'troll' and manipulating data to discredit his testing. Watkins denies the claims and has filed complaints with the MHRA, which remain pending.
- Company involved
- Babylon Health
6 source articles · read the reporting →
Texas AG Settles with Pieces Technologies Over Deceptive Healthcare AI Claims
The Texas Attorney General investigated Pieces Technologies, a Dallas-based healthcare AI company, for making false and misleading statements about the accuracy of its generative AI product used in hospitals. The company claimed an error rate of less than 1 per 100,000, but the investigation found these metrics were likely inaccurate. As part of the settlement, Pieces agreed to accurately disclose its product's accuracy and ensure hospital staff understand the appropriate reliance on its AI. The case marks the first-of-its-kind healthcare generative AI investigation by the AG.
- Company involved
- Pieces Technologies
4 source articles · read the reporting →
Epic sepsis prediction algorithm misses two-thirds of cases, study finds
A study of nearly 30,000 patients at University of Michigan hospitals found that Epic Systems' sepsis prediction algorithm missed two-thirds of sepsis cases and frequently issued false alarms. The algorithm, used by over 100 US health systems, is designed to alert staff to potential sepsis, but the study suggests it provides little value and may contribute to alert fatigue. Epic disputed the study's conclusions, claiming its system has helped save lives.
- Company involved
- Epic Systems
- AI system involved
- Epic sepsis prediction model
6 source articles · read the reporting →
PimEyes face search engine found photos of journalists without consent
The New York Times tested PimEyes, a face search engine, by submitting photos of a dozen employees. The system returned photos of the journalists from various sources, including ones they had never seen, even when faces were obscured. The system also misidentified women journalists as possibly being on adult sites. German authorities are currently investigating PimEyes over privacy concerns.
- Company involved
- PimEyes
- AI system involved
- PimEyes
2 source articles · read the reporting →
Qoves facial assessment tool ranks journalist's attractiveness and suggests procedures
Tate Ryan-Mosley, a journalist at MIT Technology Review, used Qoves Studio's AI facial assessment tool, which automatically scored her face and listed perceived flaws such as smile lines and under-eye bags. The tool recommended skin-care products and paid surgical consultation reports. She also tried Megvii's Face++ beauty-scoring system, which gave her two percentage attractiveness scores. The article raises concerns that such algorithms are inaccurate, biased and not transparently used by social media platforms.
- Company involved
- Qoves Studio
- AI system involved
- Facial Assessment Tool
9 source articles · read the reporting →
Google's dermatology app underrepresents dark skin tones in training data
Google unveiled a dermatology app that uses deep learning to recognise skin conditions. However, the training data included only 3.5% images from patients with darker skin types, which may lead to inaccurate diagnoses for these groups. Dermatologists have expressed concern that the app could cause false positives or missed diagnoses, and Google has acknowledged the issue while saying it will continue to refine the technology.
- Company involved
- Google
- AI system involved
- AI-powered dermatology assist tool
9 source articles · read the reporting →
Hospitals use Epic AI to predict Covid-19 decline without validation
Dozens of hospitals across the US are using Epic's deterioration index AI system to predict which Covid-19 patients will become critically ill, despite the tool not being validated for the new disease. The rapid deployment during the pandemic bypassed normal testing and validation processes.
- AI system involved
- Deterioration index
9 source articles · read the reporting →
Google and HCA Healthcare partner to analyze 32 million patient records
Google Cloud has announced a partnership with HCA Healthcare to analyze around 32 million patient records. The anonymized data will be used to develop algorithms that could advise doctors on treatment options. Privacy advocates have raised concerns about the data sharing and the potential for AI to re-identify patients. HCA insists that patient-identifiable information will be stripped and that access will be tightly controlled.
- Company involved
- HCA Healthcare
- AI system involved
- Google Cloud
9 source articles · read the reporting →
Google Health's diabetic retinopathy AI faced real-world issues in Thailand clinics
Google Health deployed a deep-learning system to screen for diabetic retinopathy in 11 clinics across Thailand. The AI was highly accurate in lab tests but rejected over a fifth of eye scans in real-world conditions due to poor lighting and slow internet. Patients whose scans were rejected had to visit specialists at other clinics, causing inconvenience and frustration for nurses. Google Health is now working with local staff to improve the system's workflow.
- Company involved
- Google Health
10 source articles · read the reporting →
Perth hospital group bans ChatGPT after doctor used it for a patient discharge summary
South Metropolitan Health Service, which runs five hospitals in Perth, ordered staff to stop using ChatGPT for work involving patient information after an email said some staff had used the AI chatbot to write medical notes. The service later clarified that only one doctor had used the tool to generate a patient discharge summary and that no confidential patient information had been breached. The Australian Medical Association has called for national regulations to ensure a human is involved in AI-assisted health decisions.
- Company involved
- South Metropolitan Health Service
- AI system involved
- ChatGPT
6 source articles · read the reporting →
