The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

212 incidents closest to “Butterfly iQ3” · matched on meaning · public reporting

WF-JNXGJ91 Dec 2025

AI Deepfake of RTK Journalist Used in Health Scam in Kosovo

In December 2025, a deepfake video manipulated the image and voice of RTK journalist Gëzim Bimbashi to falsely promote a fake cure for joint diseases. The video used the RTK logo to appear credible. Bimbashi denied any involvement, warning that AI-generated disinformation poses a serious risk to public figures. The incident is part of a broader trend of AI-fabricated health scams targeting well-known personalities in Kosovo and Albania.

3 source articles · read the reporting →

WF-P9E72631 Dec 2021

Greek Data Protection Authority fines Ministry of Migration and Asylum for AI surveillance systems

The Hellenic Data Protection Authority (HDPA) imposed an administrative fine of €175,000 on the Ministry of Migration and Asylum (MMA) for GDPR violations related to the 'Centaur' and 'Hyperion' programmes. These systems use AI behavioral analytics, CCTV, drones, and biometric data to monitor and control access to reception facilities for asylum seekers on Greek islands. The HDPA found that the MMA failed to conduct proper Data Protection Impact Assessments and did not cooperate with the authority. The MMA has been ordered to comply with GDPR within three months.

Company involved
Ministry of Migration and Asylum
AI system involved
Centaur and Hyperion programmes

10 source articles · read the reporting →

Adobe Firefly trained on thousands of Midjourney images, Bloomberg reports

Bloomberg has reported that Adobe's Firefly image generator was trained using thousands of images from competitor Midjourney. Adobe says these made up about 5% of the training data and were part of the Adobe Stock library. The company has marketed Firefly as ethically trained and offered enterprise customers indemnity against copyright claims. Adobe responded that all Adobe Stock images undergo moderation, but the report has raised questions about Firefly's copyright safety.

Company involved
Adobe
AI system involved
Firefly

7 source articles · read the reporting →

Manchester City to Trial Facial Recognition at Etihad Stadium

Manchester City are set to trial facial recognition software supplied by Blink Identity at the Etihad Stadium. The system would scan fans' faces to check whether they have bought tickets and allow entry, with the aim of reducing matchday queues. Fans would need to sign up using a picture of their face. The article notes concerns about the potential security of users, but the club and vendor say they are committed to protecting fans.

Company involved
Manchester City

1 source article · read the reporting →

Ukraine used AI-powered drones to target Russian bomber aircraft in Operation Spider Web

Ukraine reportedly used AI-powered autonomous drones to attack Russian bomber aircraft across multiple airbases in an operation codenamed 'Spider's Web'. The drones, developed by Ukraine's Brave1 defence tech cluster, used the SmartPilot system for autonomous targeting without human intervention. The operation damaged or destroyed over 40 Russian bombers including strategic Tu-95 and Tu-22M3 aircraft, according to Ukrainian SBU sources speaking to the BBC. The strikes targeted airbases spanning thousands of kilometres, with each sortie costing approximately $10,000.

Company involved
Ukrainian government (Security Service of Ukraine)
AI system involved
SmartPilot system

5 source articles · read the reporting →

WF-ADQZLS8 Oct 2023

IDF Used AI System Lavender to Target Tens of Thousands in Gaza

The Israeli military used an AI system called Lavender to identify suspected militants in Gaza, marking 37,000 Palestinians as targets for bombing with minimal human oversight. Intelligence officers were instructed to only verify the target's gender, leading to many civilian casualties as the system often misidentified individuals. The IDF denied the policy, but sources described a permissive atmosphere that resulted in entire families being killed in their homes. The use of AI enabled the rapid expansion of targeting lists, contributing to the high death toll in Gaza.

Company involved
Israel Defense Forces
AI system involved
Lavender

9 source articles · read the reporting →

WF-HF1YRW1 Sep 2023

Xuhui police expand facial recognition surveillance to profile 1.1 million residents

The Xuhui District branch of the Shanghai Municipal Bureau of Public Security is expanding its Intelligent Image Recognition System, adding 2,500 facial recognition cameras and increased computing capacity to build profiles of residents and flag deviations. The project, contracted to US-sanctioned FiberHome, is designed to match each face to files on more than 50 million people. Officials say the system will analyse behaviour patterns and trigger early warnings.

Company involved
Shanghai Municipal Bureau of Public Security, Xuhui District Branch
AI system involved
Intelligent Image Recognition System

3 source articles · read the reporting →

Users jailbreak Luma Labs Dream Machine to generate porn

Users have jailbroken Luma Labs' Dream Machine, an AI video generator, to create explicit videos. The system's safeguards were bypassed to generate pornographic content. The videos are crude but demonstrate the potential for widespread AI-generated porn. Luma Labs' terms of service prohibit such content.

Company involved
Luma Labs
AI system involved
Dream Machine

3 source articles · read the reporting →

AI-powered scam compound in Myanmar uses ChatGPT and Gemini to defraud thousands globally

Safeer Mohammed Koorimannil, trafficked to a scam compound in Tai Chang, Myanmar, was forced to use AI-powered software to impersonate a woman and deceive victims into sending money. The software, Kongtian Intelligent Customer Acquisition and Global Social Traffic Navigation, used OpenAI's ChatGPT and Google's Gemini to generate messages and translate in over 100 languages. Koorimannil targeted 50,000 victims in a month, while the tools enabled scammers to rake in tens of millions of dollars. U.S. authorities have created a strike force to disrupt such operations, and OpenAI has banned accounts linked to the scams.

Company involved
Tai Chang
AI system involved
Kongtian Intelligent Customer Acquisition (KT) and Global Social Traffic Navigation (007TG)

2 source articles · read the reporting →

WF-6HCRH41 Sep 2020

iRobot Roomba J7 captured intimate images later leaked by gig workers

In 2020, development versions of iRobot's Roomba J7 series robot vacuums captured images of a woman on the toilet and a child in a hallway. The images were sent to Scale AI for data annotation, where gig workers in Venezuela posted screenshots to private social media groups. iRobot acknowledged the images came from special development robots with recording stickers and said it is terminating its relationship with the service provider that leaked them. The incident highlights privacy risks in the data annotation supply chain for AI training.

Company involved
iRobot
AI system involved
Roomba J7 series

1 source article · read the reporting →

Researchers find bias in chest X-ray AI classifiers against women, Hispanic and Medicaid patients

A study by the University of Toronto, Vector Institute, and MIT found that AI classifiers trained on public chest X-ray datasets exhibited racial, gender, and socioeconomic bias. Female patients, Hispanic patients, and those with Medicaid insurance were disproportionately affected, with the classifiers often providing incorrect diagnoses for Medicaid patients. The researchers attributed the bias to imbalanced training data and called for rigorous fairness analyses before clinical deployment.

2 source articles · read the reporting →

Racial bias in lung test software led to underdiagnosis of Black men, study suggests

A study published in JAMA Network Open found that race-based adjustments in diagnostic software for lung function likely led to underdiagnosis of breathing problems in Black men. Researchers analyzed data from over 2,700 Black men and 5,700 white men at the University of Pennsylvania Health System and concluded that up to 40% more Black men might have been diagnosed if the algorithm were changed. The American Thoracic Society has recommended replacing race-focused adjustments, but changes may take time.

Company involved
University of Pennsylvania Health System

2 source articles · read the reporting →

WF-KH4UOW1 Aug 2020

Google Cloud Used in CBP AI Virtual Border Wall Contract

The Intercept reported that U.S. Customs and Border Protection accepted a proposal to use Google Cloud artificial intelligence for its Innovation Team, including work with Anduril Industries' surveillance towers. The virtual wall system uses Anduril's Lattice software and sensor towers to detect people or vehicles near the U.S.-Mexico border and relay their locations to agents. Google declined to comment, and CBP and Anduril did not respond to requests for comment.

Company involved
U.S. Customs and Border Protection (CBP)
AI system involved
Google Cloud AI Platform with Anduril Lattice and Sentry Towers

1 source article · read the reporting →

WF-CDY4BR1 Jul 2024

Perplexity AI Misdiagnosis Leads to Man's Death After He Avoids Cancer Treatment

Benjamin Riley's father, a former neuroscientist, used Perplexity AI to self-diagnose his chronic lymphocytic leukemia and was incorrectly told he had Richter's Transformation, leading him to refuse his oncologist's recommended Ven-Obi treatment for a year. Despite his son's efforts to correct the misinformation by contacting the study authors, the father's condition worsened, and he eventually started treatment but died weeks later. The incident highlights the dangers of relying on AI for medical advice.

Company involved
Perplexity AI
AI system involved
Perplexity AI

1 source article · read the reporting →

WF-GY0FZ528 Aug 2025

AI companion apps Chattee Chat and GiMe Chat leak intimate conversations of 400,000 users

Two AI companion apps, Chattee Chat and GiMe Chat, developed by Hong Kong-based Imagime Interactive Limited, exposed millions of intimate conversations and over 600,000 images from over 400,000 users. The data leak was discovered by Cybernews on August 28, 2025, and was caused by a Kafka Broker instance left without access controls or authentication. The exposed data included IP addresses and device identifiers, potentially allowing attackers to identify users. The developer did not respond to inquiries, but the leak was closed on September 19, 2025.

Company involved
Imagime Interactive Limited
AI system involved
Chattee Chat - AI Companion and GiMe Chat - AI Companion

4 source articles · read the reporting →

WF-SMARMB1 Jan 2023

AI image generation triggers psychosis in startup employee

Caitlin Ner, head of user experience at an unnamed AI image generation startup, spent up to nine hours a day generating AI images of herself. Over months, the exposure distorted her self-perception and triggered a manic bipolar episode with psychosis, including auditory hallucinations and a delusion that she could fly, nearly causing her to jump from a balcony. She left the startup, sought therapy, and recovered. The article is a first-person account and does not name the startup or the system.

2 source articles · read the reporting →

WF-DDDGZY13 Oct 2025

Xiaomi SU7 fatal crash in Chengdu raises EV safety concerns

A Xiaomi SU7 electric vehicle crashed in Chengdu, China, and caught fire. Bystanders were unable to open the electronic door handles to rescue an occupant, and a 31-year-old male driver died. The incident follows an earlier fatal SU7 crash that raised questions about the vehicle's smart driving features, and has intensified scrutiny on electronic door handle safety.

Company involved
Xiaomi
AI system involved
SU7

3 source articles · read the reporting →

Imperial College AI stethoscope falsely identified heart failure in two-thirds of flagged patients

Researchers at Imperial College London and Imperial College Healthcare NHS Trust have developed an AI-powered stethoscope that analyses heart rhythms and performs an ECG, sending results to a doctor's smartphone. A study of more than 12,000 patients across the UK found that two-thirds of patients flagged as having heart failure did not actually have the condition. The researchers acknowledged that the false positives could cause unnecessary anxiety, and many GP surgeries stopped using or barely used the device after 12 months. The team plans to deploy the stethoscope more widely.

Company involved
Imperial College London and Imperial College Healthcare NHS Trust

6 source articles · read the reporting →

Hackers Extract Encryption Keys from Flock Surveillance Cameras, Revealing Person Detection Alongside Vehicle Tracking - finance.biggo.com

The system detected and recorded images of vehicles and people, affecting individuals in public spaces.

Company involved
Flock Safety
AI system involved
Flock Safety ALPR cameras

1 source article · read the reporting →

WF-4XJW4F30 Sep 2026Portuguese

Investigation probes autopilot failure after fatal Tesla accident in the United States

Investigação apura falha no piloto automático após acidente fatal com Tesla nos Estados Unidos - dfmobilidade.com.br

A fatal crash involving a Tesla Model 3 occurred at an intersection in Buena Vista Township, New Jersey. Authorities are investigating a possible failure of the Autopilot system and whether it was active at the time, adding to scrutiny of Tesla's driver-assistance technology.

Company involved
Tesla
AI system involved
Autopilot

1 source article · read the reporting →

Sibuxiang Genuine + Genuine Sibuxiang: Tesla Autopilot in Another Accident, Owner Says System Failed to Recognize Construction Barriers, Causing Rear-End…

四不像正版+正版四不像特斯拉自动驾驶再出事故,车主称系统未识别施工路障致追尾 - Pchome电脑之家

A Tesla owner reported that the vehicle's Autopilot system did not detect construction barriers, leading to a rear-end collision. The incident was reported by Pchome.

Company involved
Tesla
AI system involved
Autopilot

1 source article · read the reporting →

WF-3XBQHG1 Sep 2023

UK schools install toilet sensors that listen for keywords to monitor pupils

Several UK schools, including St Joseph's College in Stoke-on-Trent, have installed Halo Smart Sensors in toilets that use machine learning to detect keywords such as 'help me' and 'stop it'. The sensors trigger alerts to staff when keywords are detected, aiming to crack down on vaping and bullying. Privacy advocates argue the sensors violate children's privacy and may be unlawful. The schools have not sought parental consent for the monitoring.

Company involved
St Joseph's College
AI system involved
Halo Smart Sensors

4 source articles · read the reporting →

WF-HE06HX10 May 2024

NHTSA investigation PE24015: Rear-end Collisions Involving Zoox Vehicles

Automated driving system controlled vehicle braking and acceleration, leading to rear-end collisions with other road users

Company involved
Zoox
AI system involved
Zoox vehicle

1 source article · read the reporting →

WF-V0IBRA20 Feb 2024

Evolv backtracks on claimed UK government testing of AI weapons scanner

Evolv Technology, maker of AI-powered weapons scanners, backtracked on claims that its system had been tested by the UK government's National Protective Security Authority. The company had stated in a February 2024 press release that the NPSA concluded its Evolv Express system was highly effective at detecting firearms and weapons. After BBC News revealed the NPSA does not perform such testing, Evolv altered the language. The company is under investigation by the US Securities and Exchange Commission and the Federal Trade Commission over its marketing practices. Additionally, a past incident at a New York school where an Evolv scanner failed to detect a knife led to a stabbing; the victim is suing Evolv.

Company involved
Evolv Technology
AI system involved
Evolv Express

3 source articles · read the reporting →

← Newerpage 8 of 9Older →