The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

116 incidents closest to “Evolution Rapid ID” · matched on meaning · public reporting

Presto Automation uses off-site human agents to double-check AI drive-thru orders

Presto Automation Inc, which markets an AI voice assistant for drive-thru ordering, used off-site human agents in countries including the Philippines to double-check orders in more than 70% of customer interactions, according to SEC filings reported by Bloomberg. The company told Bloomberg that the process helps train its system and should reduce human intervention over time. Presto's drive-thru AI is used in more than 400 restaurants, including Del Taco, Carl's Jr and Checkers, and its stock fell more than 10% after the reports.

Company involved
Presto Automation Inc.

8 source articles · read the reporting →

WF-LN1XAB1 Jan 2021

California EDD's automated fraud detection wrongly suspended 600,000 legitimate unemployment claims

In January 2021, the California Employment Development Department used Thompson Reuters automated batch review software to flag 1.1 million unemployment claims as potentially fraudulent. EDD stopped payments on those claims without prior notice. Later, over 600,000 were confirmed as legitimate after claimants used ID.me to verify their identity. The incident highlights the trade-off between fraud prevention and timely benefit access.

Company involved
California Employment Development Department (EDD)
AI system involved
Thompson Reuters Automated Batch Review

7 source articles · read the reporting →

N-Tech.lab's FindFace used to identify St Petersburg metro passengers without consent

Egor Tsvetkov photographed passengers on the St Petersburg metro without their permission and used N-Tech.lab's facial recognition service FindFace to match their faces to public Vkontakte profiles. He published the results in an art project called 'Your Face is Big Data', saying he wanted to show how 'digital narcissism' can lead to stalking. Privacy advocates said the project was ethically problematic because the subjects had not consented and their identities were exposed. FindFace had been launched by N-Tech.lab in February 2016.

Company involved
N-Tech.lab
AI system involved
FindFace

8 source articles · read the reporting →

WF-ROMQCH5 Feb 2024

OnlyFake site uses neural networks to generate fake IDs, bypasses OKX verification

An underground website called OnlyFake uses neural networks to generate realistic photos of fake IDs for $15. The journalist tested the service and obtained a convincing California driver's license. They then used another fake ID to successfully bypass the identity verification process on OKX, a cryptocurrency exchange. The article alleges that this technology could streamline bank fraud and money laundering, but reports no actual financial loss.

Company involved
OKX
AI system involved
OnlyFake

10 source articles · read the reporting →

Serco Leisure issued enforcement notices for unlawful biometric monitoring of employees

Serco Leisure and associated trusts used facial recognition and fingerprint scanning to monitor employee attendance at 38 leisure facilities. The ICO found they unlawfully processed biometric data of over 2,000 employees. Enforcement notices were issued ordering them to stop.

Company involved
Serco Leisure Operating Limited

8 source articles · read the reporting →

WF-87FWC322 May 2024

Salvini and Lega used non-watermarked AI images in EU election campaign

Matteo Salvini and his party Lega created and posted 19 allegedly non-watermarked AI-generated images on social media as part of their 'More Italy, less Europe' electoral campaign. The images promoted false narratives about the EU, including the 'Muslim Great Replacement' theory and EU support for war in Ukraine. The posts reached millions of users, including a paid Facebook ad that reached over 3 million people. Alliance4Europe reported the incident, highlighting that the posts violated the voluntary 2024 European Parliament Elections Code of Conduct.

Company involved
Lega

5 source articles · read the reporting →

L'Observatoire de l'Europe uses AI to steal Euronews articles

Euronews reports that a site called L'Observatoire de l'Europe uses AI to generate a fake journalist named Jean Delaunay and automatically translate Euronews articles word-for-word, republishing them without permission. The site attributes all articles to the fake journalist. Euronews states that this happens daily and that the site will likely steal this article as well.

Company involved
L'Observatoire de l'Europe

6 source articles · read the reporting →

WF-D2EVW81 Jan 2015

Dutch Ministry of Foreign Affairs used secret algorithm to score visa applicants based on nationality

The Dutch Ministry of Foreign Affairs used a secret algorithm called IOB to score short-stay visa applicants based on nationality, gender, and age. Applicants flagged as high risk were automatically moved to an intensive track with higher rejection rates and delays. The ministry's own data protection officer warned of potential ethnic discrimination, but the system continued to be used. The algorithm has profiled millions of applicants since 2015.

Company involved
Ministry of Foreign Affairs
AI system involved
IOB

10 source articles · read the reporting →

WF-0YQPL21 Jan 2019

iBorderCtrl lie detector falsely flagged honest reporter as liar

A journalist testing Europe's iBorderCtrl virtual policeman at the Serbian-Hungarian border gave honest answers but was deemed a liar by the system, scoring 48 out of 100 with four false answers flagged. The Hungarian policeman said the result suggested further checks, though none were carried out. The reporter only learned of the result after filing a data access request under European privacy laws. Experts and transparency activists have criticised the technology as pseudoscientific and potentially discriminatory.

Company involved
iBorderCtrl consortium
AI system involved
Silent Talker / iBorderCtrl virtual policeman

10 source articles · read the reporting →

AI detectors falsely flag non-native English speakers' essays as AI-generated

A study by Stanford researchers found that seven popular AI text detectors wrongly flagged over half of essays written by non-native English speakers as AI-generated. The detectors assess text perplexity, and non-native speakers' simpler word choices lead to false positives. The researchers warn that this bias could have serious implications for students and job applicants, potentially leading to discrimination.

9 source articles · read the reporting →

WF-YRL6VW4 May 2025

Worldcoin halts ID verification in Indonesia after regulatory freeze

Worldcoin, the digital identity project developed by Tools for Humanity, voluntarily paused its identity verification services in Indonesia on May 5, 2025, following a regulatory freeze by the Ministry of Communication and Digital Application. The ministry acted after preliminary investigations found that operating companies lacked required electronic system provider licenses. Worldcoin uses its Orb device to scan faces and irises to create unique digital identities. The company said it is committed to addressing any regulatory shortcomings and awaits clearer guidance.

Company involved
Worldcoin
AI system involved
World ID

3 source articles · read the reporting →

Evolv weapon detection system falsely flags Chromebooks as weapons

Evolv's AI-based weapon detection system has been reported to falsely identify certain Chromebook laptops as weapons, causing false alarms at security checkpoints. The system is used in venues such as schools. The article discusses the flaw and user experiences. No official response from Evolv has been mentioned.

Company involved
Evolv
AI system involved
Evolv

5 source articles · read the reporting →

Manchester Arena's Evolv weapon scanners fail to detect some knives, report finds

ASM Global's use of Evolv Express AI weapon scanners at Manchester Arena has been questioned after a private report found the system failed to detect large knives in 42% of walkthroughs. The report, produced by NCS4 and obtained by IPVM, also suggested the scanners may miss some bombs and components. Evolv did not dispute the findings but said it communicates capabilities and limitations to customers. ASM Global declined to comment on security matters.

Company involved
ASM Global
AI system involved
Evolv Express

5 source articles · read the reporting →

WF-V79N1I12 Jun 2024

Stable Diffusion 3 Medium release generates anatomically incorrect images

Stability AI released Stable Diffusion 3 Medium, an AI image generator, on June 12, 2024. Users on Reddit reported that the model produces mangled human anatomy, such as deformed hands and bodies. The failures are attributed to aggressive NSFW content filtering in the training data that removed images of human anatomy. The company has not responded to the criticism.

Company involved
Stability AI
AI system involved
Stable Diffusion 3 Medium

4 source articles · read the reporting →

WF-DLNNBB1 Jan 2015

French police secretly used Briefcam facial recognition since 2015

The French National Police has been using Briefcam's Video Synopsis software since 2015, according to internal documents obtained by Disclose. The software, which includes facial recognition capabilities, was deployed without the required data protection impact assessment or notification to the CNIL. The Ministry of the Interior concealed the use of this tool for eight years. The police hierarchy allegedly used the software for facial recognition without judicial requisition, and the DGPN did not respond to requests for comment.

Company involved
French National Police
AI system involved
Briefcam Video Synopsis

10 source articles · read the reporting →

DeepSeek's R1 chatbot failed to block any jailbreak prompts in security tests

Security researchers from Cisco and the University of Pennsylvania tested 50 well-known jailbreak prompts against DeepSeek's R1 reasoning model. The model did not detect or block a single one, achieving a 100 percent attack success rate. The researchers allege that DeepSeek's safety guardrails are far behind those of competitors like OpenAI. DeepSeek did not respond to requests for comment.

Company involved
DeepSeek
AI system involved
DeepSeek R1

3 source articles · read the reporting →

WF-8UTONQ22 Dec 2019

Delhi Police use facial recognition to screen PM Modi rally attendees

Delhi Police used an Automated Facial Recognition System (AFRS) to screen crowds at Prime Minister Narendra Modi's rally on December 22, 2019. The system, originally installed to find missing children, was used to identify possible disruptions. Privacy advocates called the move illegal and unconstitutional, saying it amounts to mass surveillance. Police defended the use, citing credible intelligence about possible disruptions.

Company involved
Delhi Police
AI system involved
Automated Facial Recognition System (AFRS)

6 source articles · read the reporting →

Audit of RisCanvi finds biases and reliability issues in criminal justice system

Eticas conducted an adversarial audit of RisCanvi, an AI risk assessment tool used in Catalonia's criminal justice system. The audit uncovered biases in risk classifications against specific demographics and significant reliability issues. The findings call for fairer practices in criminal justice AI.

Company involved
Catalonia's criminal justice system
AI system involved
RisCanvi

4 source articles · read the reporting →

Outabox hack exposes biometric data of patrons at bars, clubs and casinos

Hackers claiming to be former employees published a website allowing searches of Outabox's facial recognition database, exposing biometric and other sensitive data of patrons used for age verification at bars, clubs and casinos. The Surveillance Technology Oversight Project warns that the breach demonstrates the danger of facial recognition for age verification. S.T.O.P. has launched a campaign to ban facial recognition in public accommodations.

Company involved
Outabox

8 source articles · read the reporting →

WF-Q8FS1926 Oct 2025

Paper Werewolf uses AI-generated decoys and XLLs to target Russian organizations

The threat group Paper Werewolf (aka GOFFEE) is conducting a cyberespionage campaign targeting Russian defense and high-technology organizations. The campaign uses AI-generated decoy documents, such as invitations and official letters, to trick recipients into opening malicious Excel XLL add-ins that deliver a backdoor called EchoGather. The backdoor collects system information and communicates with a command-and-control server. The campaign is ongoing and was first detected in late October 2025.

Company involved
Paper Werewolf
AI system involved
EchoGather

2 source articles · read the reporting →

PimEyes facial recognition search engine identifies individuals from public photos

PimEyes, a Polish facial recognition search engine, allows users to upload a photo and find matching images from across the internet. The system scrapes billions of photos and creates biometric profiles without consent. A banker named Dylan (pseudonym) discovered that an old party photo of him was found by the system, potentially outing his private life. Privacy advocates and platforms like Instagram and YouTube are taking action against PimEyes, alleging violations of the GDPR.

Company involved
PimEyes
AI system involved
PimEyes

6 source articles · read the reporting →

42,900 OpenClaw AI agents exposed, 15,200 vulnerable to RCE

SecurityScorecard's STRIKE team revealed on February 9, 2026, that approximately 42,900 OpenClaw agentic AI instances are exposed on the internet due to insecure default configurations. Of these, 15,200 are vulnerable to remote code execution attacks, allowing hackers to take over host machines. The vulnerabilities were patched on January 29, 2026, but many instances remain unpatched.

AI system involved
OpenClaw

5 source articles · read the reporting →

WF-PQ8NMX1 Jan 2022

Stable Diffusion Accused of Stealing Artists' Styles Without Consent

Artists including Greg Rutkowski and Karla Ortiz allege that Stability AI's image generator Stable Diffusion was trained on their work without permission, enabling users to create images mimicking their distinctive styles. The artists express concern that this threatens their livelihoods and identities, as their names become prompts for generating similar art. A tool is being developed to help protect artists from such unauthorised use, but the situation remains unresolved.

Company involved
Stability AI
AI system involved
Stable Diffusion

10 source articles · read the reporting →

WF-2LAYJ824 Apr 2019

Buenos Aires city government uses live facial recognition to track minors in criminal database

The Buenos Aires city government deployed a live facial recognition system in April 2019 that scans subway passengers and matches them against CONARC, a national database of alleged offenders. Human Rights Watch found that the database includes at least 166 children, some as young as one to three years old, and that the system has led to numerous false arrests. The system was implemented without public consultation, and there is no mechanism to correct mistakes. A civil rights organization filed a lawsuit, and the government is pushing a bill to legalize the technology.

Company involved
Buenos Aires city government

1 source article · read the reporting →

← Newerpage 4 of 5Older →