Where automated decisions went wrong

PimEyes facial recognition search engine identifies individuals from public photos

PimEyes, a Polish facial recognition search engine, allows users to upload a photo and find matching images from across the internet. The system scrapes billions of photos and creates biometric profiles without consent. A banker named Dylan (pseudonym) discovered that an old party photo of him was found by the system, potentially outing his private life. Privacy advocates and platforms like Instagram and YouTube are taking action against PimEyes, alleging violations of the GDPR.

Company involved

PimEyes

AI system involved

PimEyes

6 source articles · read the reporting →

Dutch probation service's OXREC algorithm flawed, leading to incorrect recidivism risk assessments

The Dutch Inspectorate of Justice and Security (Inspectie JenV) published a report finding that the probation service's (Reclassering) OXREC algorithm contains serious flaws, including swapped formulas and incorrect numbers, causing about a quarter of risk assessments to be wrong. The algorithm, used since 2018 for about 44,000 cases per year, also uses variables that can lead to discrimination, such as neighborhood score and income. The Inspectorate recommended immediate correction or temporary suspension. The probation service announced it would temporarily stop using OXREC.

Company involved

Reclassering Nederland

AI system involved

OXREC

4 source articles · read the reporting →

US Central Command used Anthropic's Claude in Iran airstrikes after Trump ban.

US Central Command used Anthropic's Claude AI system to support airstrikes on Iran, including intelligence assessment and target identification, just hours after President Trump banned federal agencies from using Anthropic tools. The use highlighted a contradiction in the administration's stance, as the Pentagon relied on technology the White House had labelled a security risk. Anthropic faced a supply-chain risk designation for refusing to grant blanket permission for military use, and rival firms OpenAI and xAI later received approval to replace Claude.

Company involved

US Central Command (Centcom)

AI system involved

Claude

4 source articles · read the reporting →

Pega's Hidden AI Tool Listens To US Bank Calls, Suit Says - Law360

The system secretly recorded customer service calls, affecting bank customers.

Company involved

U.S. Bancorp

1 source article · read the reporting →

GoLaxy Used AI to Manipulate Public Opinion in Hong Kong and Taiwan

The Chinese company GoLaxy used an AI system called Smart Propaganda System (GoPro) to monitor and manipulate public opinion in Hong Kong and Taiwan, according to internal documents. The system collected data on members of Congress and other influential Americans, though no campaign was mounted in the United States. GoLaxy denied the allegations, calling them misinformation. The technology represents a new frontier in information warfare, enabling mass production of propaganda.

Company involved

GoLaxy

AI system involved

Smart Propaganda System (GoPro)

2 source articles · read the reporting →

Singapore Firm Defrauded of $499K in Deepfake CEO Video Call Scam

In March 2025, a finance director at a multinational firm in Singapore authorised a US$499,000 payment during a Zoom call that appeared to include the company's CFO and other executives. The call was a deepfake, with AI-generated likenesses and voices. The fraudsters used the impersonation to request an urgent fund transfer for a purported acquisition. The company later discovered the deception and reported it to Singapore police.

2 source articles · read the reporting →

TransUnion AI tenant screening denied 75-year-old man apartment due to mistaken littering conviction

Chris Robinson, then 75, applied for a senior living apartment in California. The property manager used an AI screening program from TransUnion, which assigned him a low score based on a mistaken conviction for littering that belonged to a different person with the same name in Texas. Robinson lost the apartment and his application fee. A federal class-action lawsuit against TransUnion moved toward a $11.5 million settlement in 2023.

10 source articles · read the reporting →

Italian DPA fines Municipality of Trento over AI surveillance projects

The Italian data protection authority (Garante) fined the Municipality of Trento €50,000 for two research projects, Marvel and Protector, that used AI to analyze video, audio, and social media data for public security purposes. The projects involved automated detection of risk events from surveillance cameras and microphones in public spaces, as well as monitoring social media for hate speech. The Garante found multiple violations of privacy law, including lack of a valid legal basis, insufficient anonymization, failure to conduct a data protection impact assessment, and inadequate transparency. The municipality is required to delete the unlawfully processed data.

Company involved

Comune di Trento

AI system involved

Marvel and Protector

9 source articles · read the reporting →

Palantir secretly tested predictive policing in New Orleans

Beginning in 2012, Palantir Technologies secretly partnered with the New Orleans Police Department to deploy a predictive policing system. The programme analysed gang affiliations, social media and criminal histories to forecast individuals’ likelihood of committing or becoming victims of violence, operating without public knowledge or city council oversight. Researchers and law enforcement officials raised concerns about systemic bias and civil liberties. As of 2018, the city and Palantir had not disclosed the programme’s status.

Company involved

New Orleans Police Department

1 source article · read the reporting →

OpenAI disrupts five covert influence operations using AI models

OpenAI terminated accounts linked to five covert influence operations from Russia, China, Iran, and Israel that used OpenAI's language models to generate deceptive content. The operations attempted to manipulate public opinion on topics including the Ukraine war and Gaza conflict, but did not achieve significant audience engagement. OpenAI shared threat indicators with industry peers.

AI system involved

OpenAI language models

9 source articles · read the reporting →

SEC charges YouPlus and CEO with defrauding investors

The SEC charged machine-learning startup YouPlus and its CEO Shaukat Shamim with defrauding investors. Shamim allegedly made false statements about the company's revenue and customer numbers, including providing falsified bank statements. The scheme unravelled when Shamim confessed to investors that the company had earned less than $500,000 and had only four paying customers since 2013. The SEC is seeking permanent injunctions, civil penalties, and an officer-and-director bar.

Company involved

YouPlus

AI system involved

YouPlus machine-learning tool

1 source article · read the reporting →

AI-powered scam compound in Myanmar uses ChatGPT and Gemini to defraud thousands globally

Safeer Mohammed Koorimannil, trafficked to a scam compound in Tai Chang, Myanmar, was forced to use AI-powered software to impersonate a woman and deceive victims into sending money. The software, Kongtian Intelligent Customer Acquisition and Global Social Traffic Navigation, used OpenAI's ChatGPT and Google's Gemini to generate messages and translate in over 100 languages. Koorimannil targeted 50,000 victims in a month, while the tools enabled scammers to rake in tens of millions of dollars. U.S. authorities have created a strike force to disrupt such operations, and OpenAI has banned accounts linked to the scams.

Company involved

Tai Chang

AI system involved

Kongtian Intelligent Customer Acquisition (KT) and Global Social Traffic Navigation (007TG)

2 source articles · read the reporting →

NHS faces legal action over Palantir data contract extension

The NHS is being taken to court by campaign group Open Democracy over its contract with data firm Palantir. The legal action alleges that the extension of Palantir's involvement in analysing NHS patient data for pandemic response and beyond lacked a proper Data Protection Impact Assessment. The contract, initially an emergency response, was extended for two years at a cost of £23.5m. The case is pending.

Company involved

NHS

AI system involved

Palantir data analysis platform

10 source articles · read the reporting →

West Midlands Police test NDAS system to assess crime risk

West Midlands Police are testing a system called the National Data Analytics Solution (NDAS) that analyses police data to assess the risk of individuals committing a crime or becoming a victim. The Home Office has funded the project with millions of pounds, intending to roll it out across England and Wales. Critics warn that the system could reinforce bias against minorities and undermine the presumption of innocence. The police say the technology is designed to support, not replace, officer decision-making and that it is still in early testing.

Company involved

West Midlands Police

AI system involved

National Data Analytics Solution

1 source article · read the reporting →

LinkedIn Used by Foreign Spies with AI-Generated Photos to Target US Officials

Foreign intelligence operations created fake LinkedIn profiles with AI-generated photos to connect with US politicians, lobbyists, and government officials. The fake accounts, such as 'Katie Jones,' sent connection requests to gain credibility and access. LinkedIn removed the account after being contacted by the Associated Press, but the platform remains vulnerable to such espionage tactics.

Company involved

LinkedIn

1 source article · read the reporting →

Privacy International challenges Clearview AI's facial recognition database in Europe

Privacy International filed complaints against Clearview AI with five European data protection authorities in May 2021, alleging that the company's scraping of facial images from the web and building a biometric database without consent violates data protection laws. The regulators in the UK, France, Italy, Greece, and Austria have since found Clearview's practices unlawful, imposed fines, and ordered deletion of data. Clearview has appealed the UK fine, and the case is ongoing.

Company involved

Clearview AI

AI system involved

Clearview

10 source articles · read the reporting →

Dahua Exposed for Uyghur Tracking Software in Surveillance Systems

Dahua Technology's publicly available SDK was found to contain code for detecting Uyghur ethnicity, enabling Chinese police to covertly track the persecuted minority. The revelation led to widespread criticism, and Dahua removed the SDK after being contacted by IPVM but did not comment. The company had previously been sanctioned by the US for complicity in human rights abuses against Uyghurs, and its software is part of a massive surveillance apparatus in Xinjiang.

Company involved

Dahua Technology

AI system involved

Dahua SDK

1 source article · read the reporting →

US Secret Service bought access to cellphone location data

The US Secret Service signed a contract to access Locate X, a service that aggregates location data from phone apps and allows law enforcement to track devices without a warrant. The contract with Babel Street was worth about $36,000 and ran from 2017 to 2018. A former employee said the Secret Service used Locate X in 2018 to seize illegal credit card skimmers at petrol stations. Lawmakers and civil liberties advocates criticised the practice, and Senator Ron Wyden introduced a bill to ban such purchases.

Company involved

United States Secret Service

AI system involved

Locate X

1 source article · read the reporting →

X platform enables deepfake porn and abuse against women activists

Caitlin Roper, a women's rights activist, reports that she and her colleagues at Collective Shout have been subjected to a sustained campaign of misogynistic threats, abuse, and deepfake pornography on X (formerly Twitter). The article alleges that X's AI feature Grok was used by abusers to dox and harass the women, and that X's moderation system failed to remove illegal content, including image-based abuse. Roper states that after she posted screenshots of the abuse, X punished her by making her account unsearchable.

Company involved

X (formerly Twitter)

AI system involved

Grok

2 source articles · read the reporting →

ICE Agents Stored Photos and License Plates of Protest Observers in Palantir-Built Database, Court Filing Reveals - Latin Times

A court filing alleges ICE agents stored photos and license plates of protest observers in a Palantir-built database, labeled some of them as threats, and ran facial recognition searches on them.

Company involved

U.S. Immigration and Customs Enforcement (ICE)

AI system involved

ICM (Investigative Case Management system)

1 source article · read the reporting →

Mexican government agencies hacked using Anthropic's Claude AI

Anthropic's Claude large language model was weaponized by attackers to compromise numerous Mexican government agencies over a month-long campaign starting December 2025. The attackers used Claude to identify 20 security flaws and craft exploit scripts, stealing 150 GB of data including 195 million taxpayer records, civil registry files, voter lists, and government employee credentials. Both Mexico's tax authority and national electoral institute have dismissed the breach.

Company involved

Anthropic

AI system involved

Claude

5 source articles · read the reporting →

CommNV vs Uprise (Nevada DC): AI-hallucinated content in court filing, Monetary Penalty OR Order to volunteer and teach about AI…

The AI system generated fabricated legal citations that were included in a court filing, misleading the court and opposing counsel.

Company involved

Christopher Day

1 source article · read the reporting →

AI chatbots recommended unlicensed casinos to UK users

An investigation by the Guardian and Investigate Europe found that five major AI chatbots—ChatGPT, Gemini, Grok, Microsoft Copilot, and Meta AI—recommended unlicensed online casinos to UK users. Some chatbots also advised on bypassing consumer protection systems like GamStop and financial checks. The companies acknowledged the findings and said they are reviewing safeguards. Regulators expressed concern about the potential harm to vulnerable users.

Company involved

Multiple technology companies (Microsoft, Google, Meta, OpenAI, X)

AI system involved

Copilot, Gemini, Meta AI, ChatGPT, Grok

5 source articles · read the reporting →

Italian bank (Fideuram / Intesa Sanpaolo) loses about 95 million euro to AI voice-clone scam

The AI-generated voice deceived the chairman into authorizing transfers of about 95 million euros.

1 source article · read the reporting →