The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

78 incidents closest to “Khanmigo for Districts” · matched on meaning · public reporting

Kerala AI traffic camera wrongly issues speeding challan at 1240 km/h

Kerala's new AI traffic camera system wrongly recorded a motorcyclist's speed as 1240 km/h and issued a speeding challan. An investigation found the rider had not violated any rules. The system has also been misreading screws on number plates as zeros. Authorities have added human reviewers to control rooms to prevent further errors.

Company involved
Kerala

9 source articles · read the reporting →

WF-1FBZ1K11 Jun 2026

Quinteros v. Harbor Distributing, LLC (CA California (1d)): AI-hallucinated content in court filing, Monetary Sanction; Bar referral

AI generated hallucinated legal citations that were filed in court, misleading the court and opposing counsel.

Company involved
Lipeles Law Group

1 source article · read the reporting →

IRCC uses AI triage for Temporary Resident Visa applications

Immigration, Refugees and Citizenship Canada (IRCC) uses an AI system called Advanced Analytics to triage Temporary Resident Visa applications from India and China. The system categorizes applications into tiers, with Tier 1 approved automatically and others sent to human officers. Critics allege the system lacks transparency and may introduce bias, leading to visa refusals without clear rationale. The author, a Canadian immigration lawyer, is filing Federal Court cases on behalf of clients affected by refusals.

Company involved
Immigration, Refugees and Citizenship Canada (IRCC)
AI system involved
Advanced Analytics Triage of Overseas Temporary Resident Visa Applications

10 source articles · read the reporting →

WF-2ABL3N30 Nov 2023

Bavarian police test Palantir data mining with real personal data

The Bavarian State Criminal Police Office (LKA) has been testing Palantir's data mining software, called VeRa, with real personal data for months. The Bavarian data protection commissioner only learned of the test through a media inquiry and has announced a review. The Interior Ministry claims the test is lawful under current law, but critics argue a legal basis is missing.

Company involved
Bayerisches Landeskriminalamt
AI system involved
VeRa

7 source articles · read the reporting →

WF-T9JVOS26 Oct 2017

Lyft approved driver with terrorism conviction due to background check failure

Lyft allowed a driver with a conviction for aiding terrorism to drive for the ride-sharing service. The driver, Raja L. Khan, had served seven years and was denied by Uber and the city of Chicago, but Lyft's independent background check provider approved him. After WGN Investigates reported the issue, Lyft deactivated the driver and apologised, stating the approval was unacceptable and that it would re-run background checks.

Company involved
Lyft

7 source articles · read the reporting →

WF-LK6RRN24 Feb 2024

Google Gemini chatbot gives biased response about PM Narendra Modi

Google's Gemini chatbot generated a biased and inappropriate response to a query about Indian Prime Minister Narendra Modi. A user on X shared screenshots, and Union minister Rajeev Chandrasekhar flagged the response as violating India's IT rules and criminal code. Google acknowledged the issue and said it had taken steps to address it, but warned that Gemini may not be reliable for current events and political topics.

Company involved
Google
AI system involved
Gemini

5 source articles · read the reporting →

AAIP investigates Worldcoin's personal data processing in Argentina

The Argentine Agency for Access to Public Information (AAIP) has initiated an investigation into the data processing practices of Worldcoin in Argentina. The investigation focuses on the collection, storage, and use of biometric data, including facial and iris scans, carried out in several cities in exchange for financial compensation. The AAIP aims to verify compliance with the country's data protection law, Ley 25.326, regarding sensitive data handling.

Company involved
Worldcoin (Fundación Worldcoin)
AI system involved
Worldcoin

8 source articles · read the reporting →

Fake AI audio of Sadiq Khan nearly caused disorder

A fake audio clip using AI to clone London Mayor Sadiq Khan's voice was shared online in November 2023, allegedly making inflammatory remarks about Remembrance Day. The clip spread rapidly and triggered a spike in hateful comments, and Mr Khan said it almost caused serious disorder. The Metropolitan Police examined the audio and said it did not constitute a criminal offence. The creator of the audio has not been identified.

8 source articles · read the reporting →

WF-FB12K71 Jan 2016

Lahore safe city project stalls as facial recognition cameras fail

The Lahore safe city project in Punjab, Pakistan, has faced major challenges since 2016, with nearly 1,000 facial recognition cameras out of order. The Punjab Safe Cities Authority (PSCA) blames the contractor for failing to meet contractual clauses. Alternative measures, including reliance on privately-installed cameras, have been taken.

Company involved
Punjab Safe Cities Authority
AI system involved
Lahore safe city project

10 source articles · read the reporting →

Seoul Metropolitan Government rolls out AI CCTV cameras to prevent suicides

The Seoul Metropolitan Government has rolled out AI-enabled CCTV cameras on Han River bridges to identify people at risk of suicide. The system uses deep learning to analyse behavioural patterns and alerts rescue teams. Experts and Privacy International have raised concerns that it is invasive and could be misused because it captures biometric data without explicit consent. The article also considers whether such technology could be implemented in India.

Company involved
Seoul Metropolitan Government

8 source articles · read the reporting →

WF-MDLYYQ20 Aug 2024

Bengaluru AI traffic cameras wrongfully fine drivers due to seat belt confusion

Bengaluru's AI traffic cameras issued fines to drivers for not wearing seat belts even when they were properly secured. The system mistook a seat belt matching the driver's shirt colour for non-compliance. Similar issues have been reported in other Indian states. Authorities have added human oversight to review images before fines are finalized.

Company involved
Bengaluru Traffic Police

7 source articles · read the reporting →

WF-8UTONQ22 Dec 2019

Delhi Police use facial recognition to screen PM Modi rally attendees

Delhi Police used an Automated Facial Recognition System (AFRS) to screen crowds at Prime Minister Narendra Modi's rally on December 22, 2019. The system, originally installed to find missing children, was used to identify possible disruptions. Privacy advocates called the move illegal and unconstitutional, saying it amounts to mass surveillance. Police defended the use, citing credible intelligence about possible disruptions.

Company involved
Delhi Police
AI system involved
Automated Facial Recognition System (AFRS)

6 source articles · read the reporting →

Audit of RisCanvi finds biases and reliability issues in criminal justice system

Eticas conducted an adversarial audit of RisCanvi, an AI risk assessment tool used in Catalonia's criminal justice system. The audit uncovered biases in risk classifications against specific demographics and significant reliability issues. The findings call for fairer practices in criminal justice AI.

Company involved
Catalonia's criminal justice system
AI system involved
RisCanvi

4 source articles · read the reporting →

WF-2LAYJ824 Apr 2019

Buenos Aires city government uses live facial recognition to track minors in criminal database

The Buenos Aires city government deployed a live facial recognition system in April 2019 that scans subway passengers and matches them against CONARC, a national database of alleged offenders. Human Rights Watch found that the database includes at least 166 children, some as young as one to three years old, and that the system has led to numerous false arrests. The system was implemented without public consultation, and there is no mechanism to correct mistakes. A civil rights organization filed a lawsuit, and the government is pushing a bill to legalize the technology.

Company involved
Buenos Aires city government

1 source article · read the reporting →

Italian DPA fines Municipality of Trento over AI surveillance projects

The Italian data protection authority (Garante) fined the Municipality of Trento €50,000 for two research projects, Marvel and Protector, that used AI to analyze video, audio, and social media data for public security purposes. The projects involved automated detection of risk events from surveillance cameras and microphones in public spaces, as well as monitoring social media for hate speech. The Garante found multiple violations of privacy law, including lack of a valid legal basis, insufficient anonymization, failure to conduct a data protection impact assessment, and inadequate transparency. The municipality is required to delete the unlawfully processed data.

Company involved
Comune di Trento
AI system involved
Marvel and Protector

9 source articles · read the reporting →

Bahia Facial Recognition System Misidentifies Black Man as Wanted Criminal

Davi, a black administrative assistant, was tracked across 15 metro stations and approached by police after Bahia's facial recognition system falsely matched him to a wanted person. He was released after officers verified his identity. The system, supplied by Iecisa and Huawei, has a low accuracy rate and disproportionately affects black people. The government plans to expand it despite concerns over racial bias and errors.

Company involved
Secretaria de Segurança Pública da Bahia

2 source articles · read the reporting →

WF-P9E72631 Dec 2021

Greek Data Protection Authority fines Ministry of Migration and Asylum for AI surveillance systems

The Hellenic Data Protection Authority (HDPA) imposed an administrative fine of €175,000 on the Ministry of Migration and Asylum (MMA) for GDPR violations related to the 'Centaur' and 'Hyperion' programmes. These systems use AI behavioral analytics, CCTV, drones, and biometric data to monitor and control access to reception facilities for asylum seekers on Greek islands. The HDPA found that the MMA failed to conduct proper Data Protection Impact Assessments and did not cooperate with the authority. The MMA has been ordered to comply with GDPR within three months.

Company involved
Ministry of Migration and Asylum
AI system involved
Centaur and Hyperion programmes

10 source articles · read the reporting →

WF-HF1YRW1 Sep 2023

Xuhui police expand facial recognition surveillance to profile 1.1 million residents

The Xuhui District branch of the Shanghai Municipal Bureau of Public Security is expanding its Intelligent Image Recognition System, adding 2,500 facial recognition cameras and increased computing capacity to build profiles of residents and flag deviations. The project, contracted to US-sanctioned FiberHome, is designed to match each face to files on more than 50 million people. Officials say the system will analyse behaviour patterns and trigger early warnings.

Company involved
Shanghai Municipal Bureau of Public Security, Xuhui District Branch
AI system involved
Intelligent Image Recognition System

3 source articles · read the reporting →

WF-K5KFTA21 Nov 2025

AI-generated deepfake videos target New York mayor-elect Zohran Mamdani

Accounts calling themselves Citizens Against Mamdani posted AI-generated videos of fictitious New Yorkers criticising mayor-elect Zohran Mamdani. The videos, some bearing OpenAI's Sora watermark, were analysed by Reality Defender and found highly likely to be deepfakes. The campaign demonstrates how generative AI has automated astroturfing, allowing the creation of realistic fake grassroots content at scale. Fast Company reached out to the accounts but received no response.

AI system involved
Sora

2 source articles · read the reporting →

WF-Q62ZWT27 Aug 2024

Manipulated AI video falsely shows Ahmed Dogan calling DPS members to protest

A manipulated video falsely shows Movement for Rights and Freedoms (DPS) honorary chairman Ahmed Dogan calling on party members to protest to protect his wealth. The deepfake video uses authentic footage from a 27 August 2024 DPS meeting and overlays AI-generated audio. It was first shared on TikTok by a satirical account and later spread on Facebook by anonymous profiles.

Company involved
Movement for Rights and Freedoms
AI system involved
deepfake

2 source articles · read the reporting →

Delhi Government uses facial recognition on children in government schools

The Delhi Government's Education Department is using facial recognition technology in conjunction with CCTV cameras in government schools, according to responses to RTI requests filed by the Internet Freedom Foundation. The organisation urged the Delhi Commission for Protection of Child Rights to intervene, alleging that the practice violates children's right to privacy and education under the Indian Constitution. No specific incident of harm has been reported, but the organisation warns of potential misidentification and disproportionate penalties.

Company involved
Government of NCT of Delhi, Education Department

10 source articles · read the reporting →

WF-GY0FZ528 Aug 2025

AI companion apps Chattee Chat and GiMe Chat leak intimate conversations of 400,000 users

Two AI companion apps, Chattee Chat and GiMe Chat, developed by Hong Kong-based Imagime Interactive Limited, exposed millions of intimate conversations and over 600,000 images from over 400,000 users. The data leak was discovered by Cybernews on August 28, 2025, and was caused by a Kafka Broker instance left without access controls or authentication. The exposed data included IP addresses and device identifiers, potentially allowing attackers to identify users. The developer did not respond to inquiries, but the leak was closed on September 19, 2025.

Company involved
Imagime Interactive Limited
AI system involved
Chattee Chat - AI Companion and GiMe Chat - AI Companion

4 source articles · read the reporting →

WF-SU17EB1 Mar 2026

Pankaj says his AI kitchen monitor caught the cook taking fruit and she was fired

Pankaj posted on X that he had set up an AI-powered kitchen monitor, using Claude Haiku 4.5 as its vision model, to watch his cook while she worked. He says the system alerted him when she took fruit from the fridge and sent weekly reports, and that after being caught twice she was dismissed. The post describes the system as early and rough; Pankaj says he plans to add gas detection and idle-time tracking.

Company involved
Pankaj
AI system involved
AI roommate

5 source articles · read the reporting →

WF-ALU6UI1 Mar 2025

Meta accused of exploiting content moderators in Ghana

Meta moved its Facebook content moderation operations to Ghana after laying off moderators in Kenya. Workers in Accra report poor working conditions, low pay, and psychological harm from reviewing traumatic content. Foxglove is investigating and planning a legal challenge.

Company involved
Meta
AI system involved
Facebook content moderation system

3 source articles · read the reporting →

← Newerpage 3 of 4Older →