The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

144 incidents closest to “Duolingo English Test” · matched on meaning · public reporting

WF-LIDYZZ1 Jan 2025

UK universities detect deepfake applicants in automated interviews

Some UK universities use Enroly's automated online interviews to screen international student applicants. Enroly detected about 30 cases of deepfake attempts out of 20,000 interviews during the January 2025 intake. The deepfakes used AI-generated images and audio to replace applicants' faces and voices. Enroly stated it caught the attempts using real-time detection methods.

Company involved
UK universities
AI system involved
Enroly

5 source articles · read the reporting →

WF-I4OTS330 Aug 2024

Edinburgh Airport AI trial gives passengers different parking prices

Edinburgh Airport admitted it was trialling an AI system that randomly set higher or lower parking prices for online bookers, with customers receiving different quotes for the same service. The Scottish Passenger Agents Association called for the trial to be conducted in controlled 'lab' conditions rather than on the public. The airport said the AI's pricing closely matched staff-set prices and the findings would evaluate whether to adopt the system.

Company involved
Edinburgh Airport
AI system involved
AI trial for parking pricing

3 source articles · read the reporting →

WF-U2Y7Z515 Oct 2025

Mass AI cheating scandal at Yonsei University with hundreds of students using ChatGPT

A large-scale cheating scandal has erupted at Yonsei University, where hundreds of students in a third-year online course are suspected of using AI tools such as ChatGPT to cheat on their midterm exam. The professor discovered signs of misconduct and offered students a chance to confess, with those coming forward receiving a zero but no further penalty. A poll on a student community app indicated that over half of respondents admitted to cheating. The university has not yet established clear guidelines on AI use.

Company involved
Yonsei University
AI system involved
ChatGPT

6 source articles · read the reporting →

WF-R8WWZ829 Nov 2025

Amazon Prime Video rolls out AI-generated anime dubs with poor quality

Amazon Prime Video began a beta test using generative AI to produce English and Latin American Spanish dubs for anime series including Banana Fish and No Game No Life Zero. Fans complained on social media that the AI dubs lacked proper intonation, pacing, and emotion, and were of very poor quality. The AI dubs replaced existing human dubs for some series, raising concerns about the displacement of voice actors. Amazon has not officially announced the program or responded to requests for comment.

Company involved
Amazon

5 source articles · read the reporting →

WF-04R0261 Sep 2015

Princeton Review charges higher SAT prep fees to Asian Americans

A ProPublica study found that Princeton Review's geographically-determined pricing system charges Asian American students almost twice as likely as other ethnicities to pay the highest prices for online SAT tutoring. The system sets prices based on location, with higher prices in areas like New York City where Asian Americans are concentrated. Princeton Review stated that prices reflect local costs and competition, but the disparity raises concerns about racial discrimination in automated pricing.

Company involved
Princeton Review

7 source articles · read the reporting →

WF-67SP4W1 Jan 2016

Duke University MTMC Dataset Used in Authoritarian Surveillance Research

Duke University created and openly distributed the Duke MTMC dataset, containing surveillance footage of approximately 2,000 students and visitors on campus. The dataset was used by numerous organisations, including Chinese military-linked companies like SenseTime and Hikvision, for developing person re-identification and facial recognition technologies. Following an investigation by exposing.ai and the Financial Times, Duke University terminated the dataset in May 2019. The incident highlights the privacy risks of academic datasets being repurposed for mass surveillance without consent.

Company involved
Duke University
AI system involved
Duke MTMC

1 source article · read the reporting →

WF-2ZQ1HW24 Jul 2020

PredictiveHire builds AI to predict job hopping from interviews

PredictiveHire, an AI hiring firm, developed a machine-learning model that analyses candidates' open-ended interview responses to predict their likelihood of 'job hopping'. The company used data from 45,899 applicants to build the 'flight risk' assessment, which it advertises as coming soon. Scholars warn that such tools can suppress wages by screening out workers who might seek better pay or conditions, continuing a historical trend of using personality tests to identify potential labour organisers.

Company involved
PredictiveHire
AI system involved
Phai

1 source article · read the reporting →

iTutorGroup to Pay $365,000 to Settle EEOC Discriminatory Hiring Suit

The system automatically rejected tutor applicants aged 55+ (women) or 60+ (men).

Company involved
iTutorGroup

1 source article · read the reporting →

WF-8Y0OB518 Jul 2024

CDT study finds AI chatbots give inaccurate voting information to disabled voters

The Center for Democracy and Technology tested five AI chatbots on July 18, 2024 with 77 prompts about voting with a disability. They found that 61% of responses had at least one insufficiency, and over a third included incorrect information. Some responses could dissuade, impede or prevent a user from voting. The chatbots also hallucinated laws and organizations.

AI system involved
Mixtral 8x7B v0.1, Gemini 1.5 Pro, ChatGPT-4, Claude 3 Opus, Llama 2 70b

5 source articles · read the reporting →

WF-GAQZA320 Jan 2026

Oxford study finds ChatGPT reproduces global inequalities

A study by the Oxford Internet Institute and the University of Kentucky found that ChatGPT systematically attributes positive characteristics to wealthier countries in the Global North and negative stereotypes to lower-income regions, particularly African countries. The research analysed 20.3 million queries and identified five types of bias, concluding that bias is a structural feature of generative AI. The study was published in the journal Platforms and Society on 20th January 2026.

Company involved
OpenAI
AI system involved
ChatGPT

4 source articles · read the reporting →

WF-TTAH5A1 Jan 2023

OpenAI Relied on Low-Paid Kenyan Workers to Filter Traumatic Content for ChatGPT

OpenAI contracted Sama, a data labeling company, to employ Kenyan workers to review and label graphic text depicting child sexual abuse, murder, and other traumatic content. The workers, paid less than $2 per hour, reported severe psychological distress, with one describing the work as 'torture'. Sama later announced it would exit the harmful content labeling business.

Company involved
Sama

8 source articles · read the reporting →

WF-UDZ5S924 Feb 2020

Tinder accused of age discrimination in Tinder Plus pricing

A CHOICE mystery shop found Tinder charging users over 30 up to five times more for Tinder Plus than younger users, with prices ranging from $6.99 to $34.37 per month. An anti-discrimination law expert says this likely breaches Australia's Age Discrimination Act 2004. Tinder did not respond to CHOICE's requests for comment, but settled a similar US lawsuit by stopping age-based pricing in California and compensating affected users.

Company involved
Tinder
AI system involved
Tinder Plus

4 source articles · read the reporting →

WF-UJW7PA1 May 2024

Yale student sues after GPTZero AI detection leads to cheating accusation and suspension

Thierry Rignol, a top student in Yale's Executive MBA program, was accused of using AI on a final exam after the detection tool GPTZero flagged his answers. Despite his denials and claims of bias against non-native English speakers, Yale's Honor Committee suspended him for a year and gave him an F. Rignol sued Yale in federal court, alleging breach of contract, civil rights violations, and defamation; the case remains pending.

Company involved
Yale University
AI system involved
GPTZero

1 source article · read the reporting →

WF-0TNXYT27 Apr 2021

AI Dungeon's sex filter angers fans by blocking unintended content

Latitude, developer of the procedurally generated game AI Dungeon, implemented a test system to block generation of sexual content involving minors. The system incorrectly blocked many stories with no such content, sparking community outrage over privacy and censorship. Latitude acknowledged the test's imperfections and said it reviews flagged content to enforce policies and comply with law. The test system remains in place.

Company involved
Latitude
AI system involved
AI Dungeon

10 source articles · read the reporting →

Privacy group files complaint against proctoring companies over AI surveillance

The Electronic Privacy Information Center (EPIC) filed a complaint with the District of Columbia attorney general against five exam proctoring companies—Respondus, ProctorU, Proctorio, Examity, and Honorlock—alleging unfair and deceptive business practices. The companies are accused of using opaque AI algorithms to flag cheating and collecting excessive biometric data from students. Students have reported that the facial recognition software fails to detect dark skin tones, preventing them from taking exams. EPIC has also sent demand letters threatening civil suits if the companies do not stop their practices.

Company involved
Proctorio, Respondus, ProctorU, Examity, Honorlock

10 source articles · read the reporting →

WF-XFAIXB1 Jan 2014

Dutch Tax Authority discriminated against dual nationality applicants for child care benefits using automated risk system.

The Dutch Data Protection Authority (AP) found that the Benefits Department of the Tax and Customs Administration (Belastingdienst) unlawfully processed the dual nationality of 1.4 million applicants for child care benefits. The data was used as an indicator in an automated risk-scoring system that flagged applications as high-risk, which was unnecessary and discriminatory. The AP concluded that the processing violated the GDPR and was discriminatory. The AP completed its fact-finding and will consider imposing a sanction.

Company involved
Belastingdienst

10 source articles · read the reporting →

WF-JMYGOZ12 May 2021

Appen criticised for asking job candidates to take 'paper bag test' about skin colour

Appen, an Australian AI company, is accused of racism after asking job candidates to select their skin complexion as part of an online application. Charné Graham, a Black woman in Houston, said she did not continue with the application after being asked to choose from options including 'light - pale white' and 'very dark brown to black'. Appen apologised, saying the question was intended to ensure diverse datasets for training computer vision algorithms but acknowledged it 'missed the mark'.

Company involved
Appen

10 source articles · read the reporting →

WF-DRCWLQ1 Sep 2020

Vrije Universiteit Amsterdam accused of racial discrimination by anti-cheating software

A female student with dark skin at Vrije Universiteit Amsterdam alleges that the Proctorio anti-cheating software used during online exams in the 2020/21 academic year discriminated against her on the basis of race. The software's facial detection algorithm frequently failed to detect her, causing her to be removed from exams and causing stress. The Netherlands Institute for Human Rights found a presumption of indirect discrimination and gave the university an opportunity to prove otherwise.

Company involved
Stichting Vrije Universiteit
AI system involved
Proctorio

10 source articles · read the reporting →

WF-P2510P1 Sep 2024

Alpha School parents allege harm from AI-driven learning software

Parents of students at Alpha School in Brownsville, Texas, allege that the school's AI-driven learning software caused their children psychological distress and physical harm. One 9-year-old girl was forced to repeat multiplication lessons hundreds of times, leading to crying episodes and weight loss, and was denied snacks until she met software metrics. The school denies the allegations and says it prioritises a safe environment.

Company involved
Alpha School
AI system involved
IXL

5 source articles · read the reporting →

United Airlines Chatbot Gave Wrong Travel Credit Info - Hoodline

The chatbot told a customer her $200 travel credit would last five years instead of one year.

Company involved
United Airlines

1 source article · read the reporting →

WF-QK5KCO3 Aug 2026

Delta’s AI Told A Passenger Her Flight Home Was Safe, Then Canceled It And Demanded Double To Rebook - View…

The AI canceled the passenger's entire roundtrip award ticket, including the return flight, after the passenger requested only the outbound be canceled.

Company involved
Delta Air Lines
AI system involved
Delta Concierge

1 source article · read the reporting →

Talkdesk Hit With Class Suit Over Customer Service Software - news.bloomberglaw.com

The system secretly recorded and analyzed customer service conversations of callers to Patagonia and Zumiez without consent.

Company involved
Talkdesk Inc.
AI system involved
CX Cloud

1 source article · read the reporting →

WF-V4D9MI6 Jul 2026

Who Gives A Crap suspends AI agent after email error said prices would double - SmartCompany

The AI agent incorrectly confirmed a doubled subscription price to a customer.

Company involved
Who Gives A Crap

1 source article · read the reporting →

Air Canada refund customer like chatbot said it would - TechHQ

The chatbot informed the customer they were eligible for a refund, which the airline later denied.

Company involved
Air Canada

1 source article · read the reporting →

← Newerpage 6 of 6