The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

140 incidents closest to “Duolingo English Test” · matched on meaning · public reporting

WF-QAABL31 Feb 2025

State Bar of California admits using AI to develop bar exam questions

The State Bar of California admitted that it used artificial intelligence to develop multiple-choice questions for the February 2025 bar exam. The AI-generated questions were created by ACS Ventures, the Bar's psychometrician, and were reviewed by content panels. Test takers had complained about technical problems and irregularities, and the admission has sparked further outrage. The State Bar is asking the California Supreme Court to adjust test scores, and the Committee of Bar Examiners will meet in May to discuss remedies.

Company involved
State Bar of California

7 source articles · read the reporting →

WF-ID19SZ1 Jan 2012

Dutch government to refund over 10,000 students over discriminatory DUO fraud algorithm

The Dutch government has pledged to refund over 10,000 students who were unjustly flagged for student finance fraud by a discriminatory algorithm used by the Education Executive Agency (DUO). The algorithm, implemented in 2012, used criteria that disproportionately targeted students from immigrant backgrounds, particularly those of Turkish and Moroccan descent. Following investigations by the Dutch Data Protection Authority and an independent report by PwC, the algorithm was suspended in July 2023 and replaced with a random-sampling system. The government has allocated 61 million euro for refunds.

Company involved
Education Executive Agency (DUO)
AI system involved
DUO fraud detection system

9 source articles · read the reporting →

WF-7VGPYK1 Jan 2021

OpenAI transcribed YouTube videos to train GPT-4 without permission

OpenAI used its Whisper transcription model to transcribe over a million hours of YouTube videos, according to a New York Times report. The company allegedly used the transcripts to train GPT-4 despite knowing the practice was legally questionable. Google, which owns YouTube, said it prohibits unauthorized scraping of its content. OpenAI has said it believes its use of the data constitutes fair use.

Company involved
OpenAI
AI system involved
Whisper, GPT-4

6 source articles · read the reporting →

WF-QLULHG11 Mar 2024

Google, Microsoft, OpenAI chatbots gave false EU election information

A report by Democracy Reporting International found that chatbots from Google, Microsoft, and OpenAI provided incorrect election dates and voting information to users ahead of the European Parliament election. The experiment, conducted in March 2024, tested the chatbots in multiple languages and found that they often hallucinated facts. The European Commission subsequently ordered the companies to explain their measures under the Digital Services Act. Google and Microsoft said they are restricting election-related queries.

Company involved
Google, Microsoft, OpenAI
AI system involved
ChatGPT, Gemini, Copilot

6 source articles · read the reporting →

WF-0YQPL21 Jan 2019

iBorderCtrl lie detector falsely flagged honest reporter as liar

A journalist testing Europe's iBorderCtrl virtual policeman at the Serbian-Hungarian border gave honest answers but was deemed a liar by the system, scoring 48 out of 100 with four false answers flagged. The Hungarian policeman said the result suggested further checks, though none were carried out. The reporter only learned of the result after filing a data access request under European privacy laws. Experts and transparency activists have criticised the technology as pseudoscientific and potentially discriminatory.

Company involved
iBorderCtrl consortium
AI system involved
Silent Talker / iBorderCtrl virtual policeman

10 source articles · read the reporting →

WF-IG5R9T9 Apr 2024

Texas uses AI to grade student STAAR test answers

The Texas Education Agency will use an automated scoring engine to grade written answers on the 2023 STAAR tests, replacing thousands of human graders. The system uses natural language processing and will initially score all responses, with a quarter rescored by humans. Educators have expressed concerns about the system's fairness and the potential for errors, especially for creative or non-standard answers.

Company involved
Texas Education Agency
AI system involved
automated scoring engine

10 source articles · read the reporting →

AI detectors falsely flag non-native English speakers' essays as AI-generated

A study by Stanford researchers found that seven popular AI text detectors wrongly flagged over half of essays written by non-native English speakers as AI-generated. The detectors assess text perplexity, and non-native speakers' simpler word choices lead to false positives. The researchers warn that this bias could have serious implications for students and job applicants, potentially leading to discrimination.

9 source articles · read the reporting →

WF-X0LJEA1 Jan 2020

Voice actors sue LOVO over alleged theft of voices for AI training

Voice actors Paul Skye Lehrman and Linnea Sage filed a proposed class action against AI startup LOVO, alleging the company misappropriated their voices to train its text-to-speech system Genny. The actors claim they were hired on Fiverr under the pretence of academic research but later discovered their voices were used commercially without consent. Lehrman reports a 50% decline in work and loss of control over how his voice is used. The lawsuit, filed in New York federal court, seeks to represent other affected voiceover artists and obtain a court order blocking the practice.

Company involved
LOVO
AI system involved
Genny

6 source articles · read the reporting →

WF-I724H425 Jun 2024

ChatGPT gave incorrect voting information in battleground states and UK

A CBS News investigation found that OpenAI's ChatGPT chatbot provided incorrect or incomplete answers to questions about how to vote in the US battleground states of North Carolina, Pennsylvania, Wisconsin, and Michigan, as well as in the UK. The chatbot failed to correctly state deadlines, ID requirements and other voting procedures, although some answers were later corrected. OpenAI acknowledged the issue and stated that directing users to authoritative sources is a priority, though the chatbot did not always do so.

Company involved
OpenAI
AI system involved
ChatGPT

2 source articles · read the reporting →

WF-CQREJ01 Apr 2021

Proctorio anti-cheating software failed to catch student cheaters in study

Researchers at the University of Twente in the Netherlands tested Proctorio, an anti-cheating software, by asking 30 computer science students to sit an exam while six of them cheated. Proctorio did not flag any of the cheaters and flagged some honest students for irregular behaviour. An independent human review caught only one of the six cheaters. Proctorio disputed the study's methodology and cited other research.

Company involved
Proctorio
AI system involved
Proctorio

5 source articles · read the reporting →

WF-FEIKJ921 Oct 2023

Deepfake video of Taylor Swift speaking Mandarin goes viral in China

A deepfake video of Taylor Swift speaking Mandarin, created using HeyGen's Video Translate tool, went viral on Chinese social media in October 2023. The video, which appeared to show Swift speaking fluent Chinese, was viewed millions of times. The incident sparked mixed reactions, with some praising the technology and others expressing concern about potential misuse. The tool uses AI to translate, clone voice, and sync lips.

AI system involved
Video Translate

10 source articles · read the reporting →

WF-NL6UK61 Jan 2020

Big Tech companies used YouTube videos to train AI without consent

Proof News found that subtitles from 173,536 YouTube videos were used by companies including Anthropic, Nvidia, Apple, and Salesforce to train AI models. The dataset, called YouTube Subtitles, was created by EleutherAI and published in 2020. Creators were not aware and some have expressed frustration, calling it theft. The companies have acknowledged using the dataset but argue it was publicly available.

Company involved
Anthropic, Nvidia, Apple, Salesforce, Bloomberg, Databricks
AI system involved
Claude, OpenELM

10 source articles · read the reporting →

WF-OWK2RT30 Jun 2025

Paradox security vulnerability exposed candidate data to researchers

On June 30, 2025, security researchers discovered a vulnerability in Paradox's test account that allowed access to chat interaction records. The researchers viewed five candidates' personal information including names, email addresses, phone numbers, and IP addresses. Paradox fixed the issue within hours and stated that no data was leaked publicly. The company has since implemented new security measures.

Company involved
Paradox
AI system involved
Paradox conversational AI platform

10 source articles · read the reporting →

Delta uses AI from Fetcherr for domestic ticket pricing

Delta Air Lines is using generative AI from Fetcherr to determine some domestic flight prices, currently covering 3% of its network with plans to reach 20% by end of 2025. Democratic senators expressed concern that the AI could be used for individualized pricing based on personal data, leading to higher fares. Delta denies using personal data in pricing and states it complies with regulations. No actual harm has been reported.

Company involved
Delta Air Lines
AI system involved
Fetcherr

8 source articles · read the reporting →

WF-ALQMVH1 Mar 2025

OpenAI's ChatGPT and Sora exhibit caste bias in India

An MIT Technology Review investigation found that OpenAI's ChatGPT and Sora models reproduce harmful caste stereotypes. When Dhiraj Singha used ChatGPT to polish a fellowship application, the system changed his surname to a high-caste one, causing him psychological distress. Tests showed the models consistently associated Dalits with negative stereotypes and menial jobs, while associating Brahmins with positive traits. OpenAI did not answer questions about the findings.

Company involved
OpenAI
AI system involved
ChatGPT

6 source articles · read the reporting →

WF-59TDC91 Oct 2021

Ask Delphi AI trained on Reddit posts gave unethical answers including endorsing genocide

Ask Delphi, an AI system designed to answer ethical questions, was trained on Reddit posts and crowdworker judgments. It produced responses that were racist, sexist, homophobic, and endorsed genocide if it made people happy. Researchers updated the system three times and added warnings. Critics argue that teaching AI ethics is fundamentally flawed.

AI system involved
Ask Delphi

8 source articles · read the reporting →

WF-91MFNQ30 Jun 2023

OpenAI's GPT-4 shows performance decline, study finds

A study by researchers at Stanford University and UC Berkeley found that OpenAI's GPT-4 model performed significantly worse on some tasks in June than in March, including a drop in accuracy on identifying prime numbers from 97.6% to 2.4%. The cause of the decline is unknown. OpenAI's vice-president of product, Peter Welinder, denied that the model had been made dumber, saying each new version is smarter than the previous one.

Company involved
OpenAI
AI system involved
GPT-4

6 source articles · read the reporting →

DWP algorithm approved Kickstart gateways with no trading history or based abroad

An FE Week investigation found that the Department for Work and Pensions (DWP) approved dozens of companies as Kickstart gateways through automated due diligence checks using the Cabinet Office Spotlight Tool, although some had little or no trading history or were based abroad. The DWP said gateways were subject to stringent checks and later said human checks were also used. After the findings were shared with the Treasury and the DWP, the department stopped taking gateway applications and scrapped the requirement for small employers to use gateways from 3 February.

Company involved
Department for Work and Pensions
AI system involved
Cabinet Office Spotlight Tool

3 source articles · read the reporting →

ChatGPT imitated user's voice without permission during testing

During testing of ChatGPT's Advanced Voice Mode, the AI model unintentionally imitated a user's voice without permission. The incident occurred when noisy audio input caused the model to replace the authorized voice sample with the user's voice. OpenAI acknowledged the issue in its GPT-4o system card and implemented safeguards to prevent recurrence.

Company involved
OpenAI
AI system involved
ChatGPT (GPT-4o with Advanced Voice Mode)

5 source articles · read the reporting →

ChatGPT language glitch causes Welsh output for English prompts

ChatGPT, a chatbot developed by OpenAI, suffered a glitch in which it began generating responses in Welsh instead of English when users entered English-language prompts. The issue left users confused and unable to obtain proper responses. The cause of the bug was not disclosed, and it is unclear how many users were affected or how long the glitch persisted.

Company involved
OpenAI
AI system involved
ChatGPT

7 source articles · read the reporting →

WF-5TENXX1 Aug 2023

Vanderbilt, Northwestern and University of Texas stop using Turnitin AI detector over false cheating accusations

Several US universities, including Vanderbilt, Northwestern and the University of Texas, have stopped using Turnitin's AI detection tool over concerns that it falsely marks student essays as written by ChatGPT. Vanderbilt estimated that the tool's 1% false-positive rate could have wrongly labelled about 750 of 75,000 papers submitted last year. A Texas professor came under fire for failing half his class after the software identified their essays as AI-generated. Turnitin said its technology is not meant to replace educators' professional discretion.

Company involved
Multiple universities (Vanderbilt University, Northwestern University, University of Texas)
AI system involved
Turnitin's AI detection tool

9 source articles · read the reporting →

OpenAI Estimates Hundreds of Thousands of ChatGPT Users May Experience Mental Health Crises Weekly

OpenAI released estimates that around 0.07% of active ChatGPT users show signs of psychosis or mania weekly, and 0.15% express suicidal ideation. The company updated GPT-5 to better recognise mental distress and guide users to support. This follows reports of users being hospitalised, divorced, or dying after prolonged conversations with the chatbot, with loved ones alleging it fuelled delusions.

Company involved
OpenAI
AI system involved
ChatGPT

4 source articles · read the reporting →

WF-LIDYZZ1 Jan 2025

UK universities detect deepfake applicants in automated interviews

Some UK universities use Enroly's automated online interviews to screen international student applicants. Enroly detected about 30 cases of deepfake attempts out of 20,000 interviews during the January 2025 intake. The deepfakes used AI-generated images and audio to replace applicants' faces and voices. Enroly stated it caught the attempts using real-time detection methods.

Company involved
UK universities
AI system involved
Enroly

5 source articles · read the reporting →

WF-I4OTS330 Aug 2024

Edinburgh Airport AI trial gives passengers different parking prices

Edinburgh Airport admitted it was trialling an AI system that randomly set higher or lower parking prices for online bookers, with customers receiving different quotes for the same service. The Scottish Passenger Agents Association called for the trial to be conducted in controlled 'lab' conditions rather than on the public. The airport said the AI's pricing closely matched staff-set prices and the findings would evaluate whether to adopt the system.

Company involved
Edinburgh Airport
AI system involved
AI trial for parking pricing

3 source articles · read the reporting →

← Newerpage 5 of 6Older →