The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

68 incidents closest to “Khanmigo for Teachers” · matched on meaning · public reporting

WF-R36DDB15 May 2023

Texas A&M Professor Uses ChatGPT to Falsely Accuse Students of Cheating

Jared Mumm, an instructor at Texas A&M University–Commerce, used ChatGPT to check if students' writing was AI-generated, leading him to accuse them of using the chatbot. He informed students they would receive an incomplete grade and those deemed guilty would get a zero, causing distress and temporarily withholding one student's diploma. The university investigated and stated that no students failed or were barred from graduating, and the professor is working individually with students to resolve the issue.

Company involved
Texas A&M University–Commerce
AI system involved
ChatGPT

5 source articles · read the reporting →

WF-ZY89TF9 Sep 2024

Anonymous Spanish Lawyer (Tribunal Constitucional): AI-hallucinated content in court filing, Formal Reprimand (Apercibimiento) + Referral to Barcelona Bar for Disc

The AI system generated hallucinated content that was submitted in a court filing, potentially misleading the court.

1 source article · read the reporting →

Hangzhou No. 11 Middle School installs facial recognition system to monitor students

Hangzhou No. 11 Middle School has deployed a smart classroom behaviour management system, developed with Hikvision, that uses facial recognition to take attendance and analyse students' expressions and behaviour. The system alerts teachers when students appear inattentive. The school says it records only behavioural state information, not video, and that it will be installed in all classes. Some people have raised privacy concerns.

Company involved
杭州第十一中学
AI system involved
智慧课堂行为管理系统

10 source articles · read the reporting →

WF-EF57BQ1 Jan 2019

Beijing school uses Hanwang emotion recognition system to monitor student behaviour

Niulanshan First Secondary School in Beijing has installed Hanwang Technology's Classroom Care System, which photographs students every second and analyses their facial expressions and behaviour. The system sends weekly reports to teachers and parents, and one student was described as low participation in English class despite being recorded as focused 94% of the time. The school has been highlighted in a report by Article 19, which says such emotion recognition tools rely on junk science and risk eroding privacy and human rights.

Company involved
Niulanshan First Secondary School
AI system involved
Classroom Care System

10 source articles · read the reporting →

WF-N8YCZF1 Aug 2022

Singapore government's AI therapy bot for teachers triggers gaslighting complaints

In August 2022, the Singapore Ministry of Health and Education launched Mindline at Work, an AI chatbot powered by Wysa, to help stressed teachers. Users reported that the bot gave generic advice like breathing exercises instead of listening, leading to accusations of gaslighting. The teachers felt the tool failed to address the root causes of their burnout, such as heavy workloads and large class sizes. The Ministry of Education did not respond to requests for comment, while Wysa defended the app's approach.

Company involved
Singapore Ministry of Health and Ministry of Education
AI system involved
Mindline at Work

10 source articles · read the reporting →

WF-PZC5LA1 Jan 2020

EEOC sues iTutorGroup for automated rejection of older tutor applicants

The US Equal Employment Opportunity Commission (EEOC) alleges that tutoring provider iTutorGroup programmed its recruitment software automatically to reject female applicants aged 55 or older and male applicants aged 60 or older. More than 200 qualified US-based applicants were denied work in 2020. The EEOC has filed a lawsuit in the Eastern District of New York seeking back pay and damages.

Company involved
iTutorGroup

10 source articles · read the reporting →

WF-L1LKDO1 Mar 2021

China's vocational school students exploited as data annotators for AI

Vocational schools in China force students to work as data annotators for AI companies, paying subminimum wages and taking commissions. Students like Lucy in Shandong spent months labeling data for autonomous driving and content moderation systems, with little learning or career advancement. The practice continues despite new regulations.

2 source articles · read the reporting →

WF-W8MY6X1 Sep 2020

Middle schooler beats Edgenuity grading algorithm to get perfect score

A seventh-grade student in the Los Angeles Unified School District received a failing grade on a history assignment graded by Edgenuity's automated scoring algorithm. With help from his mother, a history professor, he reverse-engineered the algorithm by writing a paragraph with relevant keywords and a jumble of words, earning a perfect score. The incident highlights concerns about the accuracy and fairness of automated grading systems in education.

Company involved
Los Angeles Unified School District
AI system involved
Edgenuity

10 source articles · read the reporting →

WF-PEFP2L8 Sep 2020

Proctorio's exam proctoring system allows human agents to review student feeds despite privacy claims

A security researcher analyzed Proctorio's Chrome extension and found evidence that the system allows human agents to review students' room scans and live ID checks, contrary to Proctorio's claims that only professors can access recordings. The system flags students based on behavioral metrics and can terminate exams. The researcher also raised concerns about discrimination against low-income students and privacy violations.

Company involved
Proctorio
AI system involved
Proctorio

10 source articles · read the reporting →

WF-SD7T9128 Sep 2020

Student flagged by ProctorU for reading aloud during exam

A college student, Dana Jo, was flagged by ProctorU test proctoring software for talking during an exam, which she says was reading a question aloud. Her professor initially gave her a zero and placed an academic infraction on her record, jeopardizing her scholarships. After reviewing a video recording, the professor apologized, reinstated her grade, and removed the infraction. ProctorU's CEO stated that the incident highlights the importance of video recordings for review.

Company involved
University (not named)
AI system involved
ProctorU

5 source articles · read the reporting →

WF-SQJYDZ22 Oct 2020

CBSE introduces facial recognition system for students to access digital documents

The Central Board of Secondary Education (CBSE) has introduced a facial recognition system for Class 10 and 12 students to access their digital academic documents. A live image of the student is compared with the photograph on their CBSE admit card and, if the match succeeds, the certificate is emailed to them. The facility is available on Digi Locker for 2020 records and is expected to help foreign students and those unable to open a Digi Locker account.

Company involved
Central Board of Secondary Education (CBSE)
AI system involved
Facial Recognition System

10 source articles · read the reporting →

Dartmouth Medical School charges 17 students with cheating based on tracking system.

Dartmouth College's Geisel School of Medicine charged 17 students with cheating on remote exams during the pandemic. The charges were based on secret tracking of student activity on the learning management system. Critics say the system is prone to errors and inappropriate for monitoring students.

Company involved
Dartmouth College Geisel School of Medicine

10 source articles · read the reporting →

WF-3NJL6T1 Jan 2016

Buona Scuola algorithm mistakenly assigned thousands of teachers in Italy

In 2016, the Buona Scuola algorithm, used by the Italian Ministry of Education, mistakenly assigned thousands of teachers to wrong professional destinations based on mobility rankings. The system was later discontinued. The incident is reported in the Automating Society Report 2020.

Company involved
Italian Ministry of Education
AI system involved
Buona Scuola algorithm

10 source articles · read the reporting →

WF-ARIFZJ13 Jul 2011

DCPS used allegedly false test scores in teacher value-added evaluations

The District of Columbia Public Schools used student test scores from schools under investigation for cheating in value-added calculations for teacher evaluations. More than 200 teachers were terminated based on these evaluations. Teachers can appeal their ratings to the chancellor, but decisions will not be made before the next school year. The school system removes affected scores only when cheating is confirmed.

Company involved
District of Columbia Public Schools
AI system involved
value-added model

10 source articles · read the reporting →

WF-64VALK1 Jan 2021

Amazon Mentor app falsely flags drivers, leading to firings and lost pay

Amazon requires contracted delivery drivers to use the Mentor app, which tracks and scores driving behavior. Drivers report that the app falsely detects infractions, such as incoming calls or picking up a device, causing their scores to drop. This leads to disciplinary actions including firing, loss of bonuses, and reduced pay. Amazon has not addressed the specific allegations.

Company involved
Amazon
AI system involved
Mentor

10 source articles · read the reporting →

Mazaheri v Law Society of Ontario (Law Society Tribunal (ON)): AI-hallucinated content in court filing, Adverse Costs Order

AI generated fabricated legal citations in a court filing by Shahryar Mazaheri.

1 source article · read the reporting →

WF-JAFZJY1 Mar 2025

South Korea's AI textbook program rolled back after complaints

South Korea's Ministry of Education launched AI-powered digital textbooks in March 2025 for math, English, and computer science. Students, teachers, and parents complained about technical problems, inaccuracies, and data privacy risks. After four months, the textbooks were stripped of official status and made optional. The program was criticized for being rushed and insufficiently tested.

Company involved
South Korean Ministry of Education
AI system involved
AI-powered textbooks

7 source articles · read the reporting →

WF-YQ79TL15 Oct 2023

Meta altered content for user including 'Palestinian' and flag

Dr. Khan, a Pakistani dentist, reported that when he included the word 'Palestinian' and the Palestinian flag emoji in a social media story, the platform altered the content. After he complained via TikTok, Meta changed the system. The incident was reported on X and picked up by a BBC reporter.

Company involved
Meta

10 source articles · read the reporting →

WF-EA6R453 Jan 2023

New York City Department of Education blocks ChatGPT on school devices

The New York City Department of Education blocked access to the AI chatbot ChatGPT on school devices and networks, citing concerns about negative impacts on student learning and the safety and accuracy of content. The ban applies to all students and teachers on education department devices and internet networks. Individual schools can still request access for studying the technology. The move is the nation's largest school system's response to the arrival of ChatGPT.

Company involved
New York City Department of Education
AI system involved
ChatGPT

9 source articles · read the reporting →

Meta tracks employee keystrokes on Google, LinkedIn, Wikipedia for AI training

Meta is using an internal tool, Model Capability Initiative (MCI), to capture employees' keystrokes, mouse movements and screen contents on work computers, including on sites such as Google, LinkedIn, Wikipedia and Slack, to train AI agents. Meta confirmed the project and said safeguards protect sensitive content and that the data is not used for other purposes. Employees raised concerns in internal messages that the tool could expose passwords, product details and personal information. A Meta memo said staff can avoid capture by not doing personal work on work computers.

Company involved
Meta
AI system involved
Model Capability Initiative (MCI)

5 source articles · read the reporting →

WF-LK6RRN24 Feb 2024

Google Gemini chatbot gives biased response about PM Narendra Modi

Google's Gemini chatbot generated a biased and inappropriate response to a query about Indian Prime Minister Narendra Modi. A user on X shared screenshots, and Union minister Rajeev Chandrasekhar flagged the response as violating India's IT rules and criminal code. Google acknowledged the issue and said it had taken steps to address it, but warned that Gemini may not be reliable for current events and political topics.

Company involved
Google
AI system involved
Gemini

5 source articles · read the reporting →

WF-UENLG626 Mar 2024

ChatGPT use linked to memory loss and procrastination in students

A study published in the International Journal of Educational Technology in Higher Education surveyed hundreds of university students in Pakistan and found that those who relied more on ChatGPT reported increased procrastination, memory loss, and lower GPAs. The researchers attribute this to the chatbot making schoolwork too easy, reducing students' cognitive effort. The study's lead author warned of a "dark side" to excessive generative AI usage.

Company involved
National University of Computer and Emerging Sciences
AI system involved
ChatGPT

4 source articles · read the reporting →

Fake AI audio of Sadiq Khan nearly caused disorder

A fake audio clip using AI to clone London Mayor Sadiq Khan's voice was shared online in November 2023, allegedly making inflammatory remarks about Remembrance Day. The clip spread rapidly and triggered a spike in hateful comments, and Mr Khan said it almost caused serious disorder. The Metropolitan Police examined the audio and said it did not constitute a criminal offence. The creator of the audio has not been identified.

8 source articles · read the reporting →

WF-IG5R9T9 Apr 2024

Texas uses AI to grade student STAAR test answers

The Texas Education Agency will use an automated scoring engine to grade written answers on the 2023 STAAR tests, replacing thousands of human graders. The system uses natural language processing and will initially score all responses, with a quarter rescored by humans. Educators have expressed concerns about the system's fairness and the potential for errors, especially for creative or non-standard answers.

Company involved
Texas Education Agency
AI system involved
automated scoring engine

10 source articles · read the reporting →

← Newerpage 2 of 3Older →