The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

92 incidents closest to “Gradescope Institutional” · matched on meaning · public reporting

WF-8VKXFD28 Jun 2020

UBC students raise concerns over Proctorio's privacy breach and discrimination

The University of British Columbia (UBC) uses Proctorio, an algorithmic test proctoring software, for remote exams. In June 2020, a UBC student reported an issue with Proctorio, and the company's CEO released the student's support chat logs, sparking privacy concerns. The AMS of UBC published an open letter alleging that Proctorio discriminates against students of colour, those with disabilities, and other groups by flagging 'abnormal' behaviours and denying access. The letter calls on UBC to end its contract with Proctorio and conduct an external audit.

Company involved
University of British Columbia
AI system involved
Proctorio

10 source articles · read the reporting →

WF-W8MY6X1 Sep 2020

Middle schooler beats Edgenuity grading algorithm to get perfect score

A seventh-grade student in the Los Angeles Unified School District received a failing grade on a history assignment graded by Edgenuity's automated scoring algorithm. With help from his mother, a history professor, he reverse-engineered the algorithm by writing a paragraph with relevant keywords and a jumble of words, earning a perfect score. The incident highlights concerns about the accuracy and fairness of automated grading systems in education.

Company involved
Los Angeles Unified School District
AI system involved
Edgenuity

10 source articles · read the reporting →

WF-JVU6VY1 Mar 2020

ProctorU threatens UC Santa Barbara faculty over privacy criticism

ProctorU, an online proctoring service, sent a legal threat letter to the University of California Santa Barbara faculty association after the association expressed concerns about ProctorU's data privacy policies. The letter, from ProctorU's lawyer, invoked defamation, copyright, and trademark claims, and was copied to state and federal prosecutors. The faculty association had asked the university to reconsider its relationship with ProctorU due to concerns about student data sharing.

Company involved
ProctorU
AI system involved
ProctorU

10 source articles · read the reporting →

WF-IL3CFJ1 Jul 2020

ProctorU data breach exposes 444,000 user records

ProctorU, an online exam proctoring service, confirmed a data breach after a threat actor leaked a database of user records on a hacker forum. The database contained email addresses, names, addresses, phone numbers, and hashed passwords for approximately 444,000 users, including students from universities such as UCLA, Princeton, and Harvard, as well as U.S. military members. ProctorU stated that the breach involved records from 2014 and that they are investigating the incident.

Company involved
ProctorU
AI system involved
ProctorU

8 source articles · read the reporting →

WF-XH2W9I1 Sep 2020

Proctortrack data breach exposed student data from online proctoring

Proctortrack, an online proctoring service used by universities, suffered a data breach in September 2020 when its source code was leaked online. An analysis by Consumer Reports found that the code contained hard-coded passwords and exposed the names and email addresses of over 150 students. The company acknowledged the leak but said no harm resulted. Students had been required to use the software, which performed facial recognition and recorded video during exams.

Company involved
Proctortrack
AI system involved
Proctortrack

10 source articles · read the reporting →

WF-PEFP2L8 Sep 2020

Proctorio's exam proctoring system allows human agents to review student feeds despite privacy claims

A security researcher analyzed Proctorio's Chrome extension and found evidence that the system allows human agents to review students' room scans and live ID checks, contrary to Proctorio's claims that only professors can access recordings. The system flags students based on behavioral metrics and can terminate exams. The researcher also raised concerns about discrimination against low-income students and privacy violations.

Company involved
Proctorio
AI system involved
Proctorio

10 source articles · read the reporting →

WF-II25VQ28 Jul 2020

ExamSoft failure during Michigan Bar Exam

On July 28, 2020, ExamSoft's software platform failed during the Michigan Bar Exam, preventing some test takers from completing the exam. ExamSoft posted a statement on X (formerly Twitter) with the hashtag #MichiganBarExam, but critics noted the company deleted and reposted the statement multiple times, allegedly to hide negative comments. The incident affected a group of examinees and caused disruption to their ability to take the exam.

Company involved
ExamSoft
AI system involved
ExamSoft platform

10 source articles · read the reporting →

WF-SD7T9128 Sep 2020

Student flagged by ProctorU for reading aloud during exam

A college student, Dana Jo, was flagged by ProctorU test proctoring software for talking during an exam, which she says was reading a question aloud. Her professor initially gave her a zero and placed an academic infraction on her record, jeopardizing her scholarships. After reviewing a video recording, the professor apologized, reinstated her grade, and removed the infraction. ProctorU's CEO stated that the incident highlights the importance of video recordings for review.

Company involved
University (not named)
AI system involved
ProctorU

5 source articles · read the reporting →

Wisconsin’s dropout prediction algorithm labels students high risk with racial bias

Wisconsin’s Dropout Early Warning System (DEWS) uses machine learning to predict middle school students' likelihood of graduating on time, relying on factors including race and income. The Markup found the algorithm frequently mislabels students, with a higher false alarm rate for Black and Hispanic students. The Wisconsin Department of Public Instruction has not informed schools of the racial disparities and continues to use the system.

Company involved
Wisconsin Department of Public Instruction
AI system involved
Dropout Early Warning System (DEWS)

10 source articles · read the reporting →

WF-53YZFN1 Jul 2023

Nevada's AI grad score model cuts funding for low-income students

Nevada replaced its income-based school funding formula with a machine learning model developed by Infinite Campus that generates a 'grad score' for each student. The model reduced the number of students eligible for supplemental funding from 288,000 to 63,000, disproportionately affecting low-income students. Critics argue the model lacks transparency and de-emphasizes economic status, potentially leaving many high-poverty schools underfunded.

Company involved
Nevada Department of Education
AI system involved
Grad score model

7 source articles · read the reporting →

Facial recognition trialled in Australian schools to mark attendance

Five Australian schools are trialling a facial recognition system from LoopLearn to automate student attendance. The system scans students' faces and sends attendance data to an app. Privacy experts have raised concerns about data security and surveillance. The Victorian government previously tried to stop the trial due to privacy risks.

Company involved
Clarendon College
AI system involved
LoopLearn

10 source articles · read the reporting →

WF-PMLNUT17 Sep 2020

California bar exam facial recognition system fails to verify Arab-American student's identity

Ahmed Alamri, an Arab-American law student, was unable to register for the practice California bar exam because ExamSoft's facial recognition system repeatedly failed to recognize his face, citing poor lighting. Alamri attempted to verify his identity over 75 times in different rooms and lighting conditions without success. He and other students filed an emergency petition with the California Supreme Court, alleging that the system discriminates against people of color. The incident highlights concerns about bias in facial recognition technology used for exam proctoring.

Company involved
California State Bar
AI system involved
ExamSoft

10 source articles · read the reporting →

Cleveland State University's room scan requirement ruled unconstitutional

A federal judge ruled that Cleveland State University's requirement for a student to undergo a 360-degree room scan before an online exam was an unreasonable search under the Fourth Amendment. The student, enrolled at the public university, was told shortly before the exam that he would need to scan his private space. The court found that the university's justifications did not outweigh the privacy protections of the home. No final judgment or injunction has been issued yet.

Company involved
Cleveland State University

10 source articles · read the reporting →

WF-701YEA11 Mar 2021

UW-Madison disables Honorlock after skin tone recognition failure

The University of Wisconsin-Madison disabled the exam pause feature of its Honorlock anti-cheating software in March 2021 after three students complained that the software failed to recognize their darker skin tones and paused their exams. The software, used since online classes began, automatically pauses exams when it cannot detect facial features. Honorlock denied the issue was related to skin tone, attributing it to students looking away from their webcams. The university responded by disabling the feature.

Company involved
University of Wisconsin-Madison
AI system involved
Honorlock

10 source articles · read the reporting →

WF-SQJYDZ22 Oct 2020

CBSE introduces facial recognition system for students to access digital documents

The Central Board of Secondary Education (CBSE) has introduced a facial recognition system for Class 10 and 12 students to access their digital academic documents. A live image of the student is compared with the photograph on their CBSE admit card and, if the match succeeds, the certificate is emailed to them. The facility is available on Digi Locker for 2020 records and is expected to help foreign students and those unable to open a Digi Locker account.

Company involved
Central Board of Secondary Education (CBSE)
AI system involved
Facial Recognition System

10 source articles · read the reporting →

Dartmouth Medical School charges 17 students with cheating based on tracking system.

Dartmouth College's Geisel School of Medicine charged 17 students with cheating on remote exams during the pandemic. The charges were based on secret tracking of student activity on the learning management system. Critics say the system is prone to errors and inappropriate for monitoring students.

Company involved
Dartmouth College Geisel School of Medicine

10 source articles · read the reporting →

Hospitals use Epic AI to predict Covid-19 decline without validation

Dozens of hospitals across the US are using Epic's deterioration index AI system to predict which Covid-19 patients will become critically ill, despite the tool not being validated for the new disease. The rapid deployment during the pandemic bypassed normal testing and validation processes.

AI system involved
Deterioration index

9 source articles · read the reporting →

WF-3NJL6T1 Jan 2016

Buona Scuola algorithm mistakenly assigned thousands of teachers in Italy

In 2016, the Buona Scuola algorithm, used by the Italian Ministry of Education, mistakenly assigned thousands of teachers to wrong professional destinations based on mobility rankings. The system was later discontinued. The incident is reported in the Automating Society Report 2020.

Company involved
Italian Ministry of Education
AI system involved
Buona Scuola algorithm

10 source articles · read the reporting →

WF-ARIFZJ13 Jul 2011

DCPS used allegedly false test scores in teacher value-added evaluations

The District of Columbia Public Schools used student test scores from schools under investigation for cheating in value-added calculations for teacher evaluations. More than 200 teachers were terminated based on these evaluations. Teachers can appeal their ratings to the chancellor, but decisions will not be made before the next school year. The school system removes affected scores only when cheating is confirmed.

Company involved
District of Columbia Public Schools
AI system involved
value-added model

10 source articles · read the reporting →

WF-JAFZJY1 Mar 2025

South Korea's AI textbook program rolled back after complaints

South Korea's Ministry of Education launched AI-powered digital textbooks in March 2025 for math, English, and computer science. Students, teachers, and parents complained about technical problems, inaccuracies, and data privacy risks. After four months, the textbooks were stripped of official status and made optional. The program was criticized for being rushed and insufficiently tested.

Company involved
South Korean Ministry of Education
AI system involved
AI-powered textbooks

7 source articles · read the reporting →

Answer.AI tests Devin and reports 14 failures in 20 tasks

Answer.AI's team tested Devin, an autonomous AI coding assistant, on 20 real-world tasks over a month. Devin succeeded in only 3 tasks, failed 14, and was inconclusive in 3. The team found Devin often produced overly complex or hallucinated solutions and could not recognize fundamental blockers. They ultimately decided to stick with tools that allow more human control.

AI system involved
Devin

5 source articles · read the reporting →

WF-EA6R453 Jan 2023

New York City Department of Education blocks ChatGPT on school devices

The New York City Department of Education blocked access to the AI chatbot ChatGPT on school devices and networks, citing concerns about negative impacts on student learning and the safety and accuracy of content. The ban applies to all students and teachers on education department devices and internet networks. Individual schools can still request access for studying the technology. The move is the nation's largest school system's response to the arrival of ChatGPT.

Company involved
New York City Department of Education
AI system involved
ChatGPT

9 source articles · read the reporting →

WF-TT1WEC30 Sep 2020

UIUC students petition to stop Proctorio exam proctoring over privacy concerns

A petition at the University of Illinois at Urbana-Champaign (UIUC) alleges that Proctorio, an online exam proctoring system, violates student privacy by accessing websites, downloads, screen content, and app settings. The petition claims the terms of service allow monitoring by 'any other means necessary', which students find unsettling. The petition, created on September 30, 2020, gathered 1,087 supporters but does not report any specific incident of harm. It calls on UIUC to discontinue use of Proctorio in favour of alternatives.

Company involved
UIUC
AI system involved
Proctorio

9 source articles · read the reporting →

WF-7YJTZ215 Feb 2024

University of Michigan halts vendor offering student data for AI training

The University of Michigan asked a vendor to stop work after a LinkedIn message offered to license student data for AI training for $25,000. The data came from past research studies and did not contain personal identifiers. The university stated that student data was never for sale and that the vendor had shared inaccurate information. The vendor was asked to halt their work.

Company involved
University of Michigan

5 source articles · read the reporting →

← Newerpage 3 of 4Older →