The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

68 incidents closest to “Relativity ODA” · matched on meaning · public reporting

WF-HM55RC24 Feb 2012

New York City Department of Education Releases Flawed Teacher Ratings Publicly

In February 2012, the New York City Department of Education publicly released individual Teacher Data Reports rating over 12,000 teachers based on value-added analysis of student test scores. The ratings, originally intended for internal use, were disclosed after a court ruled in favor of media organizations under the Freedom of Information Act. The teachers' union and education experts criticised the data as unreliable due to large margins of error and failure to account for demographic factors, warning that the release would unfairly shame or praise educators. The Department defended the ratings as a useful perspective on teacher effectiveness.

Company involved
New York City Department of Education
AI system involved
Teacher Data Reports

6 source articles · read the reporting →

WF-HT4UHG1 Jan 2011

Houston teachers sue over secret EVAAS evaluation algorithm

Houston Independent School District used the SAS Institute's EVAAS system, a proprietary algorithm, to evaluate teacher performance based on student test scores. The Houston Federation of Teachers and several teachers sued in 2014, alleging the system violated their Fourteenth Amendment procedural due process rights because the algorithm is a trade secret and teachers cannot verify the accuracy of their scores. A federal magistrate judge refused to dismiss the procedural due process claims in May 2017, ruling that teachers must have an opportunity to test the accuracy of the scores that could cost them their jobs.

Company involved
Houston Independent School District
AI system involved
Educational Value-Added Assessment System (EVAAS)

1 source article · read the reporting →

WF-PAVWKW1 Jan 2025

Russia-aligned Operation Overload uses AI deepfakes to impersonate experts

In early 2025, the Russia-aligned influence campaign Operation Overload used AI-generated audio and manipulated images to impersonate over 80 media outlets, universities, and law enforcement agencies. The operation spread disinformation targeting Germany, France, and Ukraine to undermine support for Ukraine and sow political division. One video falsely claiming USAID paid celebrities to visit Ukraine gained over 4 million views. The campaign remains ongoing, damaging the reputations of trusted organisations.

Company involved
Operation Overload

3 source articles · read the reporting →

WF-NI986730 Jun 2025

Woody Nora v. M & A Transport, Inc., et al. (E.D. Louisiana): AI-hallucinated content in court filing, Monetary Sanction; 1…

The AI tool generated fictitious case law citations that were included in a court filing, affecting the plaintiff and the court.

AI system involved
Westlaw Precision

1 source article · read the reporting →

WF-L5NTEZ27 Oct 2014

Teachers find no value in SAS EVAAS system in Southwest School District

A study examined the SAS Education Value-Added Assessment System (EVAAS®) used by the Southwest School District (SSD) to evaluate teacher effectiveness for high-stakes consequences. Teachers reported that the system produced inconsistent results, was biased by student factors, and reduced morale and collaboration. The study found that teachers did not use the data formatively as intended, and unintended consequences included heightened pressure and teaching to the test.

Company involved
Southwest School District (SSD)
AI system involved
SAS Education Value-Added Assessment System (EVAAS®)

10 source articles · read the reporting →

U.S. Department of Homeland Security Uses Secret Algorithm ATLAS to Flag Naturalized Citizens for Denaturalization

The U.S. Department of Homeland Security operates a secret algorithm called ATLAS that screens naturalized citizens for potential fraud or national security concerns, flagging them for denaturalization. The system, hosted on Amazon Web Services, analyzes biometric and other data from various federal databases, and its rules are undisclosed. Human reviewers then decide whether to refer flagged individuals to Immigration and Customs Enforcement for possible deportation. Critics allege the algorithm relies on inaccurate data and lacks transparency, leading to wrongful denaturalization proceedings.

Company involved
U.S. Department of Homeland Security
AI system involved
ATLAS

2 source articles · read the reporting →

WF-O7NVTN25 Dec 2025

Mag 7 Ltd. et al. v. Tederi et al. (District Court, Tel Aviv-Yafo): AI-hallucinated content in court filing, Monetary Sanction;…

The AI generated fabricated case law that was submitted to the court, affecting the legal proceedings.

1 source article · read the reporting →

RealPage's YieldStar algorithm pushes rents higher for tenants

RealPage's YieldStar algorithm uses private competitor data to recommend rent increases for apartment units. Landlords adopt up to 90% of the suggestions, leading to higher rents for tenants. Critics allege the software may facilitate indirect price-fixing in violation of antitrust law. The company denies wrongdoing and says the software helps eliminate collusion.

Company involved
RealPage
AI system involved
YieldStar

10 source articles · read the reporting →

WF-6UZ2Y611 Apr 2023

Photographer sues LAION e.V. over refusal to remove copyrighted images from AI training dataset

Photographer Robert Kneschke requested that LAION e.V. remove his copyrighted images from its LAION-5B dataset used to train AI image generators. LAION refused, claiming the use was covered by copyright exceptions, and demanded €887.03 in legal costs from Kneschke. Kneschke, through his lawyer, then filed a lawsuit at the Landgericht Hamburg against LAION, alleging copyright infringement and seeking removal and information.

Company involved
LAION e.V.
AI system involved
LAION-5B dataset

10 source articles · read the reporting →

UDR Sued Over Alleged Use of RealPage Algorithm to Set Rents in San Diego

A class action lawsuit alleges that UDR, Inc. used RealPage's YieldStar algorithmic pricing software to set rental rates and occupancy levels for its San Diego properties, in violation of a local ordinance. The suit claims the software relied on nonpublic competitor data, contributing to inflated rents and making housing less affordable. The lawsuit, filed in July 2026, seeks to represent all affected tenants. UDR has previously acknowledged using YieldStar as one tool among others in its decision-making.

Company involved
UDR, Inc.
AI system involved
RealPage YieldStar

1 source article · read the reporting →

Researchers demonstrate drone attack on Renault Captur ADAS with fake road signs

Security researchers from Ben Gurion University demonstrated a vulnerability in the Mobileye ADAS system used in a Renault Captur. They used drones to project fake road signs for 100 milliseconds, too quick for human perception but detectable by the system. The attack could trick the car into making unsafe maneuvers, though the system is only Level 0 (advisory). The researchers published their findings in a paper titled 'MobilBye: Attacking ADAS with Camera Spoofing'.

AI system involved
Mobileye ADAS

7 source articles · read the reporting →

WF-QWGD1Q25 Jun 2015

Delphi denies near-miss between its self-driving car and Google's autonomous vehicle

Delphi Automotive initially reported a near-miss between its self-driving Audi Q5 SUV and a Google Lexus RX400h in Palo Alto, California. A Delphi executive claimed Google's car cut off the Delphi vehicle, which took evasive action. However, Delphi later denied the incident, stating the vehicles were a lane width apart and that both cars performed correctly. Google also confirmed the cars behaved appropriately in an ordinary driving scenario.

Company involved
Delphi Automotive
AI system involved
Delphi self-driving car (Audi Q5 SUV)

7 source articles · read the reporting →

German Federal Constitutional Court restricts police automated data evaluation

The German Federal Constitutional Court ruled in a case brought by the Gesellschaft für Freiheitsrechte (GFF) that automated data evaluation by police violates fundamental rights. The court restricted the practice, setting limits on police use of automated data analysis. The decision is a success for privacy and civil liberties.

Company involved
German police

10 source articles · read the reporting →

WF-ROSQWB1 Sep 2019

Tesla Autopilot tricked into speeding by electrical tape on sign

Researchers from McAfee Labs demonstrated that a two-inch piece of black electrical tape stuck on a 35 mph speed limit sign can cause the Traffic Aware Cruise Control in older Tesla Model S and X vehicles to misread the sign as 85 mph and automatically accelerate. The vulnerability affects vehicles using the MobilEye EyeQ3 camera system (Hardware Pack 1). The driver can override by braking. Tesla and MobilEye were informed but did not commit to fixing the issue.

Company involved
Tesla
AI system involved
Tesla Autopilot (Hardware Pack 1 with MobilEye EyeQ3)

8 source articles · read the reporting →

WF-W8MY6X1 Sep 2020

Middle schooler beats Edgenuity grading algorithm to get perfect score

A seventh-grade student in the Los Angeles Unified School District received a failing grade on a history assignment graded by Edgenuity's automated scoring algorithm. With help from his mother, a history professor, he reverse-engineered the algorithm by writing a paragraph with relevant keywords and a jumble of words, earning a perfect score. The incident highlights concerns about the accuracy and fairness of automated grading systems in education.

Company involved
Los Angeles Unified School District
AI system involved
Edgenuity

10 source articles · read the reporting →

WF-E7VSR21 Jan 2021

LA's VI-SPDAT housing scoring system gives lower priority to Black and Latino people

The Los Angeles Homeless Services Authority uses the VI-SPDAT scoring system to prioritise unhoused people for subsidised permanent housing. An investigation by The Markup found that Black and Latino people consistently receive lower vulnerability scores than White people, leading to lower priority for housing. The agency has acknowledged the racial disparities and is working on a new tool, but continues to use the current system.

Company involved
Los Angeles Homeless Services Authority
AI system involved
VI-SPDAT

10 source articles · read the reporting →

Wisconsin’s dropout prediction algorithm labels students high risk with racial bias

Wisconsin’s Dropout Early Warning System (DEWS) uses machine learning to predict middle school students' likelihood of graduating on time, relying on factors including race and income. The Markup found the algorithm frequently mislabels students, with a higher false alarm rate for Black and Hispanic students. The Wisconsin Department of Public Instruction has not informed schools of the racial disparities and continues to use the system.

Company involved
Wisconsin Department of Public Instruction
AI system involved
Dropout Early Warning System (DEWS)

10 source articles · read the reporting →

WF-FST9Z61 Apr 2018

ViaQuatro's facial recognition system in São Paulo metro challenged in court

In April 2018, ViaQuatro installed the Digital Interactive Doors System, developed by AdMobilize, on the São Paulo metro's yellow line. The system used cameras to detect passengers' faces and claimed to infer their emotion, age, and gender in order to target advertisements. The Brazilian Institute of Consumer Protection (IDEC) filed a public civil action alleging that the system violated consumer and data protection laws by processing biometric data without consent and making pseudoscientific and discriminatory inferences. A judge ordered the cameras removed in August 2018, and the case is pending a final ruling.

Company involved
ViaQuatro
AI system involved
Digital Interactive Doors System (DID system)

10 source articles · read the reporting →

WF-SWP73K1 Apr 2023

USPS algorithm RRECS causes pay cuts for two-thirds of rural mail carriers

The United States Postal Service (USPS) implemented a new algorithm, RRECS, to evaluate rural carrier routes and determine pay. Due to flaws in how carriers scanned packages, the algorithm underestimated route times, resulting in pay cuts for 66 per cent of rural carriers, some losing thousands of dollars annually. Carriers report that they were not adequately trained on the system and that the cuts are scheduled to take effect, though they have been postponed multiple times. The USPS stated that the system is the result of a nationally negotiated agreement.

Company involved
United States Postal Service
AI system involved
RRECS

9 source articles · read the reporting →

WF-LKR1LN18 Aug 2026

Joann LeDoux v. Outliers, Inc. (2) (W.D. Washington): AI-hallucinated content in court filing, Expert Brief excluded/struck

The AI-generated hallucinated citations in an expert report led to the exclusion of the expert and dismissal of the plaintiff's case.

1 source article · read the reporting →

WF-QSFASR1 Jan 2019

Denmark's Udbetaling Danmark welfare algorithm illegally surveilled millions

Udbetaling Danmark (UDK), Denmark's centralized welfare payment agency, used automated algorithms to check eligibility and detect fraud. The system collected personal data on millions of beneficiaries and their relatives, including non-beneficiaries, in what a think-tank called 'systematic surveillance'. A complaint to the Danish Data Protection Authority led to a ruling that the data collection violated GDPR, and UDK eventually deleted the illegally obtained data. The system also caused erroneous payments, including underpaying 300,000 pensioners by 94 million euros annually.

Company involved
Udbetaling Danmark

10 source articles · read the reporting →

WF-4RKE3V1 May 2016

Danish students publish OkCupid user data, DPA investigates

Two Danish students scraped and published data on 70,000 OkCupid users, including sensitive sexual preferences and religious views, without anonymisation. The dataset was removed after OkCupid filed a copyright notice. The Danish Data Protection Authority (Datatilsynet) has launched an investigation into the incident.

10 source articles · read the reporting →

TOV Realty, LLC v. Suarez; Kosel Equity, LLC v. MacGregor (SC Connecticut): AI-hallucinated content in court filing, 6 hours CLE;…

The AI generated legal briefs containing fabricated citations, which were submitted to the Connecticut Supreme Court.

Company involved
GLG Law LLC
AI system involved
ChatGPT

1 source article · read the reporting →

WF-IJT56I24 Aug 2021

Unity faces employee concerns over military contract transparency

An investigation by Vice Games reported that Unity Technologies is not transparent with its developers about military contracts. Employees allege some are unaware their work on AI tools may become part of Department of Defense projects. CEO John Riccitiello stated the company will not support programs that violate its principles, but sources say many staff are angry. Unity said it cannot police all uses of its engine as it is a tool available to anyone.

Company involved
Unity Technologies
AI system involved
Unity engine

8 source articles · read the reporting →

← Newerpage 2 of 3Older →