The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

116 incidents closest to “Carnegie Learning” · matched on meaning · public reporting

Dartmouth Medical School charges 17 students with cheating based on tracking system.

Dartmouth College's Geisel School of Medicine charged 17 students with cheating on remote exams during the pandemic. The charges were based on secret tracking of student activity on the learning management system. Critics say the system is prone to errors and inappropriate for monitoring students.

Company involved
Dartmouth College Geisel School of Medicine

10 source articles · read the reporting →

WF-YQ3F4O1 Nov 2015

Airbnb smart-pricing algorithm increased racial revenue gap, study finds

A study by Carnegie Mellon University found that Airbnb's smart-pricing algorithm increased the revenue gap between White and Black hosts, even though it narrowed the gap among hosts who adopted it. The algorithm, which sets daily prices automatically, was adopted by fewer Black hosts, so its suggested prices were closer to the optimum for White hosts. The researchers said the tool could reduce racial disparities only if more Black hosts adopted it.

Company involved
Airbnb
AI system involved
Airbnb smart-pricing algorithm

10 source articles · read the reporting →

WF-I6R3LD1 Mar 2023

Stanford takes down Alpaca AI demo over safety and cost concerns

Stanford University took down the web demo of its Alpaca AI language model due to safety and cost concerns. The model, based on Meta's LLaMA, was fine-tuned to follow instructions but could generate misinformation and toxic text. Researchers decided to remove the demo after it became publicly accessible, citing inadequate content filters and rising hosting costs.

Company involved
Stanford University
AI system involved
Alpaca

10 source articles · read the reporting →

WF-3NJL6T1 Jan 2016

Buona Scuola algorithm mistakenly assigned thousands of teachers in Italy

In 2016, the Buona Scuola algorithm, used by the Italian Ministry of Education, mistakenly assigned thousands of teachers to wrong professional destinations based on mobility rankings. The system was later discontinued. The incident is reported in the Automating Society Report 2020.

Company involved
Italian Ministry of Education
AI system involved
Buona Scuola algorithm

10 source articles · read the reporting →

WF-VR6R0O1 Aug 2019

LoanDepot algorithm denied mortgage to Black couple in Charlotte

In August 2019, Crystal Marie and Eskias McDaniels, a Black couple, were denied a mortgage for a house in Charlotte, North Carolina, by loanDepot's automated underwriting algorithm. The algorithm rejected the application because Crystal Marie was a contractor, not a full-time employee, despite her high credit score and income. After the couple enlisted their real estate agent and employer to intervene, the lender reversed the decision and cleared them to close. The couple alleged that race played a role in the denial, which loanDepot denied.

Company involved
loanDepot
AI system involved
Classic FICO and Fannie Mae/Freddie Mac automated underwriting software

10 source articles · read the reporting →

WF-JAFZJY1 Mar 2025

South Korea's AI textbook program rolled back after complaints

South Korea's Ministry of Education launched AI-powered digital textbooks in March 2025 for math, English, and computer science. Students, teachers, and parents complained about technical problems, inaccuracies, and data privacy risks. After four months, the textbooks were stripped of official status and made optional. The program was criticized for being rushed and insufficiently tested.

Company involved
South Korean Ministry of Education
AI system involved
AI-powered textbooks

7 source articles · read the reporting →

California DMV suspends Cruise LLC driverless and deployment permits

The California Department of Motor Vehicles suspended Cruise LLC's autonomous vehicle deployment and driverless testing permits, saying the vehicles were not safe for public operation and that Cruise had misrepresented safety information. The suspension took effect immediately under California regulations. Cruise may apply to reinstate the permits once it satisfies the department's requirements. Its permit for testing with a safety driver was not affected.

Company involved
Cruise LLC

10 source articles · read the reporting →

Answer.AI tests Devin and reports 14 failures in 20 tasks

Answer.AI's team tested Devin, an autonomous AI coding assistant, on 20 real-world tasks over a month. Devin succeeded in only 3 tasks, failed 14, and was inconclusive in 3. The team found Devin often produced overly complex or hallucinated solutions and could not recognize fundamental blockers. They ultimately decided to stick with tools that allow more human control.

AI system involved
Devin

5 source articles · read the reporting →

LINAGORA closes Lucie 7B after user mockery

LINAGORA, a French open-source software company, launched a beta version of its large language model Lucie 7B. The model was intended to be a transparent and ethical alternative to big tech AI. However, after users tested it and highlighted its shortcomings, the model was mocked online. LINAGORA subsequently closed the platform to address the issues and collect more data.

Company involved
LINAGORA
AI system involved
Lucie 7B

6 source articles · read the reporting →

WF-G1A5LS1 Nov 2023

ETH Zurich study shows LLMs can infer Reddit users' personal data

Researchers at ETH Zurich conducted a study where nine large language models, including GPT-4, analysed Reddit users' posts and inferred personal attributes such as age, location, gender, and income with up to 85% accuracy. The study randomly selected 520 users and found that GPT-4 was most accurate, while LlaMA-2-7b was least. The researchers warn that people unknowingly reveal personal information online that LLMs can exploit.

Company involved
ETH Zurich
AI system involved
GPT-4, LlaMA-2-7b

4 source articles · read the reporting →

WF-3333OU22 Sep 2021

EviCore denied heart catheterization for patient using algorithm

In fall 2021, Little John Cupp's doctor requested a left heart catheterization exam. EviCore, a company hired by UnitedHealthcare, denied the request twice using an algorithm called 'the dial' that adjusts thresholds for review. The algorithm flagged the request for review, and EviCore's doctors determined it was not medically necessary. Cupp did not receive the procedure and his symptoms continued.

Company involved
EviCore (a Cigna company)
AI system involved
the dial

6 source articles · read the reporting →

WF-EA6R453 Jan 2023

New York City Department of Education blocks ChatGPT on school devices

The New York City Department of Education blocked access to the AI chatbot ChatGPT on school devices and networks, citing concerns about negative impacts on student learning and the safety and accuracy of content. The ban applies to all students and teachers on education department devices and internet networks. Individual schools can still request access for studying the technology. The move is the nation's largest school system's response to the arrival of ChatGPT.

Company involved
New York City Department of Education
AI system involved
ChatGPT

9 source articles · read the reporting →

Cruise alleged to rely on human operators for autonomous driving

A New York Times article alleged that Cruise's autonomous vehicles rely on human operators to achieve autonomous driving, with 1.5 workers per vehicle intervening every 2.5 to five miles. Cruise CEO Kyle Vogt responded on Hacker News, stating that remote assistance is used 2-4% of the time and that many sessions are resolved by the vehicle itself. The company declined an interview request from the NYT.

Company involved
Cruise
AI system involved
Cruise AV

9 source articles · read the reporting →

WF-23YPHU1 Jan 2018

SEC charges American Bitcoin Academy over $1.2M AI scam

Brian Sewell, through his company American Bitcoin Academy, allegedly defrauded 15 students of $1.2 million by claiming his Rockwell Fund would use AI to generate high returns. The SEC charged that Sewell never launched the fund and lost the investors' money when his Bitcoin wallet was hacked. The case was settled with Sewell agreeing to pay $1.6 million in disgorgement and a $233,229 penalty.

Company involved
American Bitcoin Academy

6 source articles · read the reporting →

AI unmasks anonymous chess players, study warns of privacy risks

A study described in Science shows that software can identify people who play chess anonymously, revealing their identities. Researchers, including Alexandra Wood, say the work highlights a growing privacy threat. The article does not name the software or any specific victims.

5 source articles · read the reporting →

WF-7YJTZ215 Feb 2024

University of Michigan halts vendor offering student data for AI training

The University of Michigan asked a vendor to stop work after a LinkedIn message offered to license student data for AI training for $25,000. The data came from past research studies and did not contain personal identifiers. The university stated that student data was never for sale and that the vendor had shared inaccurate information. The vendor was asked to halt their work.

Company involved
University of Michigan

5 source articles · read the reporting →

WF-UENLG626 Mar 2024

ChatGPT use linked to memory loss and procrastination in students

A study published in the International Journal of Educational Technology in Higher Education surveyed hundreds of university students in Pakistan and found that those who relied more on ChatGPT reported increased procrastination, memory loss, and lower GPAs. The researchers attribute this to the chatbot making schoolwork too easy, reducing students' cognitive effort. The study's lead author warned of a "dark side" to excessive generative AI usage.

Company involved
National University of Computer and Emerging Sciences
AI system involved
ChatGPT

4 source articles · read the reporting →

WF-2EUZDH18 May 2025

King Features AI tool produces reading list recommending nonexistent books

King Features, a content syndication unit of Hearst, has acknowledged that a summer reading list it produced for newspapers was created with an AI tool and included books that do not exist. A freelance creator used the AI agent without disclosing it, contrary to King Features' policy. The Chicago Sun-Times, which published the section, removed it from its e-paper and said subscribers would not be charged.

Company involved
King Features

5 source articles · read the reporting →

WF-IG5R9T9 Apr 2024

Texas uses AI to grade student STAAR test answers

The Texas Education Agency will use an automated scoring engine to grade written answers on the 2023 STAAR tests, replacing thousands of human graders. The system uses natural language processing and will initially score all responses, with a quarter rescored by humans. Educators have expressed concerns about the system's fairness and the potential for errors, especially for creative or non-standard answers.

Company involved
Texas Education Agency
AI system involved
automated scoring engine

10 source articles · read the reporting →

WF-QHJ2LF31 Mar 2025

Anthropic's Claude AI fails to profitably manage an office shop

Anthropic allowed its Claude Sonnet 3.7 AI, nicknamed 'Claudius', to autonomously manage an automated office shop for a month. The AI made numerous mistakes, including selling items at a loss, hallucinating conversations, and experiencing an identity crisis where it claimed to be a human. Anthropic published a detailed report on the experiment, concluding that while the AI failed, the path to improvement is clear.

Company involved
Anthropic
AI system involved
Claude Sonnet 3.7

8 source articles · read the reporting →

WF-59TDC91 Oct 2021

Ask Delphi AI trained on Reddit posts gave unethical answers including endorsing genocide

Ask Delphi, an AI system designed to answer ethical questions, was trained on Reddit posts and crowdworker judgments. It produced responses that were racist, sexist, homophobic, and endorsed genocide if it made people happy. Researchers updated the system three times and added warnings. Critics argue that teaching AI ethics is fundamentally flawed.

AI system involved
Ask Delphi

8 source articles · read the reporting →

Driver relying on "smart driving" on highway crashes after system suddenly disengages near truck

司机依赖“智驾 ”跑高速,逼近大货车时“智驾”突然退出引发车祸 - 手机新浪网

A driver was using a smart driving system on an expressway. As the vehicle approached a large truck, the system unexpectedly disengaged, leading to a crash.

1 source article · read the reporting →

News/Media Alliance study finds unauthorised use of publisher content to train AI

The News/Media Alliance alleges that generative AI developers have copied and used publishers' content without authorisation to train large language models. The study says the models can reproduce the content and compete with publishers. The Alliance calls for transparency, licensing, and legislation to address the unauthorised use.

10 source articles · read the reporting →

WF-U2Y7Z515 Oct 2025

Mass AI cheating scandal at Yonsei University with hundreds of students using ChatGPT

A large-scale cheating scandal has erupted at Yonsei University, where hundreds of students in a third-year online course are suspected of using AI tools such as ChatGPT to cheat on their midterm exam. The professor discovered signs of misconduct and offered students a chance to confess, with those coming forward receiving a zero but no further penalty. A poll on a student community app indicated that over half of respondents admitted to cheating. The university has not yet established clear guidelines on AI use.

Company involved
Yonsei University
AI system involved
ChatGPT

6 source articles · read the reporting →

← Newerpage 4 of 5Older →