The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

116 incidents closest to “D2L Brightspace” · matched on meaning · public reporting

Answer.AI tests Devin and reports 14 failures in 20 tasks

Answer.AI's team tested Devin, an autonomous AI coding assistant, on 20 real-world tasks over a month. Devin succeeded in only 3 tasks, failed 14, and was inconclusive in 3. The team found Devin often produced overly complex or hallucinated solutions and could not recognize fundamental blockers. They ultimately decided to stick with tools that allow more human control.

AI system involved
Devin

5 source articles · read the reporting →

WF-L8981D29 Jan 2025

DeepSeek exposed user data via open ClickHouse database

Cloud security firm Wiz discovered a ClickHouse database belonging to DeepSeek that was open to the internet without authentication, containing over a million lines of logs with chat histories, secret keys and backend details. Wiz disclosed the breach to DeepSeek, which promptly locked down the database. The incident highlights security risks in rapidly deploying AI services.

Company involved
DeepSeek
AI system involved
DeepSeek-R1

5 source articles · read the reporting →

Researchers jailbreak Stable Diffusion and DALL-E 2 to generate disturbing images

Researchers from Johns Hopkins and Duke universities developed a method called SneakyPrompt that uses reinforcement learning to bypass safety filters in text-to-image AI models. The technique allowed them to generate images of nudity and violence from Stable Diffusion and DALL-E 2. OpenAI has since fixed the vulnerability in DALL-E 2, but Stable Diffusion 1.4 remains vulnerable. Stability AI says it is working with the researchers to improve defenses.

AI system involved
Stable Diffusion 1.4 and DALL-E 2

5 source articles · read the reporting →

WF-E48MLU24 Nov 2023

DevTernity conference cancelled after fake women speakers exposed

The DevTernity software developer conference was cancelled after allegations that organiser Eduards Sizovs added fake women speakers to the lineup. Sizovs admitted at least one profile, 'Anna Boyko', was an auto-generated 'demo persona' that appeared on the site by mistake. The conference was scheduled to begin December 7 but was called off after several speakers withdrew. Sizovs denied wrongdoing and said he had fixed the code and written a test to prevent a recurrence.

Company involved
DevTernity

9 source articles · read the reporting →

WF-EA6R453 Jan 2023

New York City Department of Education blocks ChatGPT on school devices

The New York City Department of Education blocked access to the AI chatbot ChatGPT on school devices and networks, citing concerns about negative impacts on student learning and the safety and accuracy of content. The ban applies to all students and teachers on education department devices and internet networks. Individual schools can still request access for studying the technology. The move is the nation's largest school system's response to the arrival of ChatGPT.

Company involved
New York City Department of Education
AI system involved
ChatGPT

9 source articles · read the reporting →

WF-TT1WEC30 Sep 2020

UIUC students petition to stop Proctorio exam proctoring over privacy concerns

A petition at the University of Illinois at Urbana-Champaign (UIUC) alleges that Proctorio, an online exam proctoring system, violates student privacy by accessing websites, downloads, screen content, and app settings. The petition claims the terms of service allow monitoring by 'any other means necessary', which students find unsettling. The petition, created on September 30, 2020, gathered 1,087 supporters but does not report any specific incident of harm. It calls on UIUC to discontinue use of Proctorio in favour of alternatives.

Company involved
UIUC
AI system involved
Proctorio

9 source articles · read the reporting →

WF-7YJTZ215 Feb 2024

University of Michigan halts vendor offering student data for AI training

The University of Michigan asked a vendor to stop work after a LinkedIn message offered to license student data for AI training for $25,000. The data came from past research studies and did not contain personal identifiers. The university stated that student data was never for sale and that the vendor had shared inaccurate information. The vendor was asked to halt their work.

Company involved
University of Michigan

5 source articles · read the reporting →

WF-ID19SZ1 Jan 2012

Dutch government to refund over 10,000 students over discriminatory DUO fraud algorithm

The Dutch government has pledged to refund over 10,000 students who were unjustly flagged for student finance fraud by a discriminatory algorithm used by the Education Executive Agency (DUO). The algorithm, implemented in 2012, used criteria that disproportionately targeted students from immigrant backgrounds, particularly those of Turkish and Moroccan descent. Following investigations by the Dutch Data Protection Authority and an independent report by PwC, the algorithm was suspended in July 2023 and replaced with a random-sampling system. The government has allocated 61 million euro for refunds.

Company involved
Education Executive Agency (DUO)
AI system involved
DUO fraud detection system

9 source articles · read the reporting →

WF-UENLG626 Mar 2024

ChatGPT use linked to memory loss and procrastination in students

A study published in the International Journal of Educational Technology in Higher Education surveyed hundreds of university students in Pakistan and found that those who relied more on ChatGPT reported increased procrastination, memory loss, and lower GPAs. The researchers attribute this to the chatbot making schoolwork too easy, reducing students' cognitive effort. The study's lead author warned of a "dark side" to excessive generative AI usage.

Company involved
National University of Computer and Emerging Sciences
AI system involved
ChatGPT

4 source articles · read the reporting →

CJEU rules Dun & Bradstreet must explain automated credit decisions under GDPR

A customer was refused a mobile phone contract because of an automated credit assessment by Dun & Bradstreet Austria. The customer took the case to court, which found that Dun & Bradstreet had infringed the GDPR by failing to provide meaningful information about the logic involved. The CJEU ruled that data controllers must explain automated decisions and that trade secrets cannot automatically override the right of access.

Company involved
Dun & Bradstreet Austria GmbH

7 source articles · read the reporting →

Slack trains AI features on user messages and files by default

Slack uses user messages, files, and data to train its machine learning features such as channel recommendations and emoji suggestions. Users are opted in by default and cannot individually opt out; only workspace administrators can request exclusion via email. A user publicly criticized the practice, and Slack acknowledged the policy but did not change it.

Company involved
Slack

10 source articles · read the reporting →

WF-4N6UFD1 Mar 2021

Baltimore schools monitor student laptops for suicide signs using GoGuardian Beacon

Baltimore City Public Schools uses GoGuardian Beacon software to monitor student laptops for signs of suicide. Since March 2021, the system has flagged 786 alerts, with nine students taken to emergency rooms. Privacy advocates warn the monitoring could lead to disciplinary actions, outing of LGBTQ students, and disproportionately affect disadvantaged students. School officials defend the practice as a safeguard.

Company involved
Baltimore City Public Schools
AI system involved
GoGuardian Beacon

10 source articles · read the reporting →

WF-KO86VX1 Jul 2020

MIT apologizes and pulls offline dataset that taught AI to use slurs

MIT permanently removed its 80 Million Tiny Images dataset after researchers discovered it contained thousands of images labeled with racist and misogynistic slurs. The dataset, used to train AI systems for object detection, was built by automatically scraping images from the internet using a list of nouns from WordNet that included derogatory terms. MIT apologized and urged the community to delete copies of the dataset.

Company involved
Massachusetts Institute of Technology
AI system involved
80 Million Tiny Images

10 source articles · read the reporting →

WF-WW27TM1 Jul 2023

Anti-piracy group takes down Books3 dataset used to train Meta's LLaMA

The Danish anti-piracy group Rights Alliance sent a DMCA takedown request to The Eye, which hosted the Books3 dataset containing 196,640 copyrighted books. The dataset was used by Meta to train its LLaMA language model. Authors including Sarah Silverman have filed a class action lawsuit against Meta for using their works without permission. The dataset has been taken offline, but copies remain available.

Company involved
Meta
AI system involved
LLaMA

10 source articles · read the reporting →

WF-6BIZTN1 Mar 2014

Duke University recorded students' faces without proper consent for public dataset

In March 2014, Duke University researchers recorded thousands of students walking to class on campus without their knowledge or proper consent, creating the DukeMTMC dataset of over 2 million image frames. The dataset was placed on a public website and downloaded by academics, security contractors, and military researchers globally, including Chinese companies and military academies linked to surveillance of ethnic minorities. The university took down the public website in April 2019 after an Institutional Review Board investigation found the study deviated significantly from the approved protocol. The lead researcher apologized, stating he took full responsibility for his mistakes.

Company involved
Duke University
AI system involved
DukeMTMC

10 source articles · read the reporting →

WF-GEPFGZ1 Dec 2023

OpenDream AI art site allowed users to generate child sexual abuse material

OpenDream, an AI image generation platform, allowed users to generate and publicly display child sexual abuse material (CSAM) and non-consensual deepfakes from at least December 2023 until July 2024. The platform, operated by CBM Media Pte Ltd in Singapore, offered paid plans with NSFW prompts and models. Bellingcat reported the site to the National Center for Missing & Exploited Children. After Bellingcat's inquiry, the CSAM was removed from the site and search engines, and Google terminated OpenDream's AdSense account.

Company involved
CBM Media Pte Ltd
AI system involved
OpenDream

3 source articles · read the reporting →

WF-NE7SI21 Jan 2012

UCCS professor secretly photographed over 1,700 people for facial recognition research

A University of Colorado Colorado Springs professor, Terrance Boult, led a project that secretly photographed more than 1,700 students, faculty, and passers-by on campus in 2012-2013 to improve facial recognition technology. The photos were published as a public dataset from 2016 until April 2019. University officials defended the research, but a law professor questioned the ethics of the surveillance without consent.

Company involved
University of Colorado Colorado Springs

10 source articles · read the reporting →

DWP algorithm mistakenly flags 200,000 Housing Benefit claimants for fraud review

The UK Department for Work and Pensions (DWP) uses an algorithm to flag Housing Benefit claimants for possible fraud or error. According to a Big Brother Watch investigation, the algorithm has mistakenly flagged 200,000 innocent people, subjecting them to intrusive reviews. Only one in three flagged cases actually had errors, compared to two in three during the pilot. The DWP has spent £4.4 million on these pointless checks.

Company involved
Department for Work and Pensions (DWP)

6 source articles · read the reporting →

WF-RX9YRU9 Jul 2024

Lattice cancels plan to give AI digital workers employee records after backlash

Lattice, an HR software company, announced on July 9th that it would give AI digital workers official employee records. After strong backlash from HR professionals and others on LinkedIn, the company canceled the feature on July 12th, stating it 'will not further pursue digital workers in the product.' The feature was intended to manage AI bots such as Devin and Piper, but the company reversed course.

Company involved
Lattice
AI system involved
Lattice

6 source articles · read the reporting →

Chelmer Valley High School reprimanded for facial recognition DPIA failure

Chelmer Valley High School was issued a reprimand by the ICO for failing to complete a Data Protection Impact Assessment before introducing facial recognition technology for cashless catering. The reprimand was issued on 22 July 2024.

Company involved
Chelmer Valley High School

7 source articles · read the reporting →

WF-G9ODUL11 Jun 2024

Luma's Dream Machine generates video with Disney character

Luma's AI video tool Dream Machine generated a trailer that included a recognizable character from Disney's Monsters, Inc. The company's CEO said a user uploaded an image containing the character, which the system then animated. The incident raises concerns about lack of transparency in training data and potential copyright infringement. Disney has not publicly commented.

Company involved
Luma
AI system involved
Dream Machine

7 source articles · read the reporting →

DWP algorithm approved Kickstart gateways with no trading history or based abroad

An FE Week investigation found that the Department for Work and Pensions (DWP) approved dozens of companies as Kickstart gateways through automated due diligence checks using the Cabinet Office Spotlight Tool, although some had little or no trading history or were based abroad. The DWP said gateways were subject to stringent checks and later said human checks were also used. After the findings were shared with the Treasury and the DWP, the department stopped taking gateway applications and scrapped the requirement for small employers to use gateways from 3 February.

Company involved
Department for Work and Pensions
AI system involved
Cabinet Office Spotlight Tool

3 source articles · read the reporting →

WF-2PVWQU31 May 2026

CBSE OnMark portal vulnerability exposed student data to Google Gemini

A 19-year-old ethical hacker, Nisarga Adhikary, claimed to have hacked the CBSE's digital evaluation ecosystem, revealing that personal information of students was processed by Google's Gemini in automation scripts. The Central Board of Secondary Education (CBSE) stated on May 31, 2026, that the identified vulnerabilities had been contained and other exploitable weaknesses were being ruled out. The board expressed gratitude to alert citizens and ethical hackers who pointed out the weaknesses. No actual data breach was confirmed, but the incident raised concerns about student privacy.

Company involved
Central Board of Secondary Education (CBSE)
AI system involved
OnMark

1 source article · read the reporting →

WF-5TENXX1 Aug 2023

Vanderbilt, Northwestern and University of Texas stop using Turnitin AI detector over false cheating accusations

Several US universities, including Vanderbilt, Northwestern and the University of Texas, have stopped using Turnitin's AI detection tool over concerns that it falsely marks student essays as written by ChatGPT. Vanderbilt estimated that the tool's 1% false-positive rate could have wrongly labelled about 750 of 75,000 papers submitted last year. A Texas professor came under fire for failing half his class after the software identified their essays as AI-generated. Turnitin said its technology is not meant to replace educators' professional discretion.

Company involved
Multiple universities (Vanderbilt University, Northwestern University, University of Texas)
AI system involved
Turnitin's AI detection tool

9 source articles · read the reporting →

← Newerpage 4 of 5Older →