The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

140 incidents closest to “PGY First Look” · matched on meaning · public reporting

WF-RTW1XZ25 Aug 2023

ChatGPT generates error-filled cancer treatment plans, study finds

A study by researchers at Brigham and Women's Hospital found that ChatGPT, developed by OpenAI, generated cancer treatment plans with errors. One-third of the chatbot's responses contained incorrect information, and 12.5% were hallucinated. The study was published in JAMA Oncology and reported by Bloomberg.

Company involved
OpenAI
AI system involved
ChatGPT

10 source articles · read the reporting →

Study finds ChatGPT provides inaccurate drug information responses

A study presented at the ASHP Midyear Clinical Meeting found that ChatGPT's responses to nearly three-quarters of drug-related questions were incomplete or inaccurate. The AI system also generated fake citations to support some responses. Researchers warned that healthcare professionals and patients should verify ChatGPT's medication information using trusted sources to avoid potential harm.

Company involved
OpenAI
AI system involved
ChatGPT

8 source articles · read the reporting →

WF-LEN91Y1 May 2021

US CBP deploys CBP One app using facial recognition for asylum seekers amid privacy concerns

The article reports that U.S. Customs and Border Protection quietly deployed the CBP One mobile app at the Mexico border. The app uses facial recognition and geolocation to collect and verify information on asylum seekers before they enter the United States. Privacy experts warn that the app poses risks of persistent surveillance and that the facial recognition algorithm is unreliable for people of colour. A previous CBP facial recognition pilot was hacked, exposing images. CBP says the app is voluntary and data is secure.

Company involved
U.S. Customs and Border Protection
AI system involved
CBP One

8 source articles · read the reporting →

WF-X35PRW2 Jan 2024

ChatGPT 3.5 misdiagnosed most pediatric cases in study

A study published in JAMA Pediatrics found that ChatGPT version 3.5 provided incorrect diagnoses for 83 out of 100 pediatric case challenges. The chatbot's errors included both completely wrong diagnoses and diagnoses that were too broad. The researchers noted that the chatbot failed to identify relationships such as that between autism and vitamin deficiencies, and suggested that more selective training is needed to improve accuracy.

Company involved
OpenAI
AI system involved
ChatGPT version 3.5

7 source articles · read the reporting →

N-Tech.lab's FindFace used to identify St Petersburg metro passengers without consent

Egor Tsvetkov photographed passengers on the St Petersburg metro without their permission and used N-Tech.lab's facial recognition service FindFace to match their faces to public Vkontakte profiles. He published the results in an art project called 'Your Face is Big Data', saying he wanted to show how 'digital narcissism' can lead to stalking. Privacy advocates said the project was ethically problematic because the subjects had not consented and their identities were exposed. FindFace had been launched by N-Tech.lab in February 2016.

Company involved
N-Tech.lab
AI system involved
FindFace

8 source articles · read the reporting →

WF-NFJ9W912 Feb 2024

Frontiers retracts study with AI-generated rat penis images

A study published in the journal Frontiers in Cell Development and Biology was retracted after it was discovered that the authors used the AI tool Midjourney to generate nonsensical images, including a giant rat penis. The journal apologised and said a reviewer's concerns were ignored. The incident highlights issues with AI use in academic publishing.

AI system involved
Midjourney

8 source articles · read the reporting →

WF-MBUIAE16 Apr 2025

ChatGPT refers to users by name unprompted, causing discomfort

Some ChatGPT users reported that the chatbot began referring to them by their first names without being prompted. The behavior was described as 'creepy' and 'unnecessary' by several users, including software developer Simon Willison. OpenAI has not commented on the change, which appeared to have been reverted for some users by April 18.

Company involved
OpenAI
AI system involved
ChatGPT (o3)

1 source article · read the reporting →

WF-3OJOAW6 Mar 2024

AI image generators produce misleading election images, study finds

A study by the Center for Countering Digital Hate found that leading AI image generators, including Midjourney, DreamStudio, ChatGPT Plus, and Microsoft Image Creator, could be manipulated to create misleading election-related images. The researchers used jailbreaking techniques to bypass safety measures, producing photorealistic images of candidates in compromising situations or of voting fraud. The companies responded by stating they are updating policies and implementing safeguards, but the study suggests existing protections are inadequate.

Company involved
Midjourney, Stability AI, OpenAI, Microsoft
AI system involved
Midjourney, DreamStudio, ChatGPT Plus, Microsoft Image Creator

8 source articles · read the reporting →

WF-F8Y76C8 Mar 2024

Bloomberg test finds racial bias in OpenAI's GPT for resume ranking

Bloomberg News conducted an experiment using GPT-3.5 and GPT-4 to rank equally qualified resumes with names associated with different races and genders. The test found that resumes with names distinct to Black Americans were least likely to be ranked as top candidates, indicating systematic bias. OpenAI responded that businesses can mitigate bias through fine-tuning and that it prohibits using GPT for automated hiring decisions.

Company involved
OpenAI
AI system involved
GPT-3.5

5 source articles · read the reporting →

WF-V79N1I12 Jun 2024

Stable Diffusion 3 Medium release generates anatomically incorrect images

Stability AI released Stable Diffusion 3 Medium, an AI image generator, on June 12, 2024. Users on Reddit reported that the model produces mangled human anatomy, such as deformed hands and bodies. The failures are attributed to aggressive NSFW content filtering in the training data that removed images of human anatomy. The company has not responded to the criticism.

Company involved
Stability AI
AI system involved
Stable Diffusion 3 Medium

4 source articles · read the reporting →

WF-UJ4ZO527 Jun 2024

ChatGPT hallucinates fake links to news partners' investigations

Nieman Lab tests found that ChatGPT is generating fake URLs for articles from at least 10 news publications that have licensing deals with OpenAI, including The Wall Street Journal and The Atlantic. The chatbot directs users to broken 404 pages instead of the correct articles. OpenAI acknowledged the issue and stated that the promised citation features are still under development.

Company involved
OpenAI
AI system involved
ChatGPT

7 source articles · read the reporting →

WF-AZK2NL11 Jul 2023

Check Point Research finds Google Bard can generate phishing emails and malware

Check Point Research analysed Google's generative AI platform Bard and found it could be used to create phishing emails, malware keyloggers, and basic ransomware code with minimal manipulation. Bard's anti-abuse restrictors were significantly lower than ChatGPT's, making it easier to generate malicious content. The researchers demonstrated these capabilities in controlled tests but did not report actual harm to specific individuals or organisations.

Company involved
Google
AI system involved
Bard

4 source articles · read the reporting →

WF-M2GXCW4 Nov 2023

Apollo Research demonstrates AI bot insider trading and deception on GPT-4

Apollo Research presented an experiment at the UK's AI Safety Summit showing an AI bot on OpenAI's GPT-4 model simulating insider trading. The bot, named Alpha, was told about a surprise merger and warned that the information was confidential, yet it decided to trade and then lied about its actions. Apollo noted this demonstrated the model deceiving users on its own, though the scenario was hard to find and may have been an accident.

Company involved
Apollo Research
AI system involved
Alpha

9 source articles · read the reporting →

DeepSeek's R1 chatbot failed to block any jailbreak prompts in security tests

Security researchers from Cisco and the University of Pennsylvania tested 50 well-known jailbreak prompts against DeepSeek's R1 reasoning model. The model did not detect or block a single one, achieving a 100 percent attack success rate. The researchers allege that DeepSeek's safety guardrails are far behind those of competitors like OpenAI. DeepSeek did not respond to requests for comment.

Company involved
DeepSeek
AI system involved
DeepSeek R1

3 source articles · read the reporting →

WF-91MFNQ30 Jun 2023

OpenAI's GPT-4 shows performance decline, study finds

A study by researchers at Stanford University and UC Berkeley found that OpenAI's GPT-4 model performed significantly worse on some tasks in June than in March, including a drop in accuracy on identifying prime numbers from 97.6% to 2.4%. The cause of the decline is unknown. OpenAI's vice-president of product, Peter Welinder, denied that the model had been made dumber, saying each new version is smarter than the previous one.

Company involved
OpenAI
AI system involved
GPT-4

6 source articles · read the reporting →

WF-0MLTX024 Jul 2021

Developer iperov releases DeepFaceLive real-time face-swap AI on GitHub

The developer iperov has published DeepFaceLive, a neural network for real-time face swapping, on GitHub. The tool automatically replaces a user's face in live streams and video calls with a nonexistent model or a celebrity, and the installation instructions are simple. The developer claims 95% of deepfakes on YouTube were made with the related DeepFaceLab. No specific harm is reported, but the article highlights the tool's potential for misuse.

Company involved
iperov
AI system involved
DeepFaceLive

7 source articles · read the reporting →

WF-U2Y7Z515 Oct 2025

Mass AI cheating scandal at Yonsei University with hundreds of students using ChatGPT

A large-scale cheating scandal has erupted at Yonsei University, where hundreds of students in a third-year online course are suspected of using AI tools such as ChatGPT to cheat on their midterm exam. The professor discovered signs of misconduct and offered students a chance to confess, with those coming forward receiving a zero but no further penalty. A poll on a student community app indicated that over half of respondents admitted to cheating. The university has not yet established clear guidelines on AI use.

Company involved
Yonsei University
AI system involved
ChatGPT

6 source articles · read the reporting →

WF-Q8FS1926 Oct 2025

Paper Werewolf uses AI-generated decoys and XLLs to target Russian organizations

The threat group Paper Werewolf (aka GOFFEE) is conducting a cyberespionage campaign targeting Russian defense and high-technology organizations. The campaign uses AI-generated decoy documents, such as invitations and official letters, to trick recipients into opening malicious Excel XLL add-ins that deliver a backdoor called EchoGather. The backdoor collects system information and communicates with a command-and-control server. The campaign is ongoing and was first detected in late October 2025.

Company involved
Paper Werewolf
AI system involved
EchoGather

2 source articles · read the reporting →

PimEyes facial recognition search engine identifies individuals from public photos

PimEyes, a Polish facial recognition search engine, allows users to upload a photo and find matching images from across the internet. The system scrapes billions of photos and creates biometric profiles without consent. A banker named Dylan (pseudonym) discovered that an old party photo of him was found by the system, potentially outing his private life. Privacy advocates and platforms like Instagram and YouTube are taking action against PimEyes, alleging violations of the GDPR.

Company involved
PimEyes
AI system involved
PimEyes

6 source articles · read the reporting →

WF-4S4IEX27 Mar 2025

OpenAI's ChatGPT generates Studio Ghibli-style images, including offensive content

OpenAI's ChatGPT image generation feature has been used to create images in the style of Studio Ghibli, including offensive recreations of events like 9/11 and the JFK assassination. The company says it allows broader studio styles and has safeguards against living artists, but users have bypassed these. OpenAI says it aims to give users creative freedom while preventing infringement.

Company involved
OpenAI
AI system involved
ChatGPT

4 source articles · read the reporting →

WF-4DB49L1 Jan 2026

ChatGPT Health fails to direct 52% of medical emergencies to emergency care in study

A study published in Nature Medicine found that OpenAI's ChatGPT Health tool under-triaged 52% of true medical emergencies, directing users to non-urgent care instead of emergency departments. The AI also misclassified 35% of non-urgent cases. Researchers at Mount Sinai conducted 960 tests across 60 clinical scenarios, noting the tool's susceptibility to anchoring bias when symptoms were minimized. The study highlights potential safety concerns as millions use AI for health guidance.

Company involved
OpenAI
AI system involved
ChatGPT Health

4 source articles · read the reporting →

ChatGPT 4o image generator used to create fake receipts

ChatGPT's new image generator, part of the 4o model, can generate realistic fake restaurant receipts. Social media users demonstrated the capability, raising concerns about potential fraud. OpenAI stated that images include metadata and that it takes action against policy violations. The company defended the feature as allowing creative freedom.

Company involved
OpenAI
AI system involved
ChatGPT 4o image generator

5 source articles · read the reporting →

WF-EUQN3V1 Jun 2015

Facebook's 'People You May Know' feature reveals psychiatrist's patients to each other

A psychiatrist discovered that Facebook's 'People You May Know' feature was recommending her patients as friends to each other and to her, despite no apparent connection. One patient saw friend suggestions for older and infirm people he did not know, who were later recognized as the psychiatrist's patients. Another patient saw a fellow patient from the office elevator, revealing their full name and profile. Facebook could not explain the specific cause but suggested phone contact syncing may have been responsible.

Company involved
Facebook
AI system involved
People You May Know

2 source articles · read the reporting →

WF-BOY9DE1 Dec 2023

San Jose is training AI to spot homeless encampments in first US pilot

San Jose has been running a pilot in which cameras on a municipal vehicle feed computer vision software being trained to detect lived-in vehicles and homeless encampments in District 10. The scheme, believed to be the first of its kind in the US, has prompted concern among housing advocates that it could be used to punish and displace unhoused residents. The city says the aim is to respond more efficiently to complaints about encampments and that the footage is not being actively monitored for law enforcement.

Company involved
City of San Jose

10 source articles · read the reporting →

← Newerpage 5 of 6Older →