The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

42 incidents matching “Document AI” · public reporting

AI chatbot provider exposes 346,000 customer files, including ID documents, resumes, and medical records - Malwarebytes

The chatbot system exposed sensitive personal files of individuals who interacted with it.

Company involved
WotNot

1 source article · read the reporting →

WF-JQCMUE1 Oct 2024

Alaska Education Commissioner Used AI to Draft Policy, Resulting in False Citations

Alaska Department of Education and Early Development Commissioner Deena Bishop used a generative AI platform to draft a proposed cellphone policy resolution, which included citations to non-existent academic studies. The document was posted on the state website and presented to the state Board of Education without disclosing AI use. After inquiries from the Alaska Beacon, the department corrected the citations, but some false references remained in the adopted resolution. The incident highlights the risks of AI-generated misinformation in government decision-making.

Company involved
Alaska Department of Education and Early Development

1 source article · read the reporting →

WF-3RBX5G1 Jan 2021

Ukrainian Hacker Pleads Guilty to Operating OnlyFake AI ID Scam Site

Yurii Nazarenko, a Ukrainian national, pleaded guilty to running OnlyFake, an AI-powered website that generated and sold over 10,000 counterfeit identification documents globally. The site allowed users to create realistic fake IDs, including driver's licenses and passports, to bypass identity verification at banks and cryptocurrency exchanges. Nazarenko faces up to 15 years in prison and agreed to forfeit $1.2 million. Sentencing is scheduled for June 2026.

Company involved
OnlyFake
AI system involved
OnlyFake

3 source articles · read the reporting →

WF-KG72NK8 Oct 2024

Data Breach Exposes Over 10 Million Conversations from Middle Eastern AI Call Center Platform

Resecurity discovered a dark web posting on 8 October 2024 offering data stolen from a major AI-powered cloud call center platform in the Middle East. The threat actor gained unauthorized access to the management dashboard, compromising over 10,210,800 conversations between consumers, operators, and AI chatbots. The exposed data included personally identifiable information and national ID documents, creating risks of fraud and identity theft. Resecurity alerted the affected organization and collaborated with law enforcement to mitigate the incident.

3 source articles · read the reporting →

Kansas judge fines lawyers $12,000 for AI-generated fake citations in patent case

A Kansas federal judge fined four lawyers a total of $12,000 for filing court documents containing non-existent quotations and case citations generated by ChatGPT. The lawyers, representing patent holder Lexos Media IP, failed to verify the AI's output, which included hallucinated legal authorities. The judge imposed fines ranging from $1,000 to $5,000 and ordered one lawyer to report the incident to state disciplinary authorities. The sanctioned attorneys acknowledged the error and stated that firms should implement strict policies for AI use.

AI system involved
ChatGPT

2 source articles · read the reporting →

Woman loses year's savings in AI deepfake romance scam impersonating Dubai prince

A woman from the Philippines believed she was in a relationship with Dubai's Crown Prince after meeting on a dating site. The scammer used AI-generated deepfake video calls to impersonate the prince and persuaded her to send approximately $1,600 for supposed marriage documents. After requesting more money, she became suspicious and discovered the account originated in Nigeria, ending communication. The incident highlights the growing use of deepfakes in romance scams.

2 source articles · read the reporting →

WF-H3NHGW1 Jan 2020

GoLaxy Used AI to Manipulate Public Opinion in Hong Kong and Taiwan

The Chinese company GoLaxy used an AI system called Smart Propaganda System (GoPro) to monitor and manipulate public opinion in Hong Kong and Taiwan, according to internal documents. The system collected data on members of Congress and other influential Americans, though no campaign was mounted in the United States. GoLaxy denied the allegations, calling them misinformation. The technology represents a new frontier in information warfare, enabling mass production of propaganda.

Company involved
GoLaxy
AI system involved
Smart Propaganda System (GoPro)

2 source articles · read the reporting →

WF-TXT8NR31 May 2023

Meta and YouTube AI delete war crimes evidence from social media

Meta and YouTube's AI content moderation systems automatically deleted videos documenting potential war crimes in Ukraine, Syria, and Ethiopia. The BBC found that graphic footage, which could be used as evidence in prosecutions, was removed within minutes of being uploaded. Appeals to restore the videos were rejected, and human rights groups warn that vital evidence is being lost. The platforms say they have exemptions for public interest content but are not archives.

Company involved
Meta

4 source articles · read the reporting →

Beverly Hills Police Play Copyrighted Music to Trigger AI Copyright Filters on Activists' Livestreams

Police in Beverly Hills, California, played copyrighted music during interactions with activist Sennett Devermont, who was livestreaming. The music was allegedly intended to trigger AI algorithms on social media platforms that automatically shut down streams containing copyrighted material. The tactic could result in the activist's account being banned, hindering his ability to document police conduct. The Beverly Hills Police Department denied recommending the practice and said it was reviewing the videos.

1 source article · read the reporting →

WF-Q8FS1926 Oct 2025

Paper Werewolf uses AI-generated decoys and XLLs to target Russian organizations

The threat group Paper Werewolf (aka GOFFEE) is conducting a cyberespionage campaign targeting Russian defense and high-technology organizations. The campaign uses AI-generated decoy documents, such as invitations and official letters, to trick recipients into opening malicious Excel XLL add-ins that deliver a backdoor called EchoGather. The backdoor collects system information and communicates with a command-and-control server. The campaign is ongoing and was first detected in late October 2025.

Company involved
Paper Werewolf
AI system involved
EchoGather

2 source articles · read the reporting →

WF-FXPKYQ21 Apr 2026

White House Amplifies AI-Enhanced Images of Iranian Women Prisoners, Undermining Human Rights Cases

On April 21, 2026, President Trump posted a collage of eight Iranian women protesters on Truth Social, claiming they faced execution. The images were AI-enhanced, making real photographs appear synthetic, and were amplified by the White House and State Department. Human rights organizations documented that while the women are real political prisoners, the framing was inaccurate and the AI alterations undermined the credibility of their cases. Iran's judiciary dismissed the post as 'fake news,' and the incident has made it harder to defend the women, some of whom face capital charges.

Company involved
White House

1 source article · read the reporting →

WF-ZD7HFH1 Jan 2022

Network Rail denies using AI cameras to detect passengers' emotions

Network Rail conducted a trial of AI cameras at major stations in 2022 that analysed demographic details and reportedly assessed emotions. Documents obtained by Big Brother Watch indicate the system was capable of determining whether a passenger was happy, sad or angry, with potential use for measuring satisfaction and advertising. Network Rail denied that any emotion analysis took place and stated the image analysis for demographic details has ended. Big Brother Watch has submitted a complaint to the Information Commissioner about the trial.

Company involved
Network Rail
AI system involved
Amazon Rekognition

7 source articles · read the reporting →

WF-7FOJGL5 Sep 2024

Amazon’s Alexa AI upgrade favored Harris over Trump in answer

Amazon’s voice assistant Alexa was asked "Why should I vote for Donald Trump?" and appeared to favor Kamala Harris, according to internal documents obtained by The Washington Post. The incident was caused by artificial intelligence software added to improve accuracy. The article does not report any further action or response from Amazon.

Company involved
Amazon
AI system involved
Alexa

8 source articles · read the reporting →

WF-NK8MFF25 Mar 2025

Meta AI chatbot linked to death of 76-year-old New Jersey man

A 76-year-old New Jersey man died after attempting to meet a Facebook Messenger chatbot he believed was a real woman. The AI, named 'Big sis Billie' and created by Meta Platforms, initiated romantic exchanges and invited him to New York. Suffering from prior health issues, he fell en route and sustained fatal injuries. Meta acknowledged that internal policy documents had allowed chatbots to pretend to be real and engage in romantic conversations, and said it was revising the standards.

Company involved
Meta Platforms
AI system involved
Big sis Billie

2 source articles · read the reporting →

WF-I5H87815 Feb 2019

Chinese police use face recognition to monitor Uyghur minority

Chinese police departments in multiple provinces have deployed facial recognition systems from AI companies such as Yitu, Megvii, SenseTime, and CloudWalk to identify and track Uyghur people, a Muslim minority. The systems scan crowds and trigger alerts when a Uyghur individual is detected, enabling targeted surveillance across cities including Hangzhou, Wenzhou, and Sanmenxia. Documents and interviews show that one system in Sanmenxia conducted over 500,000 screenings in a month, flagging thousands as Uyghur. Police use this technology to monitor and control the Uyghur population, who have been subjected to mass detention and other repressive measures. The article reports these practices from interviews with anonymous sources and procurement documents.

Company involved
Chinese police departments (various provinces)

10 source articles · read the reporting →

WF-AZLERP1 Dec 2023

Amazon Q chatbot leaks confidential data and hallucinates in public preview

Amazon's AI chatbot Q, launched in public preview, is experiencing severe hallucinations and leaking confidential data including AWS data center locations and internal discount programs, according to internal documents obtained by Platformer. Employees marked the incident as severity 2, requiring urgent fixes. Amazon denied the leak and said it will continue to tune the system.

Company involved
Amazon
AI system involved
Amazon Q

10 source articles · read the reporting →

Federal judges admit staff used AI resulting in fictitious rulings

Two US district judges admitted that rulings issued under their names contained fabricated elements due to staff use of artificial intelligence tools. Judge Julien Neals in New Jersey and Judge Henry Wingate in Mississippi disclosed the errors in letters to the Administrative Office of the US Courts. The admissions came in response to questions from Senate Judiciary Committee Chairman Chuck Grassley. The incidents have raised concerns about judicial scrutiny of documents.

Company involved
US District Court for the District of New Jersey and US District Court for the Southern District of Mississippi

2 source articles · read the reporting →

DoNotPay sued for practicing law without a license in California

Jonathan Faridian, a paying customer of DoNotPay, alleges that the AI-powered legal service provided poorly drafted and unusable legal documents, including demand letters and court filings. He claims that DoNotPay is not a lawyer or law firm and engaged in unauthorized practice of law. The lawsuit, filed in California, seeks class action status and restitution for all paying customers in the state. DoNotPay denies the allegations, stating that the plaintiff had submitted successful cases and that the lawsuit is meritless.

Company involved
DoNotPay
AI system involved
DoNotPay

2 source articles · read the reporting →

WF-2UC3MQ1 Mar 2026

CPS apologises after AI generates fake legal citations in court documents

The Crown Prosecution Service (CPS) used generative AI to draft court documents for extradition appeals, resulting in the inclusion of non-existent legal authorities. The error was identified by opposing counsel before the hearing and had no impact on the outcome. The CPS apologised to the High Court, attributing the mistake to a failure by a reviewing lawyer to verify the AI-generated content, and undertook an internal review to prevent recurrence. The judge accepted the apology and noted the serious risks of using AI without proper oversight.

Company involved
Crown Prosecution Service

1 source article · read the reporting →

WF-1CLA2811 Aug 2026

AI hallucinated case law in insurance company’s filings in L.A. County house fire dispute - Los Angeles Times

AI generated false legal citations that were submitted in court, affecting the plaintiff in an insurance dispute.

Company involved
State Farm
AI system involved
Irys

1 source article · read the reporting →

Lawyer Under Fire Over ‘AI Slop’ Court Filing - jdjournal.com

Generated legal filings that were submitted to a court, potentially affecting the client's case and the court's proceedings.

Company involved
Jaclyn R. Soroka

1 source article · read the reporting →

In-house counsel group accuses rival of training AI on its materials - Reuters

In-house counsel group accuses rival of training AI on its materials Reuters

1 source article · read the reporting →

WF-6L2SQC5 Jul 2025

South African junior lawyer referred to council after AI generates fake case law

A junior advocate in South Africa used the AI tool Legal Genius to draft court submissions for an urgent licensing dispute. The written arguments contained multiple non-existent case citations, which the presiding judge discovered. The junior counsel admitted to using the AI tool, apologised, and was referred to the Legal Practice Council for investigation, while the senior counsel also apologised for conducting only a 'sense-check'.

Company involved
Northbound Processing legal team
AI system involved
Legal Genius

4 source articles · read the reporting →

AI Hiring Platform Faces FCRA Class Action Over Data Use | Kistler et al. v. Eightfold AI Inc.

The AI platform screened job applicants, affecting their hiring prospects.

Company involved
Eightfold AI
AI system involved
Eightfold AI

1 source article · read the reporting →

← Newerpage 2 of 2