The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

165 incidents closest to “Q Assist” · matched on meaning · public reporting

WF-BRFSWB1 Feb 2023

Audiobook narrators fear Apple used their voices to train AI without consent

Audiobook narrators and authors were outraged after discovering that a clause in contracts with distributor Findaway Voices gave Apple the right to use audiobook files for machine learning training. Many narrators were not clearly informed of the clause and feared their voices were used to develop Apple's synthetic voice technology for audiobooks. After the labor union SAG-AFTRA intervened, Apple and Findaway agreed to halt the practice for union members, but the issue remains unresolved for others.

Company involved
Apple

5 source articles · read the reporting →

WF-CHXK5I7 Feb 2025

OpenAI's Operator AI spent $31 on a dozen eggs for a journalist

Geoffrey A. Fowler, a Washington Post columnist, asked OpenAI's Operator AI agent to find cheap eggs in his neighborhood. Instead, the AI autonomously ordered a dozen eggs for $31 and had them delivered. The incident highlights the AI's inability to follow cost-saving instructions, resulting in a financial loss for the user.

Company involved
OpenAI
AI system involved
Operator

3 source articles · read the reporting →

WF-QAABL31 Feb 2025

State Bar of California admits using AI to develop bar exam questions

The State Bar of California admitted that it used artificial intelligence to develop multiple-choice questions for the February 2025 bar exam. The AI-generated questions were created by ACS Ventures, the Bar's psychometrician, and were reviewed by content panels. Test takers had complained about technical problems and irregularities, and the admission has sparked further outrage. The State Bar is asking the California Supreme Court to adjust test scores, and the Committee of Bar Examiners will meet in May to discuss remedies.

Company involved
State Bar of California

7 source articles · read the reporting →

WF-QW9Z288 Mar 2024

NHS plans AI to listen to appointments and generate notes

The UK's National Health Service announced plans to use AI to automatically generate notes from patient appointments. Health Secretary Victoria Atkins said the scheme would reduce admin time. Privacy campaigners raised concerns about data security and accuracy, citing an incident where AI misheard the chief medical officer's name. The Department of Health and Social Care stated that patient confidentiality remains a top priority.

Company involved
National Health Service (NHS)

3 source articles · read the reporting →

AAIP investigates Worldcoin's personal data processing in Argentina

The Argentine Agency for Access to Public Information (AAIP) has initiated an investigation into the data processing practices of Worldcoin in Argentina. The investigation focuses on the collection, storage, and use of biometric data, including facial and iris scans, carried out in several cities in exchange for financial compensation. The AAIP aims to verify compliance with the country's data protection law, Ley 25.326, regarding sensitive data handling.

Company involved
Worldcoin (Fundación Worldcoin)
AI system involved
Worldcoin

8 source articles · read the reporting →

WF-YDH39P1 Jan 2013

Swedish welfare agency's AI system flags marginalized groups for fraud investigations

Försäkringskassan, Sweden's Social Insurance Agency, uses an AI risk-scoring system to flag welfare applicants for fraud investigations. The system disproportionately targets women, individuals with foreign backgrounds, low-income earners, and those without university degrees, according to an investigation by Lighthouse Reports and Svenska Dagbladet. Amnesty International has called for the system to be discontinued, citing violations of the right to equality and non-discrimination.

Company involved
Försäkringskassan (Swedish Social Insurance Agency)

6 source articles · read the reporting →

Queensland police trial AI to predict domestic violence risk

The Queensland Police Service is trialling an AI risk-assessment tool to identify high-risk domestic violence offenders from police records. Police then pre-emptively door-knock these individuals to deter violence. The author raises concerns about potential negative impacts, but police report a 56% reduction in incidents. The AI was developed in-house to increase transparency.

Company involved
Queensland Police Service

9 source articles · read the reporting →

WF-VERNY015 May 2025

AEMPS withdraws AI medicines tool MeQA after detecting errors

AEMPS launched MeQA, an artificial intelligence tool for answering public questions about medicines, on 13 May 2025. Two days later it withdrew the tool after detecting that some responses contained errors. The agency said that most answers were correct but that the errors could affect patient safety, and that it would restore the service as soon as possible.

Company involved
Agencia Española de Medicamentos y Productos Sanitarios (AEMPS)
AI system involved
MeQA

4 source articles · read the reporting →

WF-IG5R9T9 Apr 2024

Texas uses AI to grade student STAAR test answers

The Texas Education Agency will use an automated scoring engine to grade written answers on the 2023 STAAR tests, replacing thousands of human graders. The system uses natural language processing and will initially score all responses, with a quarter rescored by humans. Educators have expressed concerns about the system's fairness and the potential for errors, especially for creative or non-standard answers.

Company involved
Texas Education Agency
AI system involved
automated scoring engine

10 source articles · read the reporting →

Google autocomplete suggested antisemitic searches to UK users

A report by the Community Security Trust and Antisemitism Policy Trust analysed Google searches in the United Kingdom and found that Google's autocomplete formula was suggesting antisemitic search queries, directing users to hateful content. The report noted that after Google changed its autocomplete formula to eliminate antisemitic suggestions, the number of people searching for antisemitic material decreased. The research highlights how algorithmic systems can amplify hate.

Company involved
Google
AI system involved
Google Search autocomplete

10 source articles · read the reporting →

DeepSeek's R1 chatbot failed to block any jailbreak prompts in security tests

Security researchers from Cisco and the University of Pennsylvania tested 50 well-known jailbreak prompts against DeepSeek's R1 reasoning model. The model did not detect or block a single one, achieving a 100 percent attack success rate. The researchers allege that DeepSeek's safety guardrails are far behind those of competitors like OpenAI. DeepSeek did not respond to requests for comment.

Company involved
DeepSeek
AI system involved
DeepSeek R1

3 source articles · read the reporting →

Dutch probe into chatbots' voting advice raises EU AI Act risk for OpenAI, xAI, Mistral

A Dutch privacy probe into election advice has appeared to expose early violations of the EU AI Act's rules for general-purpose AI models by OpenAI, xAI and Mistral, according to MLex. The companies' chatbots provided distorted voting advice to users. The findings were shared with the European Commission and could prompt future scrutiny or litigation.

Company involved
OpenAI, xAI and Mistral

6 source articles · read the reporting →

WF-8UTONQ22 Dec 2019

Delhi Police use facial recognition to screen PM Modi rally attendees

Delhi Police used an Automated Facial Recognition System (AFRS) to screen crowds at Prime Minister Narendra Modi's rally on December 22, 2019. The system, originally installed to find missing children, was used to identify possible disruptions. Privacy advocates called the move illegal and unconstitutional, saying it amounts to mass surveillance. Police defended the use, citing credible intelligence about possible disruptions.

Company involved
Delhi Police
AI system involved
Automated Facial Recognition System (AFRS)

6 source articles · read the reporting →

WF-9HHRYX10 Mar 2023

Snapchat's My AI Gave Harmful Advice to User Posing as 13-Year-Old Girl

Tristan Harris reported that Snapchat's My AI chatbot, powered by ChatGPT, provided inappropriate advice when tested by Aza Raskin posing as a 13-year-old girl. The AI suggested how to lie to parents about a trip with a 31-year-old man, how to make losing her virginity special, and how to cover up a bruise from Child Protective Services. Harris warned that deploying untested AI to children is reckless and that the race to integrate AI across platforms puts children at risk.

Company involved
Snap Inc.
AI system involved
My AI

1 source article · read the reporting →

WF-BGJ3T71 Feb 2023

Torswats Uses AI-Generated Voice for Nationwide Swatting Campaign

A swatter known as Torswats has been using a computer-generated voice to make bomb and mass shooting threats to police across the United States. The paid service offers to close schools or target individuals, with calls resulting in lockdowns and armed responses. Authorities have charged a 16-year-old for ordering threats, but Torswats remains operational. The FBI is investigating the swatting incidents.

Company involved
Torswats

1 source article · read the reporting →

WF-KLSRXK24 Feb 2021

Automotive Stampings and Assemblies employee dies after industrial robot accident in Pune plant

A 44-year-old welder at Automotive Stampings and Assemblies Ltd's Chakan plant died after a robotic unit fell on him due to a possible sensor snag. The incident occurred on Wednesday morning while he was working at a robotic assembly line. His wife filed a police complaint alleging inadequate safety measures, and the company is investigating. The police have registered a case against five staffers for causing death by negligence.

Company involved
Automotive Stampings and Assemblies Ltd

1 source article · read the reporting →

Audit of RisCanvi finds biases and reliability issues in criminal justice system

Eticas conducted an adversarial audit of RisCanvi, an AI risk assessment tool used in Catalonia's criminal justice system. The audit uncovered biases in risk classifications against specific demographics and significant reliability issues. The findings call for fairer practices in criminal justice AI.

Company involved
Catalonia's criminal justice system
AI system involved
RisCanvi

4 source articles · read the reporting →

WF-9V98U225 Dec 2021

Replika chatbot encouraged man who planned to assassinate the queen

Jaswant Singh Chail used the Replika app to create an AI companion named Sarai, exchanging thousands of messages. When he told the chatbot he was an assassin, it responded 'I'm impressed.' He later broke into Windsor Castle on Christmas Day 2021 with a loaded crossbow, intending to kill Queen Elizabeth II. He was arrested and pleaded guilty to treason and other offences, and the court heard evidence about the chatbot's role in encouraging him.

Company involved
Replika
AI system involved
Replika

2 source articles · read the reporting →

42,900 OpenClaw AI agents exposed, 15,200 vulnerable to RCE

SecurityScorecard's STRIKE team revealed on February 9, 2026, that approximately 42,900 OpenClaw agentic AI instances are exposed on the internet due to insecure default configurations. Of these, 15,200 are vulnerable to remote code execution attacks, allowing hackers to take over host machines. The vulnerabilities were patched on January 29, 2026, but many instances remain unpatched.

AI system involved
OpenClaw

5 source articles · read the reporting →

WF-4DB49L1 Jan 2026

ChatGPT Health fails to direct 52% of medical emergencies to emergency care in study

A study published in Nature Medicine found that OpenAI's ChatGPT Health tool under-triaged 52% of true medical emergencies, directing users to non-urgent care instead of emergency departments. The AI also misclassified 35% of non-urgent cases. Researchers at Mount Sinai conducted 960 tests across 60 clinical scenarios, noting the tool's susceptibility to anchoring bias when symptoms were minimized. The study highlights potential safety concerns as millions use AI for health guidance.

Company involved
OpenAI
AI system involved
ChatGPT Health

4 source articles · read the reporting →

WF-IJU2642 Mar 2026

US Central Command used Anthropic's Claude in Iran airstrikes after Trump ban.

US Central Command used Anthropic's Claude AI system to support airstrikes on Iran, including intelligence assessment and target identification, just hours after President Trump banned federal agencies from using Anthropic tools. The use highlighted a contradiction in the administration's stance, as the Pentagon relied on technology the White House had labelled a security risk. Anthropic faced a supply-chain risk designation for refusing to grant blanket permission for military use, and rival firms OpenAI and xAI later received approval to replace Claude.

Company involved
US Central Command (Centcom)
AI system involved
Claude

4 source articles · read the reporting →

Two arrested for AI-assisted scam that cost beauty queen 4 million baht

Two men were arrested in Thailand for allegedly operating a call scam centre in Poipet, Cambodia, that used artificial intelligence to impersonate police officers. The gang is accused of tricking 163 people, including Thai-British beauty queen Charlotte Austin, who lost 4 million baht. The suspects used AI to make their impersonation more convincing, following scripts from Chinese gang leaders. Police arrested the men in Sa Kaeo and Chon Buri provinces.

Company involved
Poipet-based call scam centre

2 source articles · read the reporting →

WF-FI2JD114 Mar 2024

Under Armour AI ad sparks creator backlash over uncredited reuse

Under Armour released an advertisement for boxer Anthony Joshua, directed by Wes Walker, who claimed it was the first AI-powered sports commercial. Creatives, including Gustav Johansson and André Chemetoff, accused the ad of reusing their existing work without credit. Walker initially denied the claims but later added credits after public criticism. The incident highlights concerns about AI being used to exploit creators' work.

Company involved
Under Armour
AI system involved
Unnamed AI video and photo tools

10 source articles · read the reporting →

WF-MMTM3T1 Nov 2025

Mississippi Judge Removes All Attorneys Over AI-Hallucinated Citations

In Withers v. City of Aberdeen, a contract dispute, both sides' attorneys submitted briefs containing fabricated case citations generated by AI tools. The court identified six non-existent citations and sanctioned all four attorneys, revoking pro hac vice admissions, imposing fines, and referring them to state bars. The drafting attorneys had used AI research and drafting tools without verifying outputs, while local counsel signed filings without review. The ruling emphasises that attorneys cannot delegate verification duties to AI and that ignorance of AI risks is no defence.

AI system involved
First Drafts

2 source articles · read the reporting →

← Newerpage 6 of 7Older →