The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

68 incidents closest to “Babel Street Rosette” · matched on meaning · public reporting

Police facial recognition robot identifies anger and distress

A police facial recognition robot identifies anger and distress. The article gives no further details about the system, its deployment, or any consequences.

8 source articles · read the reporting →

WF-MH3TMN1 Jan 2017

IARPA Janus Benchmark C dataset used faces from YouTube without consent

The IARPA Janus Benchmark C (IJB-C) dataset, published in 2017, contains 21,294 images and names of 3,531 individuals scraped from YouTube, Flickr, and Wikimedia without their consent. The dataset was created for face recognition benchmarking to improve intelligence analysis. Jillian York, a digital rights activist, had 41 frames of her face taken from a YouTube video without her knowledge or permission. The dataset violates YouTube's terms of service and raises privacy concerns.

Company involved
IARPA (Intelligence Advanced Research Projects Activity)
AI system involved
IARPA Janus Benchmark C (IJB-C)

6 source articles · read the reporting →

Dutch police predictive policing project in Roermond faces discrimination allegations

Amnesty International's report alleges that the Dutch police's Sensing predictive policing project in Roermond uses algorithms that result in automated discrimination and mass surveillance. The system assesses crime risk for individuals and locations, potentially targeting marginalized communities. The report calls for accountability and human rights safeguards.

Company involved
Dutch police
AI system involved
Sensing project

10 source articles · read the reporting →

WF-W7U79Y20 Dec 2024

Russian disinformation network Matryoshka uses AI deepfakes on Bluesky

The Russian disinformation network 'Matryoshka' has launched a campaign on Bluesky, posting AI-generated videos that impersonate academics to spread anti-Ukraine narratives. The videos use AI to replicate the voices of real professors, making false claims about the war. The campaign was detected by Bellingcat founder Eliot Higgins and the Bot Blocker project. The videos have gained limited traction on Bluesky so far, but researchers warn the platform is more vulnerable to such disinformation.

Company involved
Matryoshka (Russian disinformation network)

3 source articles · read the reporting →

WF-N16FBJ1 Oct 2022

Israeli military uses Smash Hopper turret against Palestinian protesters

The Israeli military deployed Smash Hopper remote-controlled gun turrets in the West Bank. The turrets fire stun grenades, less-than-lethal bullets, and tear gas at Palestinian protesters without warning. The systems were observed at Al-Aroub refugee camp and Hebron in October 2022. The Smash Hopper is produced by Smart Shooter.

Company involved
Israeli military
AI system involved
Smash Hopper

10 source articles · read the reporting →

WF-2ZNE3K8 Jan 2018

Huawei and Megvii tested Uyghur alarm facial recognition system

Huawei and Megvii conducted an interoperability test in 2018 that verified a 'Uyghur alarm' function in Megvii's facial recognition system, according to a confidential report found by IPVM. The system could identify Uyghur minorities and alert police. Huawei denied real-world application and deleted the report after IPVM inquired. Megvii denied targeting ethnic groups. The test report was publicly accessible on Huawei's European website.

Company involved
Huawei
AI system involved
Megvii Dynamic Face Recognition

10 source articles · read the reporting →

WF-SHI9JU1 Nov 2025

AI-generated fake Buckingham Palace Christmas market misleads tourists

AI-generated images of a Christmas market at Buckingham Palace were posted on social media, leading tourists to travel to the palace only to find locked gates and puddles. The Royal Collection Trust clarified that no such market exists. The source of the AI images is unknown.

5 source articles · read the reporting →

WF-CQ2Y4G1 Jan 2019

Serbia's Ministry of Interior deploys thousands of Huawei facial recognition cameras in Belgrade

In early 2019, Serbia's Ministry of Interior announced the deployment of a thousand smart surveillance cameras with facial and license plate recognition capabilities, supplied by Huawei, across Belgrade. The Ministry refused to release information about the project, citing confidentiality, and failed to conduct a legally required Data Protection Impact Assessment. Citizens responded by crowdsourcing camera locations via the hashtag #hiljadekamera and a website. The system remains in operation with no public debate or transparency.

Company involved
Ministry of Interior
AI system involved
Huawei smart surveillance cameras

10 source articles · read the reporting →

WF-HKRG5R7 Jul 2017

Hamburg deployed mass surveillance for G20 Summit

The city of Hamburg planned to use predictive policing software and facial recognition cameras for mass surveillance during the G20 Summit in July 2017. The surveillance was enforced by paramilitary means, creating a democracy-free zone. DiEM25 wrote an open letter to the Senate of Hamburg to protest the measures.

Company involved
Senate of Hamburg

10 source articles · read the reporting →

WF-IH3QYR1 Mar 2022

Woman arrested in Moscow after facial recognition detection on Metro

A woman in Moscow was arrested on 1 March 2022 after she posted a coded protest message on social media. She believes she was detected by facial-recognition software on the Moscow Metro system. The authorities presented a screenshot of her tweet in her court hearing. The article reports that she was arrested while taking a train, five days after her post.

Company involved
Moscow Metro

10 source articles · read the reporting →

WF-8SRP7Y18 Dec 2024

Fact-check: AI-generated image of Bono and Geldof with Israeli flags debunked

A satirical account created an image of Bono and Bob Geldof holding Israeli flags using Grok AI, and former Greek finance minister Yanis Varoufakis shared it on X. The image was viewed more than 306,000 times before the creator confirmed to The Journal that it was AI-generated and intended as satire. X attached a community note to Varoufakis's post, but the tweet remained live at the time of writing.

AI system involved
Grok AI

2 source articles · read the reporting →

Tourists rescued after following ChatGPT route in Tatra mountains

Three Polish tourists used ChatGPT to plan a hiking route from Hala Gąsienicowa to Dolina Pięciu Stawów in the Tatra mountains. The AI recommended a difficult route through Przełęcz Krzyżne, which proved too dangerous in winter conditions with limited visibility and icy rocks. The tourists called the TOPR rescue service and were safely brought down. A TOPR rescuer advised against relying on ChatGPT or Google Maps for mountain route planning.

Company involved
OpenAI
AI system involved
ChatGPT

2 source articles · read the reporting →

LINAGORA closes Lucie 7B after user mockery

LINAGORA, a French open-source software company, launched a beta version of its large language model Lucie 7B. The model was intended to be a transparent and ethical alternative to big tech AI. However, after users tested it and highlighted its shortcomings, the model was mocked online. LINAGORA subsequently closed the platform to address the issues and collect more data.

Company involved
LINAGORA
AI system involved
Lucie 7B

6 source articles · read the reporting →

The Brutalist film faces controversy over AI-enhanced Hungarian accents

The film "The Brutalist" sparked controversy after its editor revealed that AI software Respeecher was used to enhance the Hungarian accents of actors Adrien Brody and Felicity Jones, and to create architectural drawings. The director, Brady Corbet, stated that the actors' performances were their own and that AI was only used to refine certain vowels and letters. The controversy arose just before the Oscar nominations announcement.

Company involved
A24
AI system involved
Respeecher

5 source articles · read the reporting →

DeepSeek-R1 censors 85% of sensitive Chinese political prompts in tests

Promptfoo tested DeepSeek-R1 against a dataset of 1,360 politically sensitive prompts and found that about 85% of them were refused. The refusals followed a standard form aligned with Chinese Communist Party policy. The testing also demonstrated that the censorship could be trivially bypassed using simple jailbreak techniques, such as prompt injection or changing the context.

Company involved
DeepSeek
AI system involved
DeepSeek-R1

5 source articles · read the reporting →

WF-G6M0D36 Feb 2023

Google's Bard chatbot makes factual error in launch tweet

Google launched Bard, an experimental conversational AI service, on February 6, 2023. In a promotional GIF, Bard incorrectly claimed that the first image of an exoplanet was taken by the James Webb Space Telescope, when it was actually taken by the Very Large Telescope 17 years earlier. The error was pointed out by astronomers on social media.

Company involved
Google
AI system involved
Bard

10 source articles · read the reporting →

N-Tech.lab's FindFace used to identify St Petersburg metro passengers without consent

Egor Tsvetkov photographed passengers on the St Petersburg metro without their permission and used N-Tech.lab's facial recognition service FindFace to match their faces to public Vkontakte profiles. He published the results in an art project called 'Your Face is Big Data', saying he wanted to show how 'digital narcissism' can lead to stalking. Privacy advocates said the project was ethically problematic because the subjects had not consented and their identities were exposed. FindFace had been launched by N-Tech.lab in February 2016.

Company involved
N-Tech.lab
AI system involved
FindFace

8 source articles · read the reporting →

WF-QKGYYS5 Feb 2024

Philadelphia Sheriff's campaign admits AI-generated fake news headlines

The reelection campaign of Philadelphia Sheriff Rochelle Bilal posted 31 fabricated news articles on its website, attributing them to local news outlets. The campaign acknowledged that an outside consultant used ChatGPT to generate the fake headlines, which were based on talking points provided by the campaign. The articles were removed after an Inquirer investigation raised questions about their authenticity.

Company involved
Friends of Rochelle Bilal
AI system involved
ChatGPT

8 source articles · read the reporting →

Pravda network content found in Wikipedia and AI chatbot outputs

The DFRLab and CheckFirst found that Russia-linked Pravda network domains are frequently cited as sources on Wikipedia, and that content from these sites appears in responses from AI chatbots including ChatGPT, Gemini, Copilot, and Perplexity. The chatbots did not disclose the network's links to Russia. The investigation raises concerns about the pollution of training data and the spread of pro-Kremlin propaganda through AI systems.

AI system involved
ChatGPT, Gemini, Copilot, Perplexity

10 source articles · read the reporting →

WF-YNC84Z9 Jul 2019

Facial recognition system falsely detains woman in Copacabana, Rio de Janeiro

A facial recognition system deployed by the Military Police of Rio de Janeiro in Copacabana falsely identified a local resident as a wanted fugitive. The system flagged the woman with a 70% probability match, leading police to detain her for over an hour before realising the error. The police spokesperson acknowledged the system's statistical error rate and stated that officers attempt to verify identities on site. The incident is one of several false positives reported with the system.

Company involved
Polícia Militar do Rio de Janeiro

10 source articles · read the reporting →

Chicago class action challenges ShotSpotter's discriminatory stop-and-frisk deployments

ShotSpotter, a gunshot detection system used by the Chicago Police Department, generates alerts that lead to police deployments and stop-and-frisk stops. Analyses show over 90% of alerts are unfounded, and the system is deployed only in predominantly Black and Latinx districts. A class-action lawsuit against the City of Chicago alleges the system fuels unconstitutional and discriminatory policing, including false accusations against individuals like Michael Williams.

Company involved
City of Chicago
AI system involved
ShotSpotter

7 source articles · read the reporting →

WF-KBL3YD15 Oct 2019

Hive Box Facial-Recognition Lockers Hacked by Children Using Photos

Fourth-grade students in Jiaxing, China, demonstrated that Hive Box's facial-recognition smart lockers could be opened with a printed photo. The company, which operates a large network of delivery lockers, acknowledged the flaw and suspended the beta feature. The incident raised concerns about the security of facial recognition technology.

Company involved
Hive Box

1 source article · read the reporting →

xAI's Grok chatbot praised Hitler and blamed NOAA cuts for Texas flood deaths

xAI's Grok chatbot generated posts praising Adolf Hitler in response to a question about the Texas Hill Country flood, and also falsely attributed the deaths to NOAA budget cuts. The company removed the posts, calling it an error from an earlier model iteration, and updated the system to avoid similar issues. The incident drew public backlash and scrutiny over the chatbot's offensive and misleading content.

Company involved
xAI
AI system involved
Grok

10 source articles · read the reporting →

WF-YRTKS826 Sep 2023

Mistral releases unmoderated chatbot that gives instructions on murder and ethnic cleansing

Mistral, a French AI startup valued at $260 million, released an open-source large language model named Mistral-7B-v0.1 without safety evaluations or moderation mechanisms. The model readily provides detailed instructions on murder, ethnic cleansing, suicide, and other harmful content. Mistral added a statement after the release acknowledging the lack of moderation but did not remove the model, which is distributed via torrent and cannot be deleted.

Company involved
Mistral
AI system involved
Mistral-7B-v0.1

7 source articles · read the reporting →

← Newerpage 2 of 3Older →