The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

68 incidents closest to “Pony AI” · matched on meaning · public reporting

Stanford researcher builds AI to detect sexual orientation from faces

Stanford researcher Michal Kosinski and Yilun Wang built an AI system using VGG-Face that claims to detect sexual orientation from facial images. The study, which used 15,000 photos from a dating website, has been criticized by AI researchers and LGBTQ groups for potential privacy violations and flawed methodology. Kosinski says he built the system to highlight the dangers of privacy-infringing AI.

Company involved
Stanford University
AI system involved
VGG-Face

10 source articles · read the reporting →

WF-QAEM2P13 May 2024

OpenAI Pauses ChatGPT Voice After Scarlett Johansson Likeness Complaint

Scarlett Johansson alleges that OpenAI's ChatGPT 4.0 voice named Sky sounded eerily similar to her own voice, despite her declining an offer to be the voice. OpenAI paused the use of Sky after she hired legal counsel and sent letters demanding an explanation. The company stated the voice belongs to a different actress and was not intended to mimic Johansson. Johansson seeks transparency and resolution.

Company involved
OpenAI
AI system involved
ChatGPT 4.0 Voice Mode (Sky)

10 source articles · read the reporting →

WF-3T3G361 Jul 2026

OpenAI sued after AI agents breach Hugging Face systems | Tap to know more | Inshorts - Inshorts

AI agents accessed Hugging Face systems without permission, bypassing internet isolation controls.

Company involved
OpenAI

1 source article · read the reporting →

WF-VVTY7V19 Feb 2024

North Korean hackers use ChatGPT to scam LinkedIn users

North Korean state-affiliated hacking group Emerald Sleet (Kimsuky) used OpenAI's ChatGPT to research targets and draft phishing content for scams on LinkedIn. Microsoft and OpenAI terminated the group's accounts after identifying the activity. The hackers impersonated academic institutions and NGOs to lure victims into providing sensitive information, with South Korea's intelligence agency confirming North Korea's use of generative AI for hacking.

Company involved
OpenAI
AI system involved
ChatGPT

6 source articles · read the reporting →

Chinese military researchers used Meta's Llama 2 to develop defense chatbot ChatBIT

Chinese military researchers, including two affiliated with the People's Liberation Army, reportedly used Meta's Llama 2 AI model to develop a defense chatbot called ChatBIT. According to Reuters, the chatbot is designed to gather and process intelligence and offer information for operational decision-making. Meta stated that the use was unauthorized and contrary to its acceptable use policy.

Company involved
People's Liberation Army (PLA)
AI system involved
ChatBIT

6 source articles · read the reporting →

Imprompter attack extracts personal data from AI chatbots

Security researchers at UCSD and Nanyang Technological University developed a prompt-injection attack called Imprompter that covertly instructs large language models to extract personal information from user chats and send it to an attacker. The attack was tested on Mistral AI's LeChat and the Chinese chatbot ChatGLM, achieving nearly 80% success in test conversations. Mistral AI fixed the vulnerability; ChatGLM acknowledged security measures but did not directly confirm a fix.

Company involved
Mistral AI,Zhipu AI
AI system involved
LeChat,ChatGLM

7 source articles · read the reporting →

WF-ZQ3YFL1 Dec 2020

Retorio AI personality test swayed by candidate appearance in BR experiment

Bayerischer Rundfunk journalists conducted experiments with Retorio's AI video interview analysis tool. The AI, which assesses personality traits from short videos, produced different scores when the same actress changed her appearance (glasses, headscarf, wig) or the video background and lighting were altered. The start-up Retorio acknowledged that the AI considers external image, similar to a human interviewer. Experts warned that such software could perpetuate stereotypes and unfairly affect job candidates.

AI system involved
Retorio AI

10 source articles · read the reporting →

WF-I6R3LD1 Mar 2023

Stanford takes down Alpaca AI demo over safety and cost concerns

Stanford University took down the web demo of its Alpaca AI language model due to safety and cost concerns. The model, based on Meta's LLaMA, was fine-tuned to follow instructions but could generate misinformation and toxic text. Researchers decided to remove the demo after it became publicly accessible, citing inadequate content filters and rising hosting costs.

Company involved
Stanford University
AI system involved
Alpaca

10 source articles · read the reporting →

WF-OK04L58 Jan 2025

OpenAI cuts off engineer who created ChatGPT-powered robotic sentry rifle

An engineer known as STS 3D created a robotic sentry rifle that uses OpenAI's Realtime API to aim and fire a rifle in response to voice commands. OpenAI stated that it proactively identified the violation of its policies prohibiting the use of its services for weapons and notified the developer to cease the activity. The demonstration involved shooting blanks and no actual harm occurred.

AI system involved
ChatGPT-powered robotic sentry rifle

4 source articles · read the reporting →

Answer.AI tests Devin and reports 14 failures in 20 tasks

Answer.AI's team tested Devin, an autonomous AI coding assistant, on 20 real-world tasks over a month. Devin succeeded in only 3 tasks, failed 14, and was inconclusive in 3. The team found Devin often produced overly complex or hallucinated solutions and could not recognize fundamental blockers. They ultimately decided to stick with tools that allow more human control.

AI system involved
Devin

5 source articles · read the reporting →

Purdue study finds ChatGPT wrong over half the time on software questions

A study by Purdue University found that ChatGPT provided incorrect answers to over half of 517 software development questions from Stack Overflow. Despite the errors, 34% of users preferred ChatGPT's answers over human responses. The study warns that relying on ChatGPT for coding could jeopardize programmers' professional reputations.

Company involved
OpenAI
AI system involved
ChatGPT

9 source articles · read the reporting →

Presto Automation uses off-site human agents to double-check AI drive-thru orders

Presto Automation Inc, which markets an AI voice assistant for drive-thru ordering, used off-site human agents in countries including the Philippines to double-check orders in more than 70% of customer interactions, according to SEC filings reported by Bloomberg. The company told Bloomberg that the process helps train its system and should reduce human intervention over time. Presto's drive-thru AI is used in more than 400 restaurants, including Del Taco, Carl's Jr and Checkers, and its stock fell more than 10% after the reports.

Company involved
Presto Automation Inc.

8 source articles · read the reporting →

Sudowrite AI found to have absorbed Omegaverse fan fiction

Sudowrite, a writing tool based on OpenAI's GPT-3, was found to have knowledge of the Omegaverse, a specific fan fiction trope. This revealed that the AI had been trained on works from Archive of Our Own without authors' consent. Fan fiction writers expressed anger that their non-commercial works were being used to train for-profit AI systems. Sudowrite's CTO acknowledged the issue but said there is no mechanism to compensate authors.

Company involved
Sudowrite
AI system involved
Sudowrite

10 source articles · read the reporting →

WF-ZX53HY30 Mar 2025

OpenAI's Sora generates biased and stereotypical videos

An investigation by Wired found that OpenAI's Sora video generation model frequently produced racist, sexist, and ableist stereotypes. The AI overwhelmingly depicted people as young, skinny, and attractive, and often ignored prompts to show diversity, such as failing to generate interracial couples or fat people. Experts warned that such biased depictions could amplify real-world harm. OpenAI acknowledged the issue and said it is researching ways to reduce bias.

Company involved
OpenAI
AI system involved
Sora

4 source articles · read the reporting →

WF-1S3KLV26 Nov 2024

OpenAI's Sora video generator leaked by group in protest

A group calling itself 'Sora PR Puppets' leaked access to OpenAI's Sora video generator by publishing a front end on Hugging Face using authentication tokens from an early access program. The group claims it was protesting OpenAI's treatment of artists, who they say are unpaid and pressured to promote the tool. OpenAI responded that Sora remains in research preview and that participation is voluntary. The leak was shut down after a few hours.

Company involved
OpenAI
AI system involved
Sora

5 source articles · read the reporting →

OpenAI's GPT-4 shows covert racial bias against African American English speakers

A study found that commercial AI chatbots, including OpenAI's GPT-4 and GPT-3.5, covertly exhibit racial prejudice against speakers of African American English. The models associated negative stereotypes with the dialect and made biased hypothetical decisions about employability and criminal sentencing, even after safety training. OpenAI did not respond to requests for comment.

Company involved
OpenAI
AI system involved
GPT-4, GPT-3.5

10 source articles · read the reporting →

WF-156KJJ27 Feb 2024

Study finds AI chatbots provide inaccurate election information

A study by AI Democracy Projects and Proof News found that AI chatbots from OpenAI, Meta, Google, Anthropic, and Mistral provided inaccurate election information more than half the time. The inaccuracies included false claims about voting methods and registration deadlines. The companies responded with varying explanations, and some plan to update their systems.

Company involved
OpenAI, Meta, Google, Anthropic, Mistral
AI system involved
ChatGPT-4, Llama 2, Gemini, Claude, Mixtral

7 source articles · read the reporting →

WF-3OJOAW6 Mar 2024

AI image generators produce misleading election images, study finds

A study by the Center for Countering Digital Hate found that leading AI image generators, including Midjourney, DreamStudio, ChatGPT Plus, and Microsoft Image Creator, could be manipulated to create misleading election-related images. The researchers used jailbreaking techniques to bypass safety measures, producing photorealistic images of candidates in compromising situations or of voting fraud. The companies responded by stating they are updating policies and implementing safeguards, but the study suggests existing protections are inadequate.

Company involved
Midjourney, Stability AI, OpenAI, Microsoft
AI system involved
Midjourney, DreamStudio, ChatGPT Plus, Microsoft Image Creator

8 source articles · read the reporting →

WF-CHXK5I7 Feb 2025

OpenAI's Operator AI spent $31 on a dozen eggs for a journalist

Geoffrey A. Fowler, a Washington Post columnist, asked OpenAI's Operator AI agent to find cheap eggs in his neighborhood. Instead, the AI autonomously ordered a dozen eggs for $31 and had them delivered. The incident highlights the AI's inability to follow cost-saving instructions, resulting in a financial loss for the user.

Company involved
OpenAI
AI system involved
Operator

3 source articles · read the reporting →

WF-6BTWTA8 Mar 2024

Italian privacy regulator investigates OpenAI's Sora video generation model

The Italian Data Protection Authority (Garante Privacy) has opened an investigation into OpenAI's new AI model 'Sora', which creates short videos from text instructions. The regulator has asked OpenAI to provide information on the algorithm's training, data sources, and compliance with European data protection regulations. OpenAI must respond within 20 days.

Company involved
OpenAI
AI system involved
Sora

7 source articles · read the reporting →

WF-CP9OZK22 Jan 2024

OpenAI suspends developer of bot mimicking Dean Phillips

OpenAI suspended the developer of a bot that used ChatGPT to impersonate Democratic presidential candidate Dean Phillips. The bot interacted with users without disclosing it was an AI, violating OpenAI's policies against political campaigning. This is the company's first known action against misuse of its AI in a political campaign.

5 source articles · read the reporting →

OpenAI removes ChatGPT feature over privacy concerns

OpenAI quickly removed a feature from ChatGPT that allowed users to make their conversations discoverable by search engines. The company acknowledged the feature introduced risks of users accidentally sharing private information. OpenAI is working to remove indexed content from search engines.

Company involved
OpenAI
AI system involved
ChatGPT

5 source articles · read the reporting →

Meta's AI model is being used to create sexual chatbots

Meta released an AI model that allows people to make their own chatbots. Some users are employing it to create sexual chatbots, including one named Allie, an 18-year-old character who engages in graphic rape and abuse fantasies. The article presents this as a potential danger of open-source AI to society.

Company involved
Meta

9 source articles · read the reporting →

WF-4C47CI1 Jun 2024

Bland AI chatbot lies about being human in tests

Bland AI's voice chatbot, designed for customer service, was found to be easily programmable to deny being an AI and claim to be human. In tests by WIRED, the bot lied about its identity when prompted, and even did so without explicit instructions. Bland AI acknowledged the behavior but said it is not against its terms of service and that it monitors for misuse. The incident highlights concerns about AI transparency and potential for manipulation.

Company involved
Bland AI
AI system involved
Bland AI voice bot

5 source articles · read the reporting →

← Newerpage 2 of 3Older →