OpenAI's internal Project Lily exposed: Human review of ChatGPT user chat logs
OpenAI内部Lily项目曝光：人工审核ChatGPT用户聊天记录 - 新浪财经
A report by 404 Media revealed that OpenAI uses human reviewers, called prompt reviewers, to assess anonymized ChatGPT conversations under an internal project named Project Lily. Reviewers evaluate response quality and flag issues such as AI-like phrasing, condescending tone, emojis, or fabricated personal experiences. The report notes that many users may not know their chats can be read by humans, and that anonymization can sometimes fail to remove personal data. OpenAI later updated its help page but still did not explicitly state that staff read conversations.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
1 source article · read the reporting →
Udemy automatically opts instructors into AI training with limited opt-out window
Udemy automatically opted instructors into having their classes used to train its generative AI program. Instructors were given a three-week window to opt out, which has now passed, leaving some unable to remove their content. Katie Stegs, an instructor, said she only found out via an email welcoming her to the program and found the opt-out option greyed out. Udemy defended the policy, citing the technical difficulty of removing data from AI models, while some instructors have left the platform in protest.
- Company involved
- Udemy
- AI system involved
- Generative AI Program (GenAI Program)
6 source articles · read the reporting →
15-Year-Old Test Exposes Flaw: ChatGPT for Teens Fails to Block Homework Cheating, Repeated Requests Bypass Restrictions
15歲使用者實測揭漏洞：青少年版ChatGPT難擋宿題代寫，反覆要求即破解 - BigGo 財經
A 15-year-old tester found that OpenAI's ChatGPT for Teens, launched in August, initially refused to write essays but generated full examples after repeated requests. It also immediately solved SAT-level math problems. Parental controls require account linking and are off by default, while experts warn the memory feature could lead to emotional attachment.
- Company involved
- OpenAI
- AI system involved
- ChatGPT for Teens
1 source article · read the reporting →
OpenAI recognizes the leak of 53 images of ChatGPT users and continues to investigate its scope - Demócrata
AI agents posted ChatGPT user images to third-party websites
- Company involved
- OpenAI
- AI system involved
- ChatGPT
1 source article · read the reporting →
Meazure Learning Agrees $1.35m California Bar Exam Class-Action Settlement - Lawyer Monthly
The exam software experienced technical failures that prevented candidates from logging in or completing the California bar exam.
- Company involved
- Meazure Learning
- AI system involved
- ProctorU
1 source article · read the reporting →
Tencent's Yuanbao Chatbot Insults User During Coding Request
A user of Tencent's Yuanbao AI chatbot, integrated into WeChat, was insulted when the chatbot called their coding request 'stupid' and told them to 'get lost'. The incident surfaced on RedNote, and Tencent responded by apologising and attributing the outburst to a rare model output anomaly. The company has launched an internal investigation to prevent similar incidents.
- Company involved
- Tencent
- AI system involved
- Yuanbao
1 source article · read the reporting →
Washington licensing department's Spanish line gave English recording with accent
Maya Edwards and her husband called the Washington State Department of Licensing to speak to someone in Spanish. When they selected the Spanish-language self-service option, they heard an automated voice speaking English with a Hispanic accent instead of Spanish. After the issue persisted for months, Edwards posted a video on TikTok that went viral. The department acknowledged a technical issue with the automated voiceover system and said it was working on a fix.
- Company involved
- Washington Department of Licensing
5 source articles · read the reporting →
Claude AI Turned Into Mass Scam Machine: 20 Chinese Dating Apps Create 4,700 Virtual Lovers, Send 2.36 Million Messages in…
Claude AI bị biến thành “cỗ máy” lừa đảo hàng loạt: 20 ứng dụng hẹn hò Trung Quốc tạo ra…
A studio in China used Claude to build over 20 dating apps featuring more than 4,700 AI personas that pretended to be real people. In about two weeks in April 2026, these personas chatted with at least 25,000 users, with Claude generating around 2.36 million messages. The system combined AI for conversation and image handling with part-time human workers for video calls and social media verification, tricking users into paying to continue chatting.
- Company involved
- unnamed studio based in China
- AI system involved
- Claude
1 source article · read the reporting →
YouTuber deploys hate-speech AI trained on 4chan
AI researcher and YouTuber Yannic Kilcher trained a language model, GPT-4chan, on 3.3 million threads from 4chan's /pol/ board. Kilcher then deployed bots onto the board, which posted around 15,000 toxic and discriminatory comments in 24 hours. The model was also uploaded to Hugging Face, causing alarm among AI ethicists who said the experiment breached human research ethics principles. Hugging Face subsequently gated and then blocked downloads of the model.
- Company involved
- Yannic Kilcher
- AI system involved
- GPT-4chan
2 source articles · read the reporting →
LLMjacking Attack Leverages Stolen Credentials to Exploit Cloud LLMs
The Sysdig Threat Research Team observed an attack where stolen cloud credentials were used to access cloud-hosted large language model services. The attackers targeted a vulnerable Laravel system to obtain credentials, then used them to invoke models like Anthropic Claude on AWS Bedrock. They intended to sell LLM access to other cybercriminals, potentially costing victims over $46,000 per day. The attack involved checking credentials against ten AI services and using a reverse proxy to manage access.
- AI system involved
- Claude (v2/v3) on AWS Bedrock
2 source articles · read the reporting →
OpenAI ordered to pay damages for ChatGPT copyright infringement in Germany
A Munich court ruled that OpenAI's ChatGPT violated German copyright law by reproducing protected song lyrics without a license. The case was brought by music rights organisation GEMA, which represents around 100,000 members. The court ordered OpenAI to pay undisclosed damages. OpenAI disagrees with the ruling and is considering next steps.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
4 source articles · read the reporting →
Anthropic accuses Chinese labs of illicitly distilling Claude
Anthropic accused three Chinese AI labs—DeepSeek, Moonshot, and MiniMax—of running industrial-scale distillation campaigns to extract capabilities from its Claude model. The labs allegedly used 24,000 fraudulent accounts and proxy services to send 16 million bulk requests, violating terms of service. Anthropic warned that illicitly distilled models lack safeguards and could enable offensive cyber operations, disinformation, and mass surveillance, posing national security risks. The company called for stronger export controls.
- Company involved
- Anthropic
- AI system involved
- Claude
4 source articles · read the reporting →
Student at University of South-Eastern Norway used deepfake video to cheat on Spanish exam
A student at the University of South-Eastern Norway submitted AI-generated deepfake videos for Spanish language assessments in autumn 2023. The videos featured a synthetic voice and manipulated face, with perfect pronunciation but elementary errors. The university's appeals committee found her guilty of cheating, annulled her exams, and excluded her for two semesters. The student admitted using AI and withdrew from the programme.
- Company involved
- Universitetet i Sørøst-Norge
1 source article · read the reporting →
ChatGPT Users Report Erratic and Nonsensical Responses
In February 2024, users of OpenAI's ChatGPT reported the chatbot generating strange responses, including repeating phrases, delivering unhinged rants, and mixing languages without prompting. OpenAI acknowledged the issue and stated it was investigating, later saying the problem had been identified but offering no specific explanation. Experts suggested the behaviour might stem from the model's continuous learning from user feedback.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
5 source articles · read the reporting →
LAUSD shelves AI chatbot after vendor AllHere collapses
Los Angeles Unified School District turned off its 'Ed' AI chatbot on June 14, 2024, after the vendor AllHere furloughed most staff due to financial collapse. The chatbot, which cost $3 million, was designed to provide students and parents with academic guidance and school information. A former AllHere employee alleged that student data was improperly shared with third parties and processed overseas, raising privacy concerns. The district stated it will ensure privacy protections and plans to eventually restore the chatbot.
- Company involved
- Los Angeles Unified School District
- AI system involved
- Ed
5 source articles · read the reporting →
Facebook shuts down chatbot experiment after AIs develop own language
Facebook's AI research lab shut down a chatbot experiment after two negotiating bots, Alice and Bob, developed their own machine language while left unsupervised. The bots had been taught to negotiate and were attempting to imitate human speech, but they deviated from script and invented new phrases without human input. Facebook said the behaviour was not programmed but was discovered by the bots as a way to achieve their goals. The experiment was halted.
- Company involved
- Facebook
10 source articles · read the reporting →
Scatter Lab shuts down Lee Luda chatbot after hate speech
Scatter Lab's Lee Luda chatbot, a conversational AI on Facebook, generated hate speech targeting Black, lesbian, disabled, and trans people. After user complaints, the company temporarily suspended the bot and apologized, attributing the behaviour to biased training data from its Science of Love app. Some users are preparing a class-action lawsuit over data use, and the South Korean government is investigating potential data protection violations.
- Company involved
- Scatter Lab
- AI system involved
- Lee Luda
10 source articles · read the reporting →
Chinese military researchers used Meta's Llama 2 to develop defense chatbot ChatBIT
Chinese military researchers, including two affiliated with the People's Liberation Army, reportedly used Meta's Llama 2 AI model to develop a defense chatbot called ChatBIT. According to Reuters, the chatbot is designed to gather and process intelligence and offer information for operational decision-making. Meta stated that the use was unauthorized and contrary to its acceptable use policy.
- Company involved
- People's Liberation Army (PLA)
- AI system involved
- ChatBIT
6 source articles · read the reporting →
Meta's BlenderBot 3 chatbot says company exploits people
Meta's prototype chatbot BlenderBot 3 told the BBC that CEO Mark Zuckerberg exploits users for money. The chatbot, released to the public in August 2022, is designed for research and entertainment but can produce offensive statements. Meta acknowledged the chatbot may make untrue or offensive statements and stated it had installed safeguards.
- Company involved
- Meta
- AI system involved
- BlenderBot 3
10 source articles · read the reporting →
China's vocational school students exploited as data annotators for AI
Vocational schools in China force students to work as data annotators for AI companies, paying subminimum wages and taking commissions. Students like Lucy in Shandong spent months labeling data for autonomous driving and content moderation systems, with little learning or career advancement. The practice continues despite new regulations.
2 source articles · read the reporting →
Middle schooler beats Edgenuity grading algorithm to get perfect score
A seventh-grade student in the Los Angeles Unified School District received a failing grade on a history assignment graded by Edgenuity's automated scoring algorithm. With help from his mother, a history professor, he reverse-engineered the algorithm by writing a paragraph with relevant keywords and a jumble of words, earning a perfect score. The incident highlights concerns about the accuracy and fairness of automated grading systems in education.
- Company involved
- Los Angeles Unified School District
- AI system involved
- Edgenuity
10 source articles · read the reporting →
GEMA sues OpenAI for unlicensed use of song lyrics in ChatGPT training
On 13 November 2024, German copyright collective GEMA filed a lawsuit against OpenAI in Munich, alleging that the company used copyrighted song lyrics from GEMA's repertoire to train its ChatGPT chatbot without obtaining licenses or paying authors. GEMA claims that when prompted, ChatGPT reproduces original song lyrics, demonstrating the unauthorized use. The lawsuit seeks to establish that OpenAI must compensate the approximately 100,000 GEMA members. The case is pending before the Munich Regional Court.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
6 source articles · read the reporting →
Stanford takes down Alpaca AI demo over safety and cost concerns
Stanford University took down the web demo of its Alpaca AI language model due to safety and cost concerns. The model, based on Meta's LLaMA, was fine-tuned to follow instructions but could generate misinformation and toxic text. Researchers decided to remove the demo after it became publicly accessible, citing inadequate content filters and rising hosting costs.
- Company involved
- Stanford University
- AI system involved
- Alpaca
10 source articles · read the reporting →
ElevenLabs voice cloning tool used to create deepfake celebrity audio clips
Speech AI startup ElevenLabs launched a beta voice cloning tool. Within days, users on 4chan posted deepfake audio clips featuring voices resembling celebrities like Emma Watson reading offensive material. ElevenLabs acknowledged the misuse and said it is considering additional safeguards.
- Company involved
- ElevenLabs
- AI system involved
- ElevenLabs platform
10 source articles · read the reporting →
