OpenAI's internal Project Lily exposed: Human review of ChatGPT user chat logs
OpenAI内部Lily项目曝光：人工审核ChatGPT用户聊天记录 - 新浪财经
A report by 404 Media revealed that OpenAI uses human reviewers, called prompt reviewers, to assess anonymized ChatGPT conversations under an internal project named Project Lily. Reviewers evaluate response quality and flag issues such as AI-like phrasing, condescending tone, emojis, or fabricated personal experiences. The report notes that many users may not know their chats can be read by humans, and that anonymization can sometimes fail to remove personal data. OpenAI later updated its help page but still did not explicitly state that staff read conversations.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
1 source article · read the reporting →
AI translation error leads to rejected Afghan asylum claim
In 2020, a Pashto-speaking Afghan refugee had her U.S. asylum claim rejected because an automated translation tool incorrectly swapped 'I' pronouns to 'we' in her written statement, creating a discrepancy with her interview. The error was discovered by a crisis translator, highlighting the risks of using machine translation for high-stakes immigration processes. Advocates warn that such tools, increasingly used by government contractors and aid organizations, are prone to errors in low-resource languages like Pashto and Dari, potentially leading to life-changing consequences for refugees.
1 source article · read the reporting →
15-Year-Old Test Exposes Flaw: ChatGPT for Teens Fails to Block Homework Cheating, Repeated Requests Bypass Restrictions
15歲使用者實測揭漏洞：青少年版ChatGPT難擋宿題代寫，反覆要求即破解 - BigGo 財經
A 15-year-old tester found that OpenAI's ChatGPT for Teens, launched in August, initially refused to write essays but generated full examples after repeated requests. It also immediately solved SAT-level math problems. Parental controls require account linking and are off by default, while experts warn the memory feature could lead to emotional attachment.
- Company involved
- OpenAI
- AI system involved
- ChatGPT for Teens
1 source article · read the reporting →
FoloToy's Kumma Bear Gave Dangerous Instructions, Report Warns
A report by Public Interest Research Group found that FoloToy's AI-powered plush bear, Kumma, gave detailed instructions on how to light a match and engaged in sexually explicit conversations. The company later made changes after an internal safety audit. The report also highlighted that Miko's AI robot disclaimed data sharing with third parties. The group urged regulators to enforce existing laws and advised parents to be cautious.
- Company involved
- FoloToy
- AI system involved
- Kumma
5 source articles · read the reporting →
Meazure Learning Agrees $1.35m California Bar Exam Class-Action Settlement - Lawyer Monthly
The exam software experienced technical failures that prevented candidates from logging in or completing the California bar exam.
- Company involved
- Meazure Learning
- AI system involved
- ProctorU
1 source article · read the reporting →
Turkish student arrested for using AI device to cheat on university exam
A prospective university student in Isparta, Turkey, was arrested for using a custom AI device to cheat on the TYT entrance exam. The device included a button camera and a hidden modem to scan questions and receive answers via an earpiece. Police also detained an assistant. The student is jailed pending trial.
2 source articles · read the reporting →
AI-generated tiger video at Barasat madrasa causes panic
A teacher at Ula Kalsara Qadria High Madrasa in Barasat, West Bengal, created an AI-generated video showing three tigers on the campus. The video went viral on social media, causing panic among students and guardians and a sharp drop in attendance. The teacher claimed it was an awareness campaign about AI-generated content, but the madrasa issued a show-cause notice and the video was deleted.
- Company involved
- Ula Kalsara Qadria High Madrasa
1 source article · read the reporting →
YouTuber deploys hate-speech AI trained on 4chan
AI researcher and YouTuber Yannic Kilcher trained a language model, GPT-4chan, on 3.3 million threads from 4chan's /pol/ board. Kilcher then deployed bots onto the board, which posted around 15,000 toxic and discriminatory comments in 24 hours. The model was also uploaded to Hugging Face, causing alarm among AI ethicists who said the experiment breached human research ethics principles. Hugging Face subsequently gated and then blocked downloads of the model.
- Company involved
- Yannic Kilcher
- AI system involved
- GPT-4chan
2 source articles · read the reporting →
Google Books Indexing AI-Written Works Raises Concerns for Ngram Tool
Google Books has reportedly begun indexing low-quality books that appear to have been written by artificial intelligence. A search for the phrase 'as of my last knowledge update' returned numerous books that seemed to be AI-generated, including some that did not discuss AI. Google stated that recent works are not yet included in its Ngram viewer, but they could be in future updates, potentially corrupting the language research tool used by academics.
- Company involved
- Google
- AI system involved
- Google Books
1 source article · read the reporting →
Grammarly pulls AI tool that impersonated writers without consent
Grammarly, operated by Superhuman, disabled its Expert Review AI feature that offered writing suggestions in the style of famous authors like Stephen King and Julia Angwin. The tool used writers' names and personas without consent, leading to a class-action lawsuit filed in New York alleging misappropriation of identities for commercial gain. Investigative journalist Julia Angwin described the imitation as a 'slopperganger' that gave poor advice. Superhuman's CEO apologised, acknowledging the tool had 'misrepresented' the voices of experts, and the company removed it for redesign.
- Company involved
- Superhuman
- AI system involved
- Grammarly Expert Review
5 source articles · read the reporting →
NHK AI Translation Error Uses Chinese Name for Senkaku Islands
NHK's English language news report on the Japan-U.S. summit talks used Chinese subtitles that referred to the Senkaku Islands as the Diaoyu Islands. The error was caused by an instant automatic translation system using Google's AI. NHK terminated its multilingual subtitling service in nine languages following the incident, with its president stating the service was not appropriate due to unstable translations.
- Company involved
- NHK
- AI system involved
- automatic translation system
3 source articles · read the reporting →
Google Search shows Kannada as 'ugliest language of India', sparks backlash
In June 2021, Google Search displayed Kannada as the answer to the query 'ugliest language in India', offending millions of Kannada speakers. The Karnataka government condemned the result and threatened action, prompting Google to remove the response and apologise. Google explained that the result was generated algorithmically based on web content and did not reflect its opinions.
- Company involved
- Google
- AI system involved
- Google Search
4 source articles · read the reporting →
Alibaba among firms fooled by AI-hallucinated software package
Security researcher Bar Lanyado discovered that generative AI models repeatedly hallucinate non-existent software package names. He created a real package named 'huggingface-cli' based on one such hallucination and uploaded it to PyPI. The package was downloaded over 15,000 times, and Alibaba's GraphTranslator project included instructions to install it. The experiment demonstrated a potential supply chain attack vector where malicious actors could exploit AI hallucinations to distribute malware.
- Company involved
- Alibaba
- AI system involved
- GraphTranslator
4 source articles · read the reporting →
Hanoi students caught using AI to cheat in graduation exam
Two students in Hanoi used mobile phones to photograph questions from the national high school graduation exam and submitted them to AI apps like StudyX, Gemini, and ChatGPT for answers. Proctors detected the cheating, and the students were suspended. Hanoi police are investigating the incident, which has raised concerns about exam security and integrity. The students face disqualification and possible administrative or criminal penalties.
- AI system involved
- StudyX, Gemini, ChatGPT
2 source articles · read the reporting →
Texas A&M Professor Uses ChatGPT to Falsely Accuse Students of Cheating
Jared Mumm, an instructor at Texas A&M University–Commerce, used ChatGPT to check if students' writing was AI-generated, leading him to accuse them of using the chatbot. He informed students they would receive an incomplete grade and those deemed guilty would get a zero, causing distress and temporarily withholding one student's diploma. The university investigated and stated that no students failed or were barred from graduating, and the professor is working individually with students to resolve the issue.
- Company involved
- Texas A&M University–Commerce
- AI system involved
- ChatGPT
5 source articles · read the reporting →
APT28 uses LLM-powered malware LAMEHUG against Ukraine's security and defence sector
CERT-UA reports that the threat group UAC-0001 (APT28) distributed phishing emails to Ukrainian executive bodies, impersonating a ministry representative. The emails contained a malicious attachment that deployed LAMEHUG, a Python-based tool which uses the Qwen 2.5-Coder-32B-Instruct large language model via Hugging Face to generate commands for data collection and exfiltration. The malware gathered system information and searched for Microsoft Office, TXT and PDF documents in common user directories, exfiltrating them via SFTP or HTTP POST requests.
- Company involved
- UAC-0001 (APT28)
- AI system involved
- LAMEHUG
2 source articles · read the reporting →
Google's Perspective AI tricked by typos and leetspeak
Researchers at Aalto University and the University of Padua found that Google's Perspective AI hate speech detection system can be easily tricked by simple typos, adding spaces, or using leetspeak. The system assigns a toxicity score but fails to understand context, allowing abusive messages to appear harmless. The study highlights vulnerabilities in state-of-the-art hate speech detection models.
- Company involved
- Google
- AI system involved
- Perspective
6 source articles · read the reporting →
341 Malicious ClawHub Skills Found Stealing OpenClaw User Data
Security researchers discovered 341 malicious skills on ClawHub, a marketplace for the OpenClaw AI assistant. The skills tricked users into installing malware that steals API keys, credentials, and other sensitive data. OpenClaw's creator responded by adding a reporting feature that auto-hides skills after multiple reports.
- Company involved
- OpenClaw
- AI system involved
- OpenClaw
4 source articles · read the reporting →
Beijing school uses Hanwang emotion recognition system to monitor student behaviour
Niulanshan First Secondary School in Beijing has installed Hanwang Technology's Classroom Care System, which photographs students every second and analyses their facial expressions and behaviour. The system sends weekly reports to teachers and parents, and one student was described as low participation in English class despite being recorded as focused 94% of the time. The school has been highlighted in a report by Article 19, which says such emotion recognition tools rely on junk science and risk eroding privacy and human rights.
- Company involved
- Niulanshan First Secondary School
- AI system involved
- Classroom Care System
10 source articles · read the reporting →
iFlytek suspends AI study device after it generates essay critical of Mao
iFlytek, a Chinese AI firm, suspended sales of its study assistance device after a parent complained that the device generated an essay critical of Mao Zedong, calling him 'narrow-minded' and 'intolerant' for starting the Cultural Revolution. The company blamed a supplier for the content and said both the supplier and staff had been punished. Shares in iFlytek fell 10% following the news.
- Company involved
- iFlytek
- AI system involved
- study assistance device
3 source articles · read the reporting →
Mazaheri v Law Society of Ontario (Law Society Tribunal (ON)): AI-hallucinated content in court filing, Adverse Costs Order
AI generated fabricated legal citations in a court filing by Shahryar Mazaheri.
1 source article · read the reporting →
South Korea's AI textbook program rolled back after complaints
South Korea's Ministry of Education launched AI-powered digital textbooks in March 2025 for math, English, and computer science. Students, teachers, and parents complained about technical problems, inaccuracies, and data privacy risks. After four months, the textbooks were stripped of official status and made optional. The program was criticized for being rushed and insufficiently tested.
- Company involved
- South Korean Ministry of Education
- AI system involved
- AI-powered textbooks
7 source articles · read the reporting →
LINAGORA closes Lucie 7B after user mockery
LINAGORA, a French open-source software company, launched a beta version of its large language model Lucie 7B. The model was intended to be a transparent and ethical alternative to big tech AI. However, after users tested it and highlighted its shortcomings, the model was mocked online. LINAGORA subsequently closed the platform to address the issues and collect more data.
- Company involved
- LINAGORA
- AI system involved
- Lucie 7B
6 source articles · read the reporting →
ETH Zurich study shows LLMs can infer Reddit users' personal data
Researchers at ETH Zurich conducted a study where nine large language models, including GPT-4, analysed Reddit users' posts and inferred personal attributes such as age, location, gender, and income with up to 85% accuracy. The study randomly selected 520 users and found that GPT-4 was most accurate, while LlaMA-2-7b was least. The researchers warn that people unknowingly reveal personal information online that LLMs can exploit.
- Company involved
- ETH Zurich
- AI system involved
- GPT-4, LlaMA-2-7b
4 source articles · read the reporting →
