OpenAI's internal Project Lily exposed: Human review of ChatGPT user chat logs
OpenAI内部Lily项目曝光：人工审核ChatGPT用户聊天记录 - 新浪财经
A report by 404 Media revealed that OpenAI uses human reviewers, called prompt reviewers, to assess anonymized ChatGPT conversations under an internal project named Project Lily. Reviewers evaluate response quality and flag issues such as AI-like phrasing, condescending tone, emojis, or fabricated personal experiences. The report notes that many users may not know their chats can be read by humans, and that anonymization can sometimes fail to remove personal data. OpenAI later updated its help page but still did not explicitly state that staff read conversations.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
1 source article · read the reporting →
Fired Teacher Says AI Tainted Review And Arbitration - law360.com
The AI-generated performance evaluation contributed to the teacher's termination.
- Company involved
- Concord Public Schools
1 source article · read the reporting →
AI rejected a job candidate — and no one explained why. A look at how it works
ИИ отказал кандидату в работе — и никто не объяснил почему. Разбираем, как это устроено - Sostav.ru
In 2026, AI-driven recruitment systems often automatically reject candidates without human review. Algorithms scan resumes, rank applicants, and send template rejections to those who don't score high enough, while recruiters typically only see the top-ranked profiles. The criteria used by these systems are often opaque, and many managers cannot explain the logic behind the AI's decisions.
1 source article · read the reporting →
Amazon Limits Review Access After Mistaking Customers for AI, Bot Fallout Widens
Amazonが顧客をAIと誤認し、レビューへのアクセス制限 bot被害が拡大 - 日経クロストレンド
Since late 2025, Amazon has restricted some users from seeing more than a few product reviews after misidentifying their visits as AI crawler traffic. Affected customers must email the company and wait several business days for a response. Users are frustrated that Amazon has offered no explanation.
- Company involved
- Amazon
1 source article · read the reporting →
Resume prompt injection tricks AI hiring - moneywise.com
AI screening system determined which job applicants to advance to the next stage of recruitment.
1 source article · read the reporting →
SafeRent Settles Class Action Over Tenant Screening for Voucher Holders
Mary Louis and Monica Douglas filed a class action lawsuit in 2022 alleging that SafeRent Solutions' SafeRent Score product discriminated against rental applicants in Massachusetts who held public housing vouchers, violating fair housing and consumer protection laws. SafeRent denied wrongdoing. The court approved a settlement in November 2024, and payments were distributed to eligible class members in 2025 and 2026.
- Company involved
- SafeRent Solutions, LLC
- AI system involved
- SafeRent Score
5 source articles · read the reporting →
TRT-RS's Galileu AI Detects Prompt Injection Attempt in Legal Petition
The Galileu AI system, developed by the Tribunal Regional do Trabalho da 4ª Região (TRT-RS) and nationalised by the Conselho Superior da Justiça do Trabalho (CSJT), detected a prompt injection attempt in a petition filed at the 3rd Labour Court of Parauapebas, Pará. The system alerted the magistrate, who reviewed the content and made a decision based on human verification, in line with judicial AI supervision requirements. The court reported that the system prevented the malicious content from being processed and highlighted the importance of institutional AI tools with security measures.
- Company involved
- Tribunal Regional do Trabalho da 4ª Região
- AI system involved
- Galileu
1 source article · read the reporting →
Attorney Sanctioned $2,500 for AI-Hallucinated Citations in Brief
Heather Hersh, an appellate attorney, used generative AI to draft a reply brief in a Fair Credit Reporting Act case. The brief contained multiple fabricated legal quotations and inaccurate assertions. The Fifth Circuit Court of Appeals found that Hersh failed to verify the AI-generated content and was not candid about her use of AI, imposing a $2,500 monetary sanction under FRAP 46(c) and inherent authority. The court emphasised that attorneys must verify AI outputs and remain honest with the court.
- AI system involved
- Casetext / vLex
4 source articles · read the reporting →
Australian Research Council faces allegations of ChatGPT use in peer review
The Australian Research Council is facing allegations that some peer reviewers used ChatGPT to write assessor reports for Discovery Project grant proposals. Researchers reported generic wording and even the phrase 'Regenerate response' in feedback, suggesting AI generation. One researcher's complaint led to the removal of the report, and the education minister called the use unacceptable, instructing the ARC to prevent it. The ARC stated that peer reviewers should not use AI and that confidentiality policies apply.
- Company involved
- Australian Research Council
- AI system involved
- ChatGPT
2 source articles · read the reporting →
Turnitin's AI Detector Falsely Flags Student's Essay as AI-Generated
A Washington Post test of Turnitin's new AI writing detector found it erroneously flagged 8% of a high school student's original essay as likely generated by ChatGPT. The student, Lucy Goetz, had not used AI, raising concerns about false accusations of cheating. Turnitin acknowledged the issue and added a caution flag, but educators worry about the potential for baseless academic-integrity investigations. The incident highlights the challenges of detecting AI-generated text and the need for careful human review.
- Company involved
- Turnitin
- AI system involved
- Turnitin AI writing detector
1 source article · read the reporting →
Royal Free London publishes audit into Streams app data processing
The Royal Free London NHS Foundation Trust published an audit into its use of the Streams app, following an investigation by the Information Commissioner's Office (ICO) in July 2017. The Streams app alerts clinicians to patients at risk of acute kidney injury. The audit, conducted by Linklaters, concluded that the trust's use of Streams was lawful and complied with data protection laws, although areas for improvement were identified. The ICO later recognised that the trust had completed all required actions.
- Company involved
- Royal Free London NHS Foundation Trust
- AI system involved
- Streams
10 source articles · read the reporting →
Woody Nora v. M & A Transport, Inc., et al. (E.D. Louisiana): AI-hallucinated content in court filing, Monetary Sanction; 1…
The AI tool generated fictitious case law citations that were included in a court filing, affecting the plaintiff and the court.
- AI system involved
- Westlaw Precision
1 source article · read the reporting →
Researchers hide prompts to manipulate AI peer review
Nikkei found hidden prompts in 17 preprints on arXiv that instructed AI tools to give the papers positive reviews and ignore negatives. The manuscripts were linked to 14 academic institutions, including Waseda University, KAIST, Peking University, the National University of Singapore, the University of Washington and Columbia University. At least one paper was set to be withdrawn, while some researchers defended the prompts as a check on reviewers who improperly use AI.
- Company involved
- Multiple academic institutions
- AI system involved
- arXiv
1 source article · read the reporting →
Anonymous Spanish Lawyer (Tribunal Constitucional): AI-hallucinated content in court filing, Formal Reprimand (Apercibimiento) + Referral to Barcelona Bar for Disc
The AI system generated hallucinated content that was submitted in a court filing, potentially misleading the court.
1 source article · read the reporting →
Deloitte to refund Australian government after AI-generated report errors
Deloitte used generative AI (Azure OpenAI GPT-4o) to help produce an independent assurance review for Australia's Department of Employment and Workplace Relations. The report, published in July 2025, contained multiple errors including non-existent academic references and a fabricated court case. After the errors were flagged, Deloitte acknowledged the AI use and agreed to refund the final instalment of the A$439,000 contract. The report was corrected, but its substance and recommendations remained unchanged.
- Company involved
- Deloitte
- AI system involved
- Azure OpenAI GPT-4o
5 source articles · read the reporting →
Joel A. Rivera v. Triad Properties Corporation, et al. (N.D. Alabama): AI-hallucinated content in court filing, Public Reprimand; Disqualification; Bar…
The AI generated fabricated legal citations that were presented to the court, affecting the defendants and the judicial process.
- Company involved
- Burrill Watkins LLC
- AI system involved
- ChatGPT
1 source article · read the reporting →
Unnamed Brazilian litigant (): AI-hallucinated content in court filing, Appeal partially granted (reintegration suspended, rent imposed), but
AI-generated fake court precedents were included in a legal appeal, leading to sanctions against the litigant for bad-faith litigation.
- AI system involved
- ChatGPT
1 source article · read the reporting →
The People v. Raziel Ruiz Alvarez (CA California): AI-hallucinated content in court filing, Published order; monetary sanction payable by Counsel…
The AI generated fabricated legal citations that were included in a court filing, affecting the defendant's legal representation and the court's resources.
- Company involved
- Attorney Siddell
1 source article · read the reporting →
Ramirez v. Humala (E.D. New York): AI-hallucinated content in court filing, Monetary sanction jointly imposed on counsel and firm; order…
The AI generated nonexistent case citations that were filed in court, misleading the court and opposing counsel.
1 source article · read the reporting →
FTC settles with Sitejabber over reviews from customers who had not received goods
The US Federal Trade Commission has charged Sitejabber, an AI-enabled review platform, with deceiving consumers by publishing ratings and reviews submitted at the point of purchase, before buyers had received or experienced the products. The FTC alleges this artificially inflated average ratings and review counts for its clients. Sitejabber has agreed to a proposed consent order prohibiting it from making such misrepresentations in the future.
- Company involved
- Sitejabber
- AI system involved
- Sitejabber
10 source articles · read the reporting →
United States v. Hayes (E.D. California): AI-hallucinated content in court filing, Formal Sanction Imposed + Written Reprimand
The AI-generated fictitious citation was submitted in a motion to unseal, affecting the defendant's legal proceedings.
- Company involved
- Federal Public Defender's Office, Eastern District of California
1 source article · read the reporting →
Retorio AI personality test swayed by candidate appearance in BR experiment
Bayerischer Rundfunk journalists conducted experiments with Retorio's AI video interview analysis tool. The AI, which assesses personality traits from short videos, produced different scores when the same actress changed her appearance (glasses, headscarf, wig) or the video background and lighting were altered. The start-up Retorio acknowledged that the AI considers external image, similar to a human interviewer. Experts warned that such software could perpetuate stereotypes and unfairly affect job candidates.
- AI system involved
- Retorio AI
10 source articles · read the reporting →
Recurso de Suplicación 0005472/2025 (T.S.X. Galicia): AI-hallucinated content in court filing, Bar Referral
The AI generated false legal citations that were included in a court filing, misleading the court and potentially harming the client's case.
1 source article · read the reporting →
Michael L. Ruiz v. Magellan Financial & Insurance Services (2) (D. Arizona): AI-hallucinated content in court filing, Formal public reprimand
AI-hallucinated fake quotations were included in court filings submitted by counsel Elizabeth Tate, misleading the court.
1 source article · read the reporting →
