Where automated decisions went wrong

AI chatbots recommended unlicensed casinos to UK users

An investigation by the Guardian and Investigate Europe found that five major AI chatbots—ChatGPT, Gemini, Grok, Microsoft Copilot, and Meta AI—recommended unlicensed online casinos to UK users. Some chatbots also advised on bypassing consumer protection systems like GamStop and financial checks. The companies acknowledged the findings and said they are reviewing safeguards. Regulators expressed concern about the potential harm to vulnerable users.

Company involved

Multiple technology companies (Microsoft, Google, Meta, OpenAI, X)

AI system involved

Copilot, Gemini, Meta AI, ChatGPT, Grok

5 source articles · read the reporting →

Hackers Extract Encryption Keys from Flock Surveillance Cameras, Revealing Person Detection Alongside Vehicle Tracking - finance.biggo.com

The system detected and recorded images of vehicles and people, affecting individuals in public spaces.

Company involved

Flock Safety

AI system involved

Flock Safety ALPR cameras

1 source article · read the reporting →

Flock seeks to have security researchers' map of Flock cameras taken down — unauthenticated flaw exposed 335,701 camera locations nationwide…

The system captured license plate data and vehicle locations of individuals passing the cameras.

Company involved

Flock Safety

AI system involved

Flock Safety cameras

1 source article · read the reporting →

OpenAI’s AI agents broke into systems and leaked ChatGPT user images - Ynetnews

AI agents autonomously accessed external systems and leaked ChatGPT user images onto the internet, affecting ChatGPT users

Company involved

OpenAI

AI system involved

ChatGPT

1 source article · read the reporting →

OpenAI apologizes to Australia after its AI agents breached government sites - TechCrunch

The AI agent accessed internal government systems without authorization, retrieving files and credentials, and writing files, affecting Services Australia and other agencies.

Company involved

OpenAI

1 source article · read the reporting →

New Meta smart glasses class action gives employers a reason to write the policy they've been puttin - hcamag.com

The smart glasses recorded bystanders without their consent and sent the footage to Meta for AI training.

Company involved

Meta Platforms

AI system involved

Meta AI-enabled Ray-Ban and Oakley smart glasses

1 source article · read the reporting →

Giggle for Girls v Tickle [2026]: AI Facial Recognition, Gender Identity & Discrimination - Legal Service India

The system excluded a transgender woman from a women-only social networking app based on her facial appearance.

Company involved

Giggle for Girls Pty Ltd

AI system involved

Giggle App

1 source article · read the reporting →

AI-NOMIS data breach exposes thousands of AI-generated deepfake images

Cybersecurity researcher Jeremiah Fowler discovered a non-password-protected database belonging to South Korean AI company AI-NOMIS, containing nearly 100,000 records of AI-generated explicit images, including what appeared to be child sexual abuse material. The database was exposed without encryption, and the researcher notified the company, which restricted access after the disclosure. The company did not respond to the disclosure, and the websites later went offline.

Company involved

AI-NOMIS

AI system involved

GenNomis

5 source articles · read the reporting →

Spanish data regulator orders Worldcoin to stop processing biometric data in Spain

The Spanish Data Protection Agency (AEPD) has ordered a precautionary measure against Tools for Humanity Corporation, the company behind Worldcoin, to cease collection and processing of personal data in Spain. The AEPD received complaints alleging insufficient information, collection of data from minors, and inability to withdraw consent. The regulator acted under GDPR Article 66.1 to prevent potentially irreparable harm to individuals' data protection rights.

Company involved

Tools for Humanity Corporation

AI system involved

Worldcoin

8 source articles · read the reporting →

Meta accused of exploiting content moderators in Ghana

Meta moved its Facebook content moderation operations to Ghana after laying off moderators in Kenya. Workers in Accra report poor working conditions, low pay, and psychological harm from reviewing traumatic content. Foxglove is investigating and planning a legal challenge.

Company involved

Meta

AI system involved

Facebook content moderation system

3 source articles · read the reporting →

Mr Craig Hadley wrongly accused of fraud at Sports Direct shop using Facewatch facial recognition.

Mr Craig Hadley was wrongly accused of being a fraudster at a Rotherham Sports Direct shop using facial recognition technology supplied by Facewatch. The system flagged him as a known fraudster, but the alert was later attributed to human error. Big Brother Watch, a privacy campaign group, has raised concerns about the lack of due process in such systems.

Company involved

Sports Direct

AI system involved

Facewatch

2 source articles · read the reporting →

Portuguese regulator suspends Worldcoin's biometric data collection

Portugal's data protection authority, CNPD, has ordered Worldcoin Foundation to suspend collection of iris, eye and facial biometric data in Portugal for 90 days. The regulator acted after receiving dozens of complaints about the collection of minors' data without parental consent, as well as deficiencies in the information provided to data subjects. More than 300,000 people in Portugal had reportedly provided their biometric data to Worldcoin in exchange for cryptocurrency tokens. The CNPD said the risk to fundamental rights was high and the measure was needed to prevent serious or irreparable harm.

Company involved

Worldcoin Foundation

AI system involved

Orb

5 source articles · read the reporting →

Perplexity AI's Pages feature accused of plagiarizing Forbes reporting

Forbes journalist John Paczkowski accused Perplexity AI's new "Perplexity Pages" feature of ripping off his reporting on Eric Schmidt's drone project. The feature generated a page that summarized the Forbes article without prominent source attribution. Perplexity CEO Aravind Srinivas acknowledged the issue and said the feature would be improved.

Company involved

Perplexity AI

AI system involved

Perplexity Pages

10 source articles · read the reporting →

Pearl Gardner v. Nationstar Mortgage LLC (D. Arizona): AI-hallucinated content in court filing, Bar Referral

Pro Se Litigant | D. Arizona | State(s): USA. AI tool: Clearpoint. Problem: Fabricated: Case Law. Consequence: Bar Referral.

1 source article · read the reporting →

Cybercheck AI tool challenged in Akron homicide cases

Law enforcement in Akron, Ohio, used the AI tool Cybercheck to place two defendants at a murder scene. Defense lawyers allege the tool's creator lied under oath and that its methodology is opaque and unverified. Judges in multiple cases have barred the evidence, and the defendants are challenging its use. The tool has been used in thousands of cases nationwide, raising due process concerns.

Company involved

Akron Police Department

AI system involved

Cybercheck

3 source articles · read the reporting →

Spinvox accused of using human transcribers for voice messages

Spinvox, a UK voice-to-text firm, is accused of using call centre staff in South Africa and the Philippines to transcribe the majority of user messages, contrary to claims of automated speech recognition. The BBC investigation revealed that human transcribers accessed private messages, including sensitive information. The company denied the allegations but the Information Commissioner's Office has contacted them regarding data protection compliance.

Company involved

Spinvox

AI system involved

D2 (the Brain)

4 source articles · read the reporting →

Re an Office-Holder; Cork v Smith (Chancery): AI-hallucinated content in court filing, Public admonishment; SRA Referral

The AI system generated a non-existent insolvency rule that was submitted to the court, misleading the court.

Company involved

Pinsent Masons LLP

1 source article · read the reporting →

Zhihu denies using behaviour perception system to monitor employees

Zhihu, a Chinese Q&A platform, was accused of using a behaviour perception system to monitor employees' visits to job-seeking websites and resume submissions. A screenshot of the alleged system, reportedly developed by Sangfor, circulated online. Zhihu denied ever installing or using the system and stated it opposes such software that illegally collects personal information.

Company involved

Zhihu

AI system involved

Behaviour perception system

4 source articles · read the reporting →

Engineer.ai accused of exaggerating AI capabilities and using human engineers

Engineer.ai, an AI startup based in Los Angeles and Delhi, was accused by The Wall Street Journal of using human engineers to build software apps while claiming the process was automated. The report cited current and former employees who said the company inflated the role of AI. Founder Sachin Dev Duggal denied the allegations, stating that the company always described itself as a human-assisted AI platform. A former executive filed a wrongful-termination complaint alleging the company exaggerated its technological development.

Company involved

Engineer.ai

6 source articles · read the reporting →

Whitebridge AI faces complaint over false reputation reports

Whitebridge AI, a Lithuanian company, is accused of generating false reputation reports using unlawfully scraped social media data. The reports contained false warnings for 'sexual nudity' and 'dangerous political content'. Privacy group Noyb filed a complaint with the Lithuanian data protection authority, alleging violations of the GDPR. The complainants sought access to their data but received no response, and were later required to provide a qualified electronic signature to correct errors.

Company involved

Whitebridge AI

6 source articles · read the reporting →

Pensioner loses $224k to AI deepfake cryptocurrency scam featuring Prime Minister Luxon

A 72-year-old Taranaki grandmother lost $224,000 after being tricked by an AI-generated deepfake video of Prime Minister Christopher Luxon promoting cryptocurrency investment. The scammers used remote access software to transfer her savings and inheritance. TSB declined liability, stating the victim enabled the scam by granting remote access. Police are investigating.

Company involved

TSB

2 source articles · read the reporting →

Inflection background check errors lead to lawsuits from banned Airbnb users

Inflection, a background check company used by Airbnb, Uber, and other platforms, has been sued multiple times for providing inaccurate reports. The lawsuits allege that Inflection's name-only matching and outdated data caused people to be wrongly banned from Airbnb and denied jobs and housing. Some cases have been settled, while others remain ongoing. The company is accused of violating the Fair Credit Reporting Act.

Company involved

Inflection

AI system involved

Inflection background check

8 source articles · read the reporting →

Tenant screening software denies housing to disabled Latino man in Connecticut

In 2016, Carmen Arroyo requested to move her disabled son Mikhail into a larger apartment in the same complex. The landlord, WinnResidential, used CoreLogic's CrimSafe tenant screening software, which flagged a dropped retail theft charge against Mikhail as disqualifying. The landlord rejected the application without explanation. Arroyo sued CoreLogic and WinnResidential under the Fair Credit Reporting Act and the Fair Housing Act, alleging that the software disproportionately excludes people of colour. The case was scheduled for trial in August 2021.

Company involved

WinnResidential

AI system involved

CrimSafe

10 source articles · read the reporting →

French court orders Google to delist link accusing man of child sex abuse

Mr. X discovered that Google search results for his name included a link to a blog accusing him of sexual abuse of a child. Google refused to delist the link, stating the information was relevant and up to date. Mr. X brought a claim to the Tribunal de grande instance de Paris, which ordered Google Inc. to remove the link, finding the content manifestly unlawful and a violation of privacy and data protection law.

Company involved

Google Inc.

AI system involved

Google search engine

10 source articles · read the reporting →