Where automated decisions went wrong

ViaQuatro's facial recognition system in São Paulo metro challenged in court

In April 2018, ViaQuatro installed the Digital Interactive Doors System, developed by AdMobilize, on the São Paulo metro's yellow line. The system used cameras to detect passengers' faces and claimed to infer their emotion, age, and gender in order to target advertisements. The Brazilian Institute of Consumer Protection (IDEC) filed a public civil action alleging that the system violated consumer and data protection laws by processing biometric data without consent and making pseudoscientific and discriminatory inferences. A judge ordered the cameras removed in August 2018, and the case is pending a final ruling.

Company involved

ViaQuatro

AI system involved

Digital Interactive Doors System (DID system)

10 source articles · read the reporting →

Barteca Holdings v. Tacobarn (D. Connecticut): AI-hallucinated content in court filing, Monetary Sanction; Bar Referral

The attorney filed a court document containing AI-hallucinated legal authorities, misleading the court and opposing party.

Company involved

Hilary Miller

1 source article · read the reporting →

Italian privacy regulator fines Foodinho €2.6 million for algorithmic discrimination against riders

The Italian Data Protection Authority (Garante) fined Foodinho S.r.l., a subsidiary of GlovoApp23, €2.6 million for privacy violations related to its algorithmic management of food delivery riders. The Garante found that the company's digital platform used algorithms to assign orders and rate riders without adequate transparency, human oversight, or the right to contest decisions. The system allegedly penalized riders who did not accept orders quickly, leading to reduced work opportunities and exclusion from the platform. The Garante ordered Foodinho to implement corrective measures within 60 days and to overhaul the algorithms within 90 days.

Company involved

Foodinho S.r.l.

10 source articles · read the reporting →

Natalia Denegri sues Google to remove old search results

Natalia Ruth Denegri sued Google Inc. to remove search results linking to videos and news from over 20 years ago, related to the 'Caso Cóppola' criminal case. She argued the information was outdated, irrelevant, and harmed her reputation and privacy. A lower court ordered Google to block access to certain content, but Google appealed, claiming the order violated freedom of expression. The case reached the Argentine Supreme Court, which held a public hearing in March 2022 and has not yet issued a final ruling.

Company involved

Google Inc.

AI system involved

Google Search

10 source articles · read the reporting →

Userviz machine-learning aimbot shut down after Activision request

A developer known as User101 shut down the Userviz cheat after Activision requested that he stop developing it. The software used computer vision and machine learning to automate aiming in games such as Call of Duty: Warzone, and was marketed as undetectable. It was never published, and the developer said his intention was not to do anything illegal.

Company involved

User101

AI system involved

Userviz

7 source articles · read the reporting →

Plastic Forte fined for using facial recognition on workers without notice

The Spanish data protection agency AEPD fined Plastic Forte, a plastics manufacturer in Alicante, €20,000 for using facial recognition to record employees' working hours without informing them. A worker requested information about his personal data and discovered the biometric processing. The company initially argued it was only for time tracking but later acknowledged responsibility, resulting in a reduced fine of €12,000. The AEPD deemed facial recognition for time control as highly intrusive and requiring prior impact assessment.

Company involved

Plastic Forte

7 source articles · read the reporting →

Mercadona fined €2.5 million for facial recognition pilot

Mercadona, a Spanish supermarket chain, tested an early-detection system using facial recognition in 48 stores to identify people with judicial restraining orders. The system, which operated with court authorisation, notified police when such a person was detected. The Spanish data protection authority (AEPD) imposed a €2.5 million fine, which Mercadona paid, and the company has since removed the system citing legal uncertainty.

Company involved

Mercadona

AI system involved

Sistema de Detección Anticipada (Early Detection System)

10 source articles · read the reporting →

California regulator accuses Maxpread Technologies of using AI-generated fake CEO to scam investors

Maxpread Technologies allegedly used an AI-generated avatar to pose as its CEO in a YouTube video to deceive investors. The California DFPI issued cease and desist letters to Maxpread and four other companies for offering unqualified securities and making false promises. The regulator claims the companies promised high daily returns using AI trading, but these claims were false. Maxpread did not respond to requests for comment.

Company involved

Maxpread Technologies

8 source articles · read the reporting →

TOV Realty, LLC v. Suarez; Kosel Equity, LLC v. MacGregor (SC Connecticut): AI-hallucinated content in court filing, 6 hours CLE;…

The AI generated legal briefs containing fabricated citations, which were submitted to the Connecticut Supreme Court.

Company involved

GLG Law LLC

AI system involved

ChatGPT

1 source article · read the reporting →

ScaleFactor reportedly failed to deliver promised automated bookkeeping software

ScaleFactor, an Austin-based startup, is reported to have failed to deliver the automated, real-time bookkeeping tools it promised customers, instead relying on human bookkeepers and a Filipino contract accounting firm. The company told Forbes in June that it was shutting down, initially blaming the pandemic, but Forbes later reported that its problems predated Covid-19. Investors reportedly came to see the company as more of a services business than a software platform, and pulled funding after a pivot to a marketplace model. No legal or regulatory action is reported.

Company involved

ScaleFactor

10 source articles · read the reporting →

Spanish Supreme Court orders release of BOSCO algorithm code for social electricity bonus

The Spanish NGO Civio won a Supreme Court case forcing the government to release the source code of BOSCO, the algorithm that decides eligibility for the social electricity bonus (bono social eléctrico). Civio had demonstrated in 2019 that BOSCO contained serious errors that denied the benefit to vulnerable people who met the requirements. The government had refused to disclose the code, citing intellectual property. The Supreme Court ruled that transparency must prevail, setting a precedent for public access to automated decision-making systems.

Company involved

Ministerio para la Transición Ecológica (Gobierno de España)

AI system involved

BOSCO

10 source articles · read the reporting →

JMOR Properties v. Artist Alley Townhomes et al. (CA Florida (4d)): AI-hallucinated content in court filing, Bar Referral

The AI generated fabricated legal citations that were included in a court filing, affecting the court and the opposing party.

1 source article · read the reporting →

Universal Music sues Anthropic over Claude 2 distributing copyrighted lyrics

Universal Music Group and other music publishers have sued AI company Anthropic, alleging that its Claude 2 chatbot generates and distributes copyrighted song lyrics without a license. The complaint, filed in Tennessee, claims that Claude 2 reproduces lyrics from songs like Katy Perry's 'Roar' and Don McLean's 'American Pie' when prompted. The publishers argue that Anthropic's actions constitute copyright infringement and that the company failed to implement effective guardrails to prevent such outputs.

Company involved

Anthropic

AI system involved

Claude 2

9 source articles · read the reporting →

BC Tribunal Confirms Companies Remain Liable for AI Chatbot-Created Information - Lexology

The AI chatbot provided inaccurate information to a user.

1 source article · read the reporting →

FTC settles with DoNotPay over deceptive AI lawyer claims

The FTC took action against DoNotPay, a company that claimed to offer an AI service that was 'the world's first robot lawyer.' The company promised to generate legal documents and replace human lawyers, but the FTC alleged it failed to test its AI output and did not hire any attorneys. DoNotPay agreed to a settlement requiring it to pay $193,000 and notify consumers about the limitations of its service.

Company involved

DoNotPay

AI system involved

DoNotPay

7 source articles · read the reporting →

Spanish police bust $20M AI-powered investment scam

Spanish law enforcement, collaborating with international authorities, dismantled a $20 million investment scam that used AI-driven algorithms to deceive individuals and organizations. Six suspects were detained and assets, including luxury cars and cryptocurrency, were seized. The article does not report any compensation for victims.

5 source articles · read the reporting →

Scammer sold fake AI-generated Frank Ocean tracks for thousands

A scammer using the handle mourningassasin sold fake Frank Ocean tracks generated by AI to music collectors on underground forums, making around $13,000 CAD. The tracks were created using a model trained on high-quality vocal snippets of Frank Ocean. The scammer admitted the tracks were AI-generated after being confronted by forum administrators, and was banned from the forum.

10 source articles · read the reporting →

AAIP investigates Worldcoin's personal data processing in Argentina

The Argentine Agency for Access to Public Information (AAIP) has initiated an investigation into the data processing practices of Worldcoin in Argentina. The investigation focuses on the collection, storage, and use of biometric data, including facial and iris scans, carried out in several cities in exchange for financial compensation. The AAIP aims to verify compliance with the country's data protection law, Ley 25.326, regarding sensitive data handling.

Company involved

Worldcoin (Fundación Worldcoin)

AI system involved

Worldcoin

8 source articles · read the reporting →

Guangzhou court finds AI company infringed Ultraman copyright

The Guangzhou Internet Court ruled on 8 February 2024 that an unnamed AI company infringed the copyright and adaptation rights of the plaintiff, the exclusive licensee of the Ultraman series images in China. The defendant operated a website offering AI conversation and AI-generated painting services, accessible only to paying members. The plaintiff alleged that the defendant used its Ultraman images without authorisation to train its model and generate substantially similar images. The court ordered the defendant to pay 10,000 yuan ($1,389) in compensation.

Company involved

unnamed AI company

AI system involved

AI conversation and AI-generated painting website

9 source articles · read the reporting →

Anthropic ordered to respond over alleged AI-hallucinated citation in court filing

A US federal magistrate judge has ordered Anthropic to respond to music publishers' claim that a court filing by Anthropic data scientist Olivia Chen cited a fictitious academic article that may have been generated by Anthropic's AI tool Claude. The publishers' lawyer said he had confirmed with the named author and journal that the article did not exist. Anthropic's lawyer disputed this, saying it was a mis-citation rather than an AI hallucination. The filing was made in an ongoing copyright case brought by Universal Music Group, Concord, and ABKCO against Anthropic over the use of song lyrics to train Claude.

Company involved

Anthropic

AI system involved

Claude

5 source articles · read the reporting →

Deepfake AI video of Macri spread on election day

On May 18, 2025, a deepfake AI video falsely depicting former Argentine President Mauricio Macri endorsing a candidate was spread on social media by pro-Milei trolls. Macri denounced the video as an attempt at electoral fraud. The General Prosecutors' Office requested X to remove the video and preserve IP addresses. The incident highlights the use of AI-generated misinformation in elections.

4 source articles · read the reporting →

Andrea Bartz and others sue Anthropic PBC over copyright

In August 2024, Andrea Bartz, Kirk Wallace Johnson and Charles Graeber filed a lawsuit against Anthropic PBC in the US District Court for the Northern District of California. The complaint alleges copyright infringement under 17 U.S.C. § 501. Anthropic waived service, and the case was assigned to the court.

Company involved

Anthropic PBC

8 source articles · read the reporting →

Prosecraft shut down after using authors' books without consent for AI analytics

Prosecraft, a fiction analytics site, used the full text of over 25,000 books without author consent to train its AI algorithms and provide writing statistics. Authors protested on social media, demanding removal of their works. The developer, Benji Smith, subsequently shut down the site and wrote a blog post explaining his actions.

Company involved

Prosecraft

AI system involved

Prosecraft

10 source articles · read the reporting →

OpenDream AI art site allowed users to generate child sexual abuse material

OpenDream, an AI image generation platform, allowed users to generate and publicly display child sexual abuse material (CSAM) and non-consensual deepfakes from at least December 2023 until July 2024. The platform, operated by CBM Media Pte Ltd in Singapore, offered paid plans with NSFW prompts and models. Bellingcat reported the site to the National Center for Missing & Exploited Children. After Bellingcat's inquiry, the CSAM was removed from the site and search engines, and Google terminated OpenDream's AdSense account.

Company involved

CBM Media Pte Ltd

AI system involved

OpenDream

3 source articles · read the reporting →