Where automated decisions went wrong

France uses AI to monitor mask-wearing on public transport

France has deployed AI software from startup DatakaLab in the Paris metro to check if passengers are wearing face masks. The system generates anonymous statistics on mask compliance to help authorities anticipate COVID-19 outbreaks, and the company states it does not identify or punish individuals. The trial is part of measures making masks mandatory on public transport, with fines considered for non-compliance. Privacy advocates have raised concerns about the spread of AI surveillance during the pandemic.

1 source article · read the reporting →

Steak 'n Shake sued over facial recognition kiosks under BIPA

A class action lawsuit alleges that Steak 'n Shake illegally collects facial biometrics from customers using PopID-powered self-ordering kiosks without proper consent. Plaintiff Michael Massel claims the restaurant chain failed to provide notice and obtain written consent as required by Illinois' Biometric Information Privacy Act (BIPA). The suit seeks damages of up to $5,000 per violation for anyone whose biometric data was collected in Illinois in the past five years.

Company involved

Steak 'n Shake

AI system involved

PopID biometric kiosks

6 source articles · read the reporting →

People Training OpenAI’s AI Fired for Using AI to Train the AI - 404 Media

Contractors were fired for using AI to complete their work.

Company involved

Mercor

1 source article · read the reporting →

New Meta smart glasses class action gives employers a reason to write the policy they've been puttin - hcamag.com

The smart glasses recorded bystanders without their consent and sent the footage to Meta for AI training.

Company involved

Meta Platforms

AI system involved

Meta AI-enabled Ray-Ban and Oakley smart glasses

1 source article · read the reporting →

Temple Police officer resigns after internal investigation into misuse of Flock Safety automatic license plate reader system - tdtnews.com

An officer conducted an unauthorized search of the Flock Safety automatic license plate reader system without a legitimate law enforcement purpose.

Company involved

Temple Police Department

AI system involved

Flock Safety automatic license plate reader (ALPR) system

1 source article · read the reporting →

Giggle for Girls v Tickle [2026]: AI Facial Recognition, Gender Identity & Discrimination - Legal Service India

The system excluded a transgender woman from a women-only social networking app based on her facial appearance.

Company involved

Giggle for Girls Pty Ltd

AI system involved

Giggle App

1 source article · read the reporting →

Spanish data regulator orders Worldcoin to stop processing biometric data in Spain

The Spanish Data Protection Agency (AEPD) has ordered a precautionary measure against Tools for Humanity Corporation, the company behind Worldcoin, to cease collection and processing of personal data in Spain. The AEPD received complaints alleging insufficient information, collection of data from minors, and inability to withdraw consent. The regulator acted under GDPR Article 66.1 to prevent potentially irreparable harm to individuals' data protection rights.

Company involved

Tools for Humanity Corporation

AI system involved

Worldcoin

8 source articles · read the reporting →

Meta accused of exploiting content moderators in Ghana

Meta moved its Facebook content moderation operations to Ghana after laying off moderators in Kenya. Workers in Accra report poor working conditions, low pay, and psychological harm from reviewing traumatic content. Foxglove is investigating and planning a legal challenge.

Company involved

Meta

AI system involved

Facebook content moderation system

3 source articles · read the reporting →

First Accident Before Test Phase: Robotaxis Launch in Furttal

Erster Unfall, bevor die Testphase beginnt: Robotaxis starten im Furttal - nzz.ch

Switzerland's first self-driving taxi service launched in the Furttal region near Zurich, but three days before the start a test vehicle had a minor accident in Dänikon, damaging a ground plate and two tires. The pilot project, called Iamo, aims to complement public transport, with a safety driver on board during the test phase.

Company involved

Swiss Transit Lab

1 source article · read the reporting →

Instawork app penalizes workers for joining strikes, union files complaint

Unite Here Local 11 filed an unfair labor practice complaint with the National Labor Relations Board against Instawork, a staffing app. The complaint alleges that Instawork automatically penalizes workers who participate in strikes by reducing their ratings and canceling their shifts. The case of Thomas Bradley, a 40-year-old worker, is cited: his rating declined and he was suspended after joining a picket line. Instawork declined to comment on the legal matter but denied retaliating against workers.

Company involved

Instawork

AI system involved

Instawork

10 source articles · read the reporting →

Paris taxi company G7 suspends Tesla Model 3 after fatal crash

A Paris taxi company, G7, suspended Tesla Model 3 cars from its fleet after an off-duty driver's car was involved in a collision that killed one person and injured 20 others. It was not clear whether Tesla's Autopilot was active. G7 said Tesla's initial inquiry found no malfunction but that the car appeared to accelerate when the driver tried to brake. The company said it would pay affected drivers' lost wages while a police investigation continues.

Company involved

G7

AI system involved

Tesla Model 3

7 source articles · read the reporting →

CNAF algorithm flags single mother Juliette for welfare fraud investigation

In 2022, Juliette, a single mother on welfare in France, was flagged by CNAF's secretive fraud detection algorithm. A fraud investigator later determined she owed thousands of euros, which were deducted from her monthly payments. The algorithm, which scores half of France's population, is accused of discriminating against vulnerable people by using factors like single parenthood and low income.

Company involved

CNAF

10 source articles · read the reporting →

Engineer.ai accused of exaggerating AI capabilities and using human engineers

Engineer.ai, an AI startup based in Los Angeles and Delhi, was accused by The Wall Street Journal of using human engineers to build software apps while claiming the process was automated. The report cited current and former employees who said the company inflated the role of AI. Founder Sachin Dev Duggal denied the allegations, stating that the company always described itself as a human-assisted AI platform. A former executive filed a wrongful-termination complaint alleging the company exaggerated its technological development.

Company involved

Engineer.ai

6 source articles · read the reporting →

Poland's Ministry of Labor profiles unemployed with discriminatory algorithm

In May 2015, Poland's Ministry of Labor and Social Policy introduced an algorithmic system to profile unemployed people into three categories based on their readiness to work, location, and disabilities. The system is accused of being discriminatory and lacking transparency, as it uses default criteria that may lead to social exclusion. A report by the Panoptykon Foundation in October 2015 highlighted these issues and recommended reforms, including granting the right to appeal.

Company involved

Ministry of Labor and Social Policy

10 source articles · read the reporting →

Dutch Tax Authority discriminated against dual nationality applicants for child care benefits using automated risk system.

The Dutch Data Protection Authority (AP) found that the Benefits Department of the Tax and Customs Administration (Belastingdienst) unlawfully processed the dual nationality of 1.4 million applicants for child care benefits. The data was used as an indicator in an automated risk-scoring system that flagged applications as high-risk, which was unnecessary and discriminatory. The AP concluded that the processing violated the GDPR and was discriminatory. The AP completed its fact-finding and will consider imposing a sanction.

Company involved

Belastingdienst

10 source articles · read the reporting →

Just Eat UK couriers deactivated by AI for minor overpayments

Just Eat UK couriers were deactivated from the platform after an AI system flagged them for alleged overpayments as small as £1.35. The system used GPS data to detect drivers moving away from restaurants while waiting for orders. Drivers claim they were asked to move by restaurant staff and that the deactivation was done with little human oversight. A report by Worker Info Exchange details eleven cases, and the company says each case is reviewed by a human before deactivation.

Company involved

Just Eat

10 source articles · read the reporting →

ElevenLabs recreates Alain Dorval's voice using AI without family approval

ElevenLabs, a UK-based AI startup, recreated the voice of deceased French dubbing artist Alain Dorval using AI for the film 'Armor'. His daughter, Aurore Bergé, criticized the company for publishing the cloned voice without her approval, stating she only agreed to a trial. ElevenLabs confirmed plans to recreate the voice and said it is working with the family.

Company involved

ElevenLabs

6 source articles · read the reporting →

Zoox issues voluntary software recall after robotaxi collision in Las Vegas

Zoox voluntarily recalled its robotaxi software after an unoccupied autonomous vehicle collided with a passenger vehicle in Las Vegas. The robotaxi misjudged the other car's intention, braking hard but making contact unavoidable. No injuries were reported and damage was minor. Zoox issued a software update and filed a report with NHTSA.

Company involved

Zoox

AI system involved

Zoox robotaxi

7 source articles · read the reporting →

Patagonia Targeted by Privacy Class Action for Use of Customer Service AI Software - Law.com

The system intercepted, recorded, and mined data from customer service calls, affecting customers' privacy.

Company involved

Patagonia

AI system involved

Talkdesk

1 source article · read the reporting →

Call Vendors Skirt Wiretap Suit Over AI Transcription Tool - Law360

The system recorded and analyzed patient phone calls without their consent.

Company involved

Heartland Dental LLC

1 source article · read the reporting →

Dialpad class action claims AI tech company illegally ‘taps’ T-Mobile customer service calls - Top Class Actions

The system intercepted and recorded telephone conversations of T-Mobile customers calling customer service without their consent.

Company involved

Dialpad Inc.

AI system involved

Dialpad conversation intelligence platform, DialpadGPT

1 source article · read the reporting →

Not Again! A Second Talkdesk Customer Is Sued for Its Use of Contact Center AI - cxtoday.com

The system intercepted, recorded, and analyzed the communications of LegalShield customers without their consent.

Company involved

LegalShield

AI system involved

Talkdesk CCaaS

1 source article · read the reporting →

Wingstop class action alleges company captured, stored customer voiceprints from phone orders - Top Class Actions

The system collected and stored voiceprints of customers placing phone orders without their consent.

Company involved

Wingstop

AI system involved

ConverseNow

1 source article · read the reporting →

Talkdesk Hit With Class Suit Over Customer Service Software - news.bloomberglaw.com

The system secretly recorded and analyzed customer service conversations of callers to Patagonia and Zumiez without consent.

Company involved

Talkdesk Inc.

AI system involved

CX Cloud

1 source article · read the reporting →