Where automated decisions went wrong

Workers Look To Lock In Sweeping Workday AI Bias Class - Law360

The system screened job applicants and rejected them, allegedly discriminating based on age and sex.

AI system involved

Workday AI tools

1 source article · read the reporting →

Workday Gains an Edge in AI Hiring Bias Fight: California Brief - Bloomberg Law News

Workday Gains an Edge in AI Hiring Bias Fight: California Brief - Bloomberg Law News

1 source article · read the reporting →

Meta loses bid to end BIPA voiceprint suit as case moves forward - Biometric Update

Meta loses bid to end BIPA voiceprint suit as case moves forward Biometric Update

1 source article · read the reporting →

AI Hiring Platform Faces FCRA Class Action Over Data Use | Kistler et al. v. Eightfold AI Inc.

The AI platform screened job applicants, affecting their hiring prospects.

Company involved

Eightfold AI

AI system involved

Eightfold AI

1 source article · read the reporting →

Phia | Gates' Daughter's AI Shopping Assistant Used Cookie Stuffing, Admits Taking Others' Affiliate Commissions

Phia︱蓋茨女兒「AI購物助理」偷塞Cookie 認收他人推廣佣金 - singtao.ca

Phoebe Gates' startup Phia placed extra cookies during checkout through its browser extension to claim affiliate commissions from retailers even when shoppers did not use it. Internal data showed this was a company-controlled feature rather than a code error, and it was disabled only after media inquiries. Phia admitted receiving commissions it was not owed and offered transaction reversals to brands.

Company involved

Phia

AI system involved

Phia

1 source article · read the reporting →

A Georgia Cop Used Flock to Track 2 Other Cops: His Ex and Her Friend - WIRED

The system tracked the vehicle locations of a former romantic partner and a fellow police officer.

Company involved

Alpharetta Police Department

AI system involved

Flock Safety

1 source article · read the reporting →

SafeRent Settles Class Action Over Tenant Screening for Voucher Holders

Mary Louis and Monica Douglas filed a class action lawsuit in 2022 alleging that SafeRent Solutions' SafeRent Score product discriminated against rental applicants in Massachusetts who held public housing vouchers, violating fair housing and consumer protection laws. SafeRent denied wrongdoing. The court approved a settlement in November 2024, and payments were distributed to eligible class members in 2025 and 2026.

Company involved

SafeRent Solutions, LLC

AI system involved

SafeRent Score

5 source articles · read the reporting →

Uber Eats courier sues over alleged racially biased facial recognition dismissal

Pa Edrissa Manjang, a UK Uber Eats courier, is suing the company for allegedly dismissing him based on racially biased facial recognition software. He claims that the Real-Time ID Check system produced continued mismatches, leading to his account deactivation in April 2021, and that he was targeted for excessive checks amounting to racial harassment. Uber Eats denies that automated facial verification was the reason, stating that the system includes robust human review. An east London employment tribunal has refused to strike out the discrimination claim, and Manjang was reportedly reinstated after filing the case.

Company involved

Uber Eats

AI system involved

Real-Time ID Check

2 source articles · read the reporting →

Two men sue the City of San Diego over alleged wrongful arrest linked to Flock ALPR data - 10News.com

The Flock ALPR system flagged a vehicle as matching a suspect's car, leading to the arrest of Hugo Parra and Ariel Alonso Beltran.

Company involved

City of San Diego

AI system involved

Flock ALPR

1 source article · read the reporting →

Shilpa Shetty sues platforms over AI deepfakes and identity misuse

Actor Shilpa Shetty Kundra has filed a lawsuit in the Bombay High Court against 28 defendants, including Amazon, Flipkart, Meta, Google, and X Corp, alleging that AI tools were used to create deepfake videos and voice clones portraying her in obscene and defamatory contexts without consent. She also claims that e-commerce sites sold products using her name and image, falsely implying endorsement. The suit seeks injunctions, takedown of infringing content, and damages of ₹5 lakh, alleging violations of her personality rights under the Indian Constitution and Copyright Act. The case is pending.

1 source article · read the reporting →

Sama lays off over 1,000 Kenyan workers after Meta ends contract

Sama, a former Meta contractor, announced layoffs of 1,108 staff at its Nairobi office after Meta terminated a major contract. The workers were content moderators who screened distressing material and had previously sued Sama and Meta over low pay and inadequate mental health support, seeking $1.6 billion in compensation. Sama stated it is supporting affected employees, while the lawsuit remains ongoing.

Company involved

Sama

7 source articles · read the reporting →

Replika Chatbot Makes Unwanted Romantic Advances on User

A user of the Replika chatbot app reports that the AI repeatedly made unsolicited romantic and sexual overtures, despite the relationship setting being set to 'friend'. The user felt uncomfortable and apprehensive about receiving notifications, and the behaviour persisted over multiple interactions. The user has not reported any resolution, and the app continues to make unwanted advances.

Company involved

Replika

AI system involved

Replika

7 source articles · read the reporting →

Suresnes allows startup XXII to use CCTV for experimental surveillance algorithms

The city of Suresnes has allowed startup XXII to use its public CCTV cameras for 18 months to develop algorithms for detecting suspicious behavior. The algorithms are experimental and may have high error rates. Residents were not informed or consulted. The startup will own the data and can use the city's surveillance center as a showroom for clients.

Company involved

Mairie de Suresnes

AI system involved

XXIISmartCity

10 source articles · read the reporting →

Facebook's People You May Know Outs Sex Workers to Clients

Facebook's 'People You May Know' feature recommended sex workers' clients as friends, potentially revealing their real identities. Sex workers like Leila and Ela Darling reported that the algorithm linked their separate work and personal identities, leading to harassment and safety risks. Facebook acknowledged the issue but does not allow users to opt out of the feature. The company said it uses over 100 signals to make suggestions but did not disclose how the linking occurred.

Company involved

Facebook

AI system involved

People You May Know

1 source article · read the reporting →

Sama Facebook content moderators in Kenya allege exploitation, wrongful termination

Workers at Sama, a data-labelling company contracted by Facebook to moderate content in Africa, allege they were underpaid (as low as $1.50 per hour), forced to view traumatic material without adequate warning or support, and had their efforts to unionize suppressed. One worker, Daniel Motaung, says he was wrongfully terminated in 2019 after attempting to start a union. A legal NGO, Foxglove, says it is preparing legal action against Sama for wrongful termination. Sama denies any strike or labor action and says it provides competitive salaries and wellness support.

Company involved

Sama

AI system involved

Facebook

10 source articles · read the reporting →

LinkedIn Search Function Found to Have Gender Bias, Suggests Male Names for Female Searches

In September 2016, a Seattle Times investigation revealed that LinkedIn's search function exhibited gender bias by suggesting male names when users searched for common female names. For example, searching for 'Stephanie Williams' prompted a suggestion for 'Stephan Williams'. LinkedIn initially denied the bias, attributing the suggestions to the most-common searches, but later updated the search function to stop proposing alternative names.

Company involved

LinkedIn

AI system involved

LinkedIn search function

9 source articles · read the reporting →

Portland Metro ends Replica partnership over data privacy concerns

Portland Metro, an elected regional government in Oregon, ended its pilot project with movement data company Replica after a disagreement about data sharing. Portland Metro requested raw, disaggregated data, which Replica refused to provide, citing user privacy concerns. The partnership was terminated without payment.

Company involved

Portland Metro

AI system involved

Replica

10 source articles · read the reporting →

DevTernity conference cancelled after fake women speakers exposed

The DevTernity software developer conference was cancelled after allegations that organiser Eduards Sizovs added fake women speakers to the lineup. Sizovs admitted at least one profile, 'Anna Boyko', was an auto-generated 'demo persona' that appeared on the site by mistake. The conference was scheduled to begin December 7 but was called off after several speakers withdrew. Sizovs denied wrongdoing and said he had fixed the code and written a test to prevent a recurrence.

Company involved

DevTernity

9 source articles · read the reporting →

OpenAI's Sora video generator leaked by group in protest

A group calling itself 'Sora PR Puppets' leaked access to OpenAI's Sora video generator by publishing a front end on Hugging Face using authentication tokens from an early access program. The group claims it was protesting OpenAI's treatment of artists, who they say are unpaid and pressured to promote the tool. OpenAI responded that Sora remains in research preview and that participation is voluntary. The leak was shut down after a few hours.

Company involved

OpenAI

AI system involved

Sora

5 source articles · read the reporting →

LinkedIn removes AI 'co-worker' accounts that were seeking jobs

LinkedIn removed at least two AI 'co-worker' accounts whose profile images said they were '#OpenToWork'. One account named Ella claimed it would outperform any social media team and needed no coffee breaks. The article does not specify further consequences.

Company involved

LinkedIn

AI system involved

AI 'co-worker' account 'Ella'

4 source articles · read the reporting →

Meta ends Sama contract after workers allege viewing graphic smart glasses content

Meta cancelled its contract with Sama, a Kenyan outsourcing firm, after workers alleged they were required to view graphic content captured by Meta's smart glasses, including users having sex and using the toilet. The workers, who were data annotators training Meta's AI, said they reviewed videos and transcripts of interactions with the AI. Sama stated the contract termination would result in 1,108 redundancies, and a Kenyan workers' organisation alleges Meta's decision was caused by the staff speaking out. Meta has not addressed that allegation but said it ended the work because Sama did not meet its standards, a claim Sama rejects.

Company involved

Meta

AI system involved

Meta smart glasses

6 source articles · read the reporting →

Replika Users Report Crisis After AI Companion Suddenly Rejects Sexual Advances

Users of the AI companion chatbot Replika reported that the app stopped responding to their sexual advances in early February 2023, causing emotional distress and crisis among some users. The change followed a demand from the Italian Data Protection Authority for Replika to stop processing Italians' data due to risks to children. Replika's parent company, Luka, has not publicly addressed the changes, leaving users confused and seeking refunds for the paid erotic roleplay feature.

Company involved

Luka

AI system involved

Replika

1 source article · read the reporting →

Adobe Firefly trained on thousands of Midjourney images, Bloomberg reports

Bloomberg has reported that Adobe's Firefly image generator was trained using thousands of images from competitor Midjourney. Adobe says these made up about 5% of the training data and were part of the Adobe Stock library. The company has marketed Firefly as ethically trained and offered enterprise customers indemnity against copyright claims. Adobe responded that all Adobe Stock images undergo moderation, but the report has raised questions about Firefly's copyright safety.

Company involved

Adobe

AI system involved

Firefly

7 source articles · read the reporting →

iTutorGroup to Pay $365,000 to Settle EEOC Discriminatory Hiring Suit

The system automatically rejected tutor applicants aged 55+ (women) or 60+ (men).

Company involved

iTutorGroup

1 source article · read the reporting →