Where automated decisions went wrong

Meta sues Voyager Labs for scraping Facebook and Instagram user data

Meta filed a legal action against Voyager Labs, alleging that the company used fake accounts and proprietary software to scrape user data from Facebook and Instagram. The scraping collected profile information, posts, friends lists, photos and comments. Meta disabled Voyager's accounts and sought a permanent injunction. The case was settled in December 2024, with Voyager agreeing to a permanent injunction and monetary payment.

Company involved

Voyager Labs

10 source articles · read the reporting →

ChatGPT falsely tells users OpenCage offers phone lookup service

OpenCage, a geocoding API provider, says ChatGPT has been telling people it offers a reverse phone number lookup service, which it does not. Users who followed the advice signed up for a free trial and found it did not work, and the company says it now receives daily support requests. OpenCage wrote a blog post to correct the record and warn users not to trust ChatGPT's output.

AI system involved

ChatGPT

7 source articles · read the reporting →

Study finds Italian car insurers charge more based on birthplace

A study by the Universities of Padua, Udine and Carnegie Mellon found that Italian car insurers, including Genertel, Mps, Quixa and Con.Te, use birthplace and citizenship in pricing algorithms, charging some drivers over €1,000 more. The practice was ruled against in a 2018 decree involving Linear, but the study says it continues. The companies contacted denied or explained the findings.

Company involved

Genertel, Mps, Quixa, Con.Te

8 source articles · read the reporting →

Portland Metro ends Replica partnership over data privacy concerns

Portland Metro, an elected regional government in Oregon, ended its pilot project with movement data company Replica after a disagreement about data sharing. Portland Metro requested raw, disaggregated data, which Replica refused to provide, citing user privacy concerns. The partnership was terminated without payment.

Company involved

Portland Metro

AI system involved

Replica

10 source articles · read the reporting →

DeepScore markets facial and voice analysis app for trustworthiness scoring despite experts' doubts

DeepScore, a Tokyo-based company, is marketing an app that uses facial and voice recognition to score people's trustworthiness for lenders and insurers in Japan, Indonesia, Vietnam and the Philippines. The company says the app can detect deception with 70 per cent accuracy, but researchers and privacy advocates say there is no reliable scientific basis for such judgments and warn of discrimination and privacy harms. The chief executive said the system is only one part of lenders' and insurers' decision-making and that people can choose not to use it. Critics respond that an unequal balance of power makes consent difficult.

Company involved

DeepScore

AI system involved

DeepScore

6 source articles · read the reporting →

Facebook Allowed Age-Targeted Credit Card Ads Violating Its Policy

The Markup found that four companies—Aspiration, Hometap, Chime, and Varo Bank—ran Facebook ads for credit cards and home equity loans that were targeted by age, excluding users under 25 or 35. This practice violates Facebook's own anti-discrimination policy and may violate the Equal Credit Opportunity Act and California's Unruh Civil Rights Act. Facebook did not respond to requests for comment, and some advertisers said they would review their ad targeting.

Company involved

Facebook

AI system involved

Facebook Ad Platform

9 source articles · read the reporting →

Clearview AI to pull out of Canada and stop working with RCMP amid privacy investigation

Clearview AI, the US facial recognition company, has agreed to stop offering its services in Canada amid a joint investigation by Canadian privacy regulators. The company's technology, which matches images against billions of photos scraped from the internet, was used by the RCMP and more than 20 other police services across Canada. The Office of the Privacy Commissioner of Canada announced the 'indefinite suspension' of Clearview's contract with the RCMP, its last remaining Canadian client. The investigation into whether the technology breaches Canadian privacy laws will continue.

Company involved

Clearview AI

AI system involved

Clearview AI

10 source articles · read the reporting →

iRobot considers selling Roomba home mapping data to advertisers

iRobot's Roomba vacuum cleaners with mapping technology collect detailed floor plan data of users' homes. The company's CEO stated he is considering selling this data to companies like Amazon, Apple, or Google for advertising purposes. Robotics experts express concern about the lack of user consent and potential privacy implications.

Company involved

iRobot

AI system involved

Roomba 960/980

10 source articles · read the reporting →

EPIC lawsuit challenges USPS secret surveillance program using facial recognition

The United States Postal Service's law enforcement wing, the Postal Inspection Service, ran a secret surveillance program called the Internet Covert Operations Program (iCOP) that used facial recognition from Clearview AI and social media monitoring tools to infiltrate online communities and monitor protests. The Electronic Privacy Information Center (EPIC) sued the USPS for failing to conduct and publish a privacy impact assessment as required by the E-Government Act. A federal district court dismissed the lawsuit on standing grounds, ruling that EPIC did not suffer a cognizable injury from the agency's refusal to disclose information about the program.

Company involved

United States Postal Service

AI system involved

Internet Covert Operations Program (iCOP)

10 source articles · read the reporting →

DeepSeek exposed user data via open ClickHouse database

Cloud security firm Wiz discovered a ClickHouse database belonging to DeepSeek that was open to the internet without authentication, containing over a million lines of logs with chat histories, secret keys and backend details. Wiz disclosed the breach to DeepSeek, which promptly locked down the database. The incident highlights security risks in rapidly deploying AI services.

Company involved

DeepSeek

AI system involved

DeepSeek-R1

5 source articles · read the reporting →

EviCore denied heart catheterization for patient using algorithm

In fall 2021, Little John Cupp's doctor requested a left heart catheterization exam. EviCore, a company hired by UnitedHealthcare, denied the request twice using an algorithm called 'the dial' that adjusts thresholds for review. The algorithm flagged the request for review, and EviCore's doctors determined it was not medically necessary. Cupp did not receive the procedure and his symptoms continued.

Company involved

EviCore (a Cigna company)

AI system involved

the dial

6 source articles · read the reporting →

Bavarian police test Palantir data mining with real personal data

The Bavarian State Criminal Police Office (LKA) has been testing Palantir's data mining software, called VeRa, with real personal data for months. The Bavarian data protection commissioner only learned of the test through a media inquiry and has announced a review. The Interior Ministry claims the test is lawful under current law, but critics argue a legal basis is missing.

Company involved

Bayerisches Landeskriminalamt

AI system involved

VeRa

7 source articles · read the reporting →

Five Illinois residents sue facial recognition company Pimeyes under BIPA

Five Illinois residents are suing Pimeyes, a facial recognition search engine, alleging that the company violated the Illinois Biometric Information Privacy Act by collecting and using their facial data without consent. The lawsuit claims that Pimeyes made the residents' photos and information available in search results, enabling unauthorized access. The company has not yet been served with the lawsuit, and its CEO stated he had no prior knowledge of the case.

Company involved

Pimeyes

AI system involved

Pimeyes

7 source articles · read the reporting →

Paradox security vulnerability exposed candidate data to researchers

On June 30, 2025, security researchers discovered a vulnerability in Paradox's test account that allowed access to chat interaction records. The researchers viewed five candidates' personal information including names, email addresses, phone numbers, and IP addresses. Paradox fixed the issue within hours and stated that no data was leaked publicly. The company has since implemented new security measures.

Company involved

Paradox

AI system involved

Paradox conversational AI platform

10 source articles · read the reporting →

Dutch DPA fines Clearview AI €30.5 million for GDPR violations

The Dutch Data Protection Authority fined Clearview AI €30.5 million for creating an illegal database of biometric codes scraped from social media without consent. The regulator accused the company of failing to inform individuals about how their data is used and continuing violations after investigation. Clearview AI denies the allegations, stating it has no presence in the Netherlands or EU and that the fine is unenforceable.

Company involved

Clearview AI

AI system involved

Clearview AI facial recognition database

7 source articles · read the reporting →

NHTSA investigation PE24016: Unexpected ADS behavior

Waymo's automated driving system exhibited unexpected behavior during driving.

Company involved

Waymo

1 source article · read the reporting →

TransUnion AI tenant screening denied 75-year-old man apartment due to mistaken littering conviction

Chris Robinson, then 75, applied for a senior living apartment in California. The property manager used an AI screening program from TransUnion, which assigned him a low score based on a mistaken conviction for littering that belonged to a different person with the same name in Texas. Robinson lost the apartment and his application fee. A federal class-action lawsuit against TransUnion moved toward a $11.5 million settlement in 2023.

10 source articles · read the reporting →

Didi fined for over-collecting personal data of users

The Cyberspace Administration of China fined Didi Global Inc. for violating data protection laws. The investigation found that Didi had over-collected personal data from passengers and drivers, including facial recognition, location, and clipboard information, totaling billions of records. The violations began in 2015 and continued until the investigation in 2021. Didi was ordered to pay a penalty and correct its practices.

Company involved

滴滴全球股份有限公司 (Didi Global Inc.)

AI system involved

Didi ride-hailing apps

8 source articles · read the reporting →

Clearview AI facial recognition app scrapes billions of images and is used by police

Clearview AI, a secretive start-up, built a facial recognition app that matches photos to a database of more than three billion images scraped from social media and websites. More than 600 law enforcement agencies, including the FBI and Department of Homeland Security, have used the tool to identify suspects in crimes such as shoplifting, identity theft and murder. The company monitored officers who ran a reporter's photo through the app, and its founder acknowledged designing an augmented-reality prototype but said there were no plans to release it. Critics warned the tool could end anonymity and enable misuse.

Company involved

Clearview AI

AI system involved

Clearview AI facial recognition app

2 source articles · read the reporting →

SEC charges YouPlus and CEO with defrauding investors

The SEC charged machine-learning startup YouPlus and its CEO Shaukat Shamim with defrauding investors. Shamim allegedly made false statements about the company's revenue and customer numbers, including providing falsified bank statements. The scheme unravelled when Shamim confessed to investors that the company had earned less than $500,000 and had only four paying customers since 2013. The SEC is seeking permanent injunctions, civil penalties, and an officer-and-director bar.

Company involved

YouPlus

AI system involved

YouPlus machine-learning tool

1 source article · read the reporting →

Privacy International challenges Clearview AI's facial recognition database in Europe

Privacy International filed complaints against Clearview AI with five European data protection authorities in May 2021, alleging that the company's scraping of facial images from the web and building a biometric database without consent violates data protection laws. The regulators in the UK, France, Italy, Greece, and Austria have since found Clearview's practices unlawful, imposed fines, and ordered deletion of data. Clearview has appealed the UK fine, and the case is ongoing.

Company involved

Clearview AI

AI system involved

Clearview

10 source articles · read the reporting →

US Secret Service bought access to cellphone location data

The US Secret Service signed a contract to access Locate X, a service that aggregates location data from phone apps and allows law enforcement to track devices without a warrant. The contract with Babel Street was worth about $36,000 and ran from 2017 to 2018. A former employee said the Secret Service used Locate X in 2018 to seize illegal credit card skimmers at petrol stations. Lawmakers and civil liberties advocates criticised the practice, and Senator Ron Wyden introduced a bill to ban such purchases.

Company involved

United States Secret Service

AI system involved

Locate X

1 source article · read the reporting →

California AG declares out-of-state ALPR data sharing unlawful

California Attorney General Rob Bonta issued guidance confirming that sharing automated license plate reader (ALPR) data with out-of-state or federal agencies violates state law SB 34. The decision follows years of advocacy by EFF and the ACLU, who demonstrated through public records that many California law enforcement agencies were sharing data with hundreds of external agencies, including ICE and CBP. The guidance mandates that all California agencies review their data sharing and cut off access to out-of-state and federal entities.

Company involved

California law enforcement agencies

AI system involved

Automated license plate readers (ALPRs)

1 source article · read the reporting →

Clearview AI settles with ACLU over facial recognition database sales

Clearview AI has agreed to stop selling its facial recognition database to most private US companies as part of a proposed settlement with the ACLU. The company scraped billions of images from social media without consent to build its database, violating Illinois' Biometric Information Privacy Act. The settlement requires Clearview to delete old facial vectors and allow Illinois residents to opt out. The company can still sell its technology to law enforcement and government agencies.

Company involved

Clearview AI

AI system involved

Clearview AI facial recognition database

8 source articles · read the reporting →