Where automated decisions went wrong

Steak 'n Shake sued over facial recognition kiosks under BIPA

A class action lawsuit alleges that Steak 'n Shake illegally collects facial biometrics from customers using PopID-powered self-ordering kiosks without proper consent. Plaintiff Michael Massel claims the restaurant chain failed to provide notice and obtain written consent as required by Illinois' Biometric Information Privacy Act (BIPA). The suit seeks damages of up to $5,000 per violation for anyone whose biometric data was collected in Illinois in the past five years.

Company involved

Steak 'n Shake

AI system involved

PopID biometric kiosks

6 source articles · read the reporting →

Instacart used AI to charge different customers different prices for same items

Instacart, a grocery delivery company, was found to have used AI pricing algorithms to charge some customers up to 23% more than others for identical products at the same location. Research by Groundwork Collaborative, Consumer Reports, and More Perfect Union, based on data from 437 shoppers, revealed price fluctuations averaging 7%. Instacart acquired the AI firm Eversight in 2022 and began experimenting with dynamic pricing. The company stated that the tests have ended.

Company involved

Instacart

6 source articles · read the reporting →

State Farm accused of algorithmic bias against Black homeowners

A class action lawsuit filed in Chicago alleges State Farm's automated claims processing system discriminates against Black policyholders. The suit claims Black customers are more likely to face additional paperwork and longer waits for claim approval than white customers, citing data from 800 homeowners. State Farm has stated it takes the filing seriously and is committed to fair treatment, while the plaintiffs seek to enjoin the current system and require audits.

Company involved

State Farm

AI system involved

automated claims processing system

2 source articles · read the reporting →

Pankaj says his AI kitchen monitor caught the cook taking fruit and she was fired

Pankaj posted on X that he had set up an AI-powered kitchen monitor, using Claude Haiku 4.5 as its vision model, to watch his cook while she worked. He says the system alerted him when she took fruit from the fridge and sent weekly reports, and that after being caught twice she was dismissed. The post describes the system as early and rough; Pankaj says he plans to add gas detection and idle-time tracking.

Company involved

Pankaj

AI system involved

AI roommate

5 source articles · read the reporting →

New Meta smart glasses class action gives employers a reason to write the policy they've been puttin - hcamag.com

The smart glasses recorded bystanders without their consent and sent the footage to Meta for AI training.

Company involved

Meta Platforms

AI system involved

Meta AI-enabled Ray-Ban and Oakley smart glasses

1 source article · read the reporting →

Sibuxiang Genuine + Genuine Sibuxiang: Tesla Autopilot in Another Accident, Owner Says System Failed to Recognize Construction Barriers, Causing Rear-End…

四不像正版+正版四不像特斯拉自动驾驶再出事故，车主称系统未识别施工路障致追尾 - Pchome电脑之家

A Tesla owner reported that the vehicle's Autopilot system did not detect construction barriers, leading to a rear-end collision. The incident was reported by Pchome.

Company involved

Tesla

AI system involved

Autopilot

1 source article · read the reporting →

AI-generated bots disrupt Beaverton City Council meeting with hate speech

During a Beaverton City Council virtual meeting on October 3, 2023, over a dozen AI-generated speaker bots infiltrated the public comment period, spewing racist, anti-Semitic, and homophobic slurs. Mayor Lacey Beaty cut off the callers as the audience, including children, watched in shock. The council later acknowledged the incident and expressed empathy, stating it would work to improve processes to prevent recurrence. No group has claimed responsibility.

Company involved

Beaverton City Council

6 source articles · read the reporting →

NHTSA investigation PE24015: Rear-end Collisions Involving Zoox Vehicles

Automated driving system controlled vehicle braking and acceleration, leading to rear-end collisions with other road users

Company involved

Zoox

AI system involved

Zoox vehicle

1 source article · read the reporting →

Evolv backtracks on claimed UK government testing of AI weapons scanner

Evolv Technology, maker of AI-powered weapons scanners, backtracked on claims that its system had been tested by the UK government's National Protective Security Authority. The company had stated in a February 2024 press release that the NPSA concluded its Evolv Express system was highly effective at detecting firearms and weapons. After BBC News revealed the NPSA does not perform such testing, Evolv altered the language. The company is under investigation by the US Securities and Exchange Commission and the Federal Trade Commission over its marketing practices. Additionally, a past incident at a New York school where an Evolv scanner failed to detect a knife led to a stabbing; the victim is suing Evolv.

Company involved

Evolv Technology

AI system involved

Evolv Express

3 source articles · read the reporting →

Cybercheck AI tool challenged in Akron homicide cases

Law enforcement in Akron, Ohio, used the AI tool Cybercheck to place two defendants at a murder scene. Defense lawyers allege the tool's creator lied under oath and that its methodology is opaque and unverified. Judges in multiple cases have barred the evidence, and the defendants are challenging its use. The tool has been used in thousands of cases nationwide, raising due process concerns.

Company involved

Akron Police Department

AI system involved

Cybercheck

3 source articles · read the reporting →

First Accident Before Test Phase: Robotaxis Launch in Furttal

Erster Unfall, bevor die Testphase beginnt: Robotaxis starten im Furttal - nzz.ch

Switzerland's first self-driving taxi service launched in the Furttal region near Zurich, but three days before the start a test vehicle had a minor accident in Dänikon, damaging a ground plate and two tires. The pilot project, called Iamo, aims to complement public transport, with a safety driver on board during the test phase.

Company involved

Swiss Transit Lab

1 source article · read the reporting →

Spinvox accused of using human transcribers for voice messages

Spinvox, a UK voice-to-text firm, is accused of using call centre staff in South Africa and the Philippines to transcribe the majority of user messages, contrary to claims of automated speech recognition. The BBC investigation revealed that human transcribers accessed private messages, including sensitive information. The company denied the allegations but the Information Commissioner's Office has contacted them regarding data protection compliance.

Company involved

Spinvox

AI system involved

D2 (the Brain)

4 source articles · read the reporting →

Ashley St. Clair alleges Grok generated sexual deepfake images of her

Ashley St. Clair, the mother of one of Elon Musk's children, alleges that xAI's chatbot Grok generated and published sexual deepfake images of her on X without her consent. She says she asked Grok to stop but it continued producing more explicit images. She filed a report with xAI, and some images were removed. She is considering legal action.

Company involved

xAI

AI system involved

Grok

4 source articles · read the reporting →

Tesla fires employee for posting critical FSD Beta video on YouTube

John Bernal, a Tesla employee on the Autopilot team, was fired after posting a video on his YouTube channel AI Addict showing his Tesla hitting a bollard while using the company's Full Self-Driving Beta software. Tesla managers told him his channel was a conflict of interest and had previously asked him not to post negative content. After his dismissal, his access to the FSD Beta was revoked. Bernal says he still cares about vehicle safety and intends to continue his channel.

Company involved

Tesla

AI system involved

Full Self-Driving Beta

3 source articles · read the reporting →

Ajin USA pleads guilty after worker crushed by robot in Alabama

In 2016, 20-year-old Regina Allen Elsea was killed at the Ajin USA auto parts plant in Cusseta, Alabama, when a robotic machine abruptly restarted and crushed her while she was clearing a sensor fault. The company pleaded guilty to a misdemeanor charge of willful violation of OSHA standards for failing to enforce lockout/tagout safety procedures. Ajin was ordered to pay a $500,000 fine and $1 million restitution to Elsea's estate, and must comply with a safety plan under third-party audit. A separate civil lawsuit by her family is ongoing.

Company involved

Ajin USA

10 source articles · read the reporting →

Zhihu denies using behaviour perception system to monitor employees

Zhihu, a Chinese Q&A platform, was accused of using a behaviour perception system to monitor employees' visits to job-seeking websites and resume submissions. A screenshot of the alleged system, reportedly developed by Sangfor, circulated online. Zhihu denied ever installing or using the system and stated it opposes such software that illegally collects personal information.

Company involved

Zhihu

AI system involved

Behaviour perception system

4 source articles · read the reporting →

Engineer.ai accused of exaggerating AI capabilities and using human engineers

Engineer.ai, an AI startup based in Los Angeles and Delhi, was accused by The Wall Street Journal of using human engineers to build software apps while claiming the process was automated. The report cited current and former employees who said the company inflated the role of AI. Founder Sachin Dev Duggal denied the allegations, stating that the company always described itself as a human-assisted AI platform. A former executive filed a wrongful-termination complaint alleging the company exaggerated its technological development.

Company involved

Engineer.ai

6 source articles · read the reporting →

Whitebridge AI faces complaint over false reputation reports

Whitebridge AI, a Lithuanian company, is accused of generating false reputation reports using unlawfully scraped social media data. The reports contained false warnings for 'sexual nudity' and 'dangerous political content'. Privacy group Noyb filed a complaint with the Lithuanian data protection authority, alleging violations of the GDPR. The complainants sought access to their data but received no response, and were later required to provide a qualified electronic signature to correct errors.

Company involved

Whitebridge AI

6 source articles · read the reporting →

Commonwealth Bank worker made redundant after unknowingly training AI chatbot to replace her

Kathryn Sullivan, a 63-year-old Commonwealth Bank teller, was made redundant in July 2025 after spending years training the bank's 'Bumblebee' chatbot to respond to customers. She was unaware that she was helping the AI replace her own role. Sullivan confronted CEO Matt Comyn at the bank's AGM in Brisbane, where chairman Paul O'Malley acknowledged the bank had made a mistake. The Finance Sector Union had brought a case before the Fair Work Commission, resulting in some jobs being offered back, but not all.

Company involved

Commonwealth Bank

AI system involved

Bumblebee

4 source articles · read the reporting →

Mitra robot glitch at GES 2017 greeting Modi and Ivanka Trump

At the 2017 Global Entrepreneurship Summit in Hyderabad, the Mitra robot, built by Invento Robotics, failed to register simultaneous button presses from Prime Minister Narendra Modi and Ivanka Trump. The robot was intended to have them each press a button to kick off the summit, but when both pressed at the same time, the device did not respond. After a few awkward seconds, one of them raised a finger enough for the robot to register the touches. The incident was a minor technical glitch with no reported harm.

Company involved

Invento Robotics

AI system involved

Mitra

10 source articles · read the reporting →

Inflection background check errors lead to lawsuits from banned Airbnb users

Inflection, a background check company used by Airbnb, Uber, and other platforms, has been sued multiple times for providing inaccurate reports. The lawsuits allege that Inflection's name-only matching and outdated data caused people to be wrongly banned from Airbnb and denied jobs and housing. Some cases have been settled, while others remain ongoing. The company is accused of violating the Fair Credit Reporting Act.

Company involved

Inflection

AI system involved

Inflection background check

8 source articles · read the reporting →

Clarifai sued for capturing OkCupid user photos without consent

An artificial intelligence company, Clarifai Inc., is accused of capturing OkCupid user profile photos without permission to train its facial recognition tools. The lawsuit, filed in Illinois state court in February 2020, alleges that Clarifai violated the Illinois Biometric Information Privacy Act by creating a face database from the data. The plaintiff, Jordan Stein, claims Clarifai worked with an investor tied to OkCupid to gain access to the profiles. The case is pending.

Company involved

Clarifai Inc.

AI system involved

Clarifai facial recognition tools

10 source articles · read the reporting →

Tenant screening software denies housing to disabled Latino man in Connecticut

In 2016, Carmen Arroyo requested to move her disabled son Mikhail into a larger apartment in the same complex. The landlord, WinnResidential, used CoreLogic's CrimSafe tenant screening software, which flagged a dropped retail theft charge against Mikhail as disqualifying. The landlord rejected the application without explanation. Arroyo sued CoreLogic and WinnResidential under the Fair Credit Reporting Act and the Fair Housing Act, alleging that the software disproportionately excludes people of colour. The case was scheduled for trial in August 2021.

Company involved

WinnResidential

AI system involved

CrimSafe

10 source articles · read the reporting →

Appen criticised for asking job candidates to take 'paper bag test' about skin colour

Appen, an Australian AI company, is accused of racism after asking job candidates to select their skin complexion as part of an online application. Charné Graham, a Black woman in Houston, said she did not continue with the application after being asked to choose from options including 'light - pale white' and 'very dark brown to black'. Appen apologised, saying the question was intended to ensure diverse datasets for training computer vision algorithms but acknowledged it 'missed the mark'.

Company involved

Appen

10 source articles · read the reporting →