Where automated decisions went wrong

ZTO Express unmanned vehicle drags fallen e-bike in Shaanxi

On 8 April 2025, a ZTO Express unmanned delivery vehicle operated by Jiushi Intelligent Technology in Xianyang, Shaanxi, failed to detect a fallen electric bicycle and dragged it along the road. The vehicle's safety system triggered braking after contact but then resumed speed. No injuries were reported, and the company suspended the vehicle, apologised, and is cooperating with traffic police investigation.

Company involved

ZTO Express

AI system involved

九识无人车

4 source articles · read the reporting →

Uber fined €825 million for suspending drivers through algorithms without human review - MediaNama

Uber fined €825 million for suspending drivers through algorithms without human review MediaNama

1 source article · read the reporting →

Uber set for €825mn Dutch fine over automating driver suspensions - Financial Times

The system automatically suspended Uber drivers.

Company involved

Uber

1 source article · read the reporting →

Workers Look To Lock In Sweeping Workday AI Bias Class - Law360

The system screened job applicants and rejected them, allegedly discriminating based on age and sex.

AI system involved

Workday AI tools

1 source article · read the reporting →

Workday Gains an Edge in AI Hiring Bias Fight: California Brief - Bloomberg Law News

Workday Gains an Edge in AI Hiring Bias Fight: California Brief - Bloomberg Law News

1 source article · read the reporting →

Meta Agrees to Record KES 2.3 Trillion Settlement Over Addictive Teenage Social Media Design - streamlinefeed.co.ke

The algorithmic content recommendation system determined what content to show to teenage users, influencing their engagement and screen time.

Company involved

Meta Platforms

AI system involved

Facebook and Instagram

1 source article · read the reporting →

AI Lawsuit Pushes the Boundaries of AI Litigation—and May Signal a New Wave

Ranked job applicants' likelihood of success, affecting their hiring prospects.

Company involved

Eightfold AI Inc.

AI system involved

Eightfold AI Hiring Platform

1 source article · read the reporting →

Lawsuit claims discrimination by Workday’s hiring tech prevented people over 40 from getting hired | CNN Business

The system screened and ranked job applicants, leading to rejections for individuals over 40.

Company involved

Workday

AI system involved

HiredScore AI

1 source article · read the reporting →

Uber Drivers Sue Over ‘Scary’ AI Pay Algorithm Across Europe - Memeburn

The system determined driver pay and trip allocation for Uber drivers.

Company involved

Uber

AI system involved

Dynamic Pay

1 source article · read the reporting →

Il monte dans un Tesla Cybercab pour un trajet de 10 minutes et n’arrive que 70 minutes plus tard -…

The Cybercab selected an inefficient route through surface streets, turning a planned 10-minute trip into a 70-minute journey for the passenger.

Company involved

Tesla

AI system involved

Tesla Cybercab

1 source article · read the reporting →

Meta Scraped User Photos for Secret Smart-Glasses Tech, Class Action Claims - The SOFX Report

Meta harvested user photographs and created biometric faceprints to enable a smart-glasses system that could identify strangers in public without consent.

Company involved

Meta Platforms, Inc.

AI system involved

NameTag

1 source article · read the reporting →

Meta Agrees to Alter Ad Technology in Settlement With U.S. (Published 2022) - The New York Times

The ad delivery system allegedly discriminated in housing ad delivery on the basis of protected characteristics.

Company involved

Meta

AI system involved

ad technology

1 source article · read the reporting →

Starship delivery robot collides with car in Frisco, driver compensated

Jisuk Mok's car was hit by a Starship food delivery robot while she was stopped at a light in Frisco, Texas, causing over $2,600 in damage. She struggled to get insurance information and contact Starship, which initially refused to release video of the incident. After NBC 5 Responds intervened, Starship acknowledged the incident and agreed to pay for the damages. The company stated such incidents are rare, with robots crossing 15,000 roads daily without issue.

Company involved

Starship Technologies

AI system involved

Starship

2 source articles · read the reporting →

Court orders suspension of facial recognition in São Paulo metro

A court in São Paulo ordered the suspension of a facial recognition system in the city's metro stations following a lawsuit by civil society groups. The system, SecurOS by ISS and operated by ViaQuatro, was capturing biometric data of millions of daily users without adequate transparency or risk assessment. The court also barred the metro operator, METRO, from installing new biometric equipment and imposed daily fines for non-compliance. METRO stated it would appeal the ruling and prove compliance with data protection regulations.

Company involved

Companhia do Metropolitano de São Paulo (METRO)

AI system involved

SecurOS

3 source articles · read the reporting →

FACIL'iti sues accessibility consultant Julie Moynat over critical tweet

In November 2020, Julie Moynat, a web accessibility consultant, tweeted criticism of FACIL'iti, an accessibility product. FACIL'iti sent a formal notice to her employer and later subpoenaed her for denigration in May 2021. Moynat is raising funds for her legal defense. The case is ongoing at the Judiciary Tribunal of Paris.

Company involved

FACIL'iti

AI system involved

FACIL'iti

10 source articles · read the reporting →

Xsolla fires 150 employees after big data analysis tags them as unproductive

Payment services company Xsolla terminated 150 employees at its Perm, Russia office after a big data analysis of their activity in Jira, Confluence, Gmail, chats, and documents tagged them as unengaged and unproductive. CEO Aleksandr Agapitov sent an email to affected employees stating that Xsolla was not for them, sparking backlash. The company later held a press conference, explaining the layoffs were due to slowing growth, and offered affected employees medical insurance and severance pay equal to four to six months' salary.

Company involved

Xsolla

4 source articles · read the reporting →

Amazon tracked every minute of JFK8 warehouse workers' shifts, documents show

Internal documents show Amazon's Time Off Task tool tracked every minute of JFK8 warehouse workers' shifts using handheld scanners and identified a 'top offender' per shift. Managers questioned those workers about each period of inactivity, including bathroom breaks, and issued warnings or fired them for 30 to 120 minutes of time off task. The article reports that 18 workers were terminated between January and February 2020, and workers said they skipped water and bathroom breaks for fear of discipline. Amazon did not respond to a request for comment.

Company involved

Amazon

AI system involved

Time Off Task Tool

3 source articles · read the reporting →

Kronos Algorithmic Scheduling Blamed for Erratic Work Hours for Retail Workers

Kronos workforce management software is used by major retail chains to algorithmically generate employee schedules. The software has been criticised for causing erratic and unpredictable work hours, leading to financial instability and emotional distress for low-wage workers. Following an investigation by the New York attorney general and negative press, Kronos announced new features aimed at improving schedule stability and worker control, though critics argue the changes are optional and lack enforcement mechanisms.

Company involved

Various retailers (e.g., Gap, Target, J.C. Penney, Starbucks)

AI system involved

Kronos scheduling software

9 source articles · read the reporting →

Deliveroo's rider-ranking algorithm ruled discriminatory by Italian court

An Italian court in Bologna ruled that Deliveroo's reputational-ranking algorithm discriminated against delivery riders. The algorithm failed to distinguish between legally protected reasons for not working, such as sickness or strike action, and other reasons. The court ordered Deliveroo to pay €50,000 to the applicants and publish the ruling. Deliveroo stated that the shift-reservation system linked to the algorithm is no longer in use and disagreed with the decision.

Company involved

Deliveroo

2 source articles · read the reporting →

Suresnes allows startup XXII to use CCTV for experimental surveillance algorithms

The city of Suresnes has allowed startup XXII to use its public CCTV cameras for 18 months to develop algorithms for detecting suspicious behavior. The algorithms are experimental and may have high error rates. Residents were not informed or consulted. The startup will own the data and can use the city's surveillance center as a showroom for clients.

Company involved

Mairie de Suresnes

AI system involved

XXIISmartCity

10 source articles · read the reporting →

国内ベンチャー企業A社、AIなりすまし面談で実在エンジニアになりすまされる

A domestic Japanese logistics SaaS venture, referred to as Company A, conducted an online casual interview for an engineer role on 4 March 2026. An applicant used AI-generated video to impersonate a real engineer, Yoshii Kenbun, whose CV details had been submitted without his knowledge. The interviewer noticed discrepancies in language ability and unnatural video, and the company confirmed with Yoshii that he had not applied or attended. Company A and Yoshii publicised the incident on X the next day.

3 source articles · read the reporting →

France: CNAF's discriminatory risk-scoring algorithm must be stopped

Amnesty International and coalition partners filed a complaint with the Council of State against CNAF's risk-scoring algorithm used to detect benefit overpayments. The algorithm assigns risk scores based on criteria that discriminate against vulnerable groups, including those with disabilities, single parents, and low-income households. The complaint alleges the system violates human rights to equality and privacy. The EU AI Act's social scoring ban may apply, but its definition remains unclear.

Company involved

CNAF

2 source articles · read the reporting →

AMS algorithm lacks transparency and may discriminate against job seekers

The Austrian Public Employment Service (AMS) uses an algorithm to classify job seekers into categories A, B, and C, determining their access to benefits and training. Scientists from TU Wien, WU Wien, and University of Vienna have criticised the algorithm for lacking transparency, as only two of 96 model variants have been published. They allege that the system may discriminate against women and people with migration background, and that job seekers are not informed about how the algorithm works or given a chance to appeal.

Company involved

AMS (Arbeitsmarktservice Österreich)

AI system involved

AMS-Algorithmus

10 source articles · read the reporting →

Shipt's new V2 algorithm reduces pay for 41% of workers, study finds

A study by MIT Media Lab and Coworker.org found that Shipt's new V2 payment algorithm reduced pay for 41% of workers by an average of 11% per shop. The algorithm, rolled out in September 2020, replaced a transparent payment system with a black-box model. Workers pooled data to analyze the impact, revealing uneven distribution of earnings.

Company involved

Shipt

AI system involved

V2

10 source articles · read the reporting →