Where automated decisions went wrong

NHTSA investigation PE24016: Unexpected ADS behavior

Waymo's automated driving system exhibited unexpected behavior during driving.

Company involved

Waymo

1 source article · read the reporting →

Tencent launches Zero-Point Cruise facial recognition to enforce night-time game curfew

Tencent has introduced a feature called Zero-Point Cruise in its games, which subjects accounts registered as adults that play at night beyond a set time to facial recognition. Anyone who refuses or fails the verification is treated as a minor and logged out. Tencent says this is intended to stop children using adult identities to evade the game curfew, and that adults who mistakenly refuse can wait for the next authentication.

Company involved

Tencent

AI system involved

零点巡航 (Zero-Point Cruise)

10 source articles · read the reporting →

Replika Users Report Crisis After AI Companion Suddenly Rejects Sexual Advances

Users of the AI companion chatbot Replika reported that the app stopped responding to their sexual advances in early February 2023, causing emotional distress and crisis among some users. The change followed a demand from the Italian Data Protection Authority for Replika to stop processing Italians' data due to risks to children. Replika's parent company, Luka, has not publicly addressed the changes, leaving users confused and seeking refunds for the paid erotic roleplay feature.

Company involved

Luka

AI system involved

Replika

1 source article · read the reporting →

Lovable security flaw exposed user data from 170 apps

Lovable, a Swedish startup, failed to fix a critical security flaw in its vibe coding service. Researchers found 170 Lovable-created web apps that exposed users' personal data, including names, emails, financial information, and API keys. Lovable acknowledged the issue and implemented a security scan, but the flaw remained unresolved.

Company involved

Lovable

AI system involved

Lovable

5 source articles · read the reporting →

Starship delivery robot destroyed by train at Oregon State University

An autonomous Starship food delivery robot was struck and destroyed by a freight train at a railroad crossing on the Oregon State University campus. The robot reportedly became stranded on the tracks, a recurring issue for these delivery robots at this location. No injuries were reported. The incident was captured on video and shared on social media.

Company involved

Starship Technologies

AI system involved

Starship delivery robot

2 source articles · read the reporting →

Starship delivery robot drives into planter and down stairs at UCLA

A Starship Technologies delivery robot operating autonomously at UCLA fell into a planter after its wheels slipped off a path edge. Bystanders pulled it free, but the robot turned back towards the planter before being pushed clear. A remote human operator then took control, but the robot subsequently drove down a flight of stairs and later struck a scooter due to a sensor blind spot.

Company involved

Starship Technologies

2 source articles · read the reporting →

Ola drivers file GDPR lawsuit over algorithmic management and data access

Two ride-hailing drivers are taking Ola to court in Amsterdam, alleging the company violated GDPR data access rights by not providing full personal data, including GPS and ratings information. They argue this hinders their ability to challenge algorithmic decisions, such as a driver who had pay docked after Ola's system incorrectly flagged trips as invalid. The case, supported by the App Drivers & Couriers Union, seeks to compel Ola to comply with data protection law and faces fines for non-compliance. Ola says it has not received notification and complies with all applicable laws.

Company involved

Ola

AI system involved

Guardian

1 source article · read the reporting →

Tesla Model X on Autopilot hits crash attenuator in Mountain View, killing driver

On 23 March 2018, a Tesla Model X P100D operating with Autopilot hit a damaged crash attenuator on US Highway 101 in Mountain View, California, killing the driver. The NTSB found that Autopilot steered the vehicle into a highway gore area because of system limitations, and that the driver was distracted and over-reliant on the partial driving automation. Two other vehicles were struck; one driver sustained minor injuries.

Company involved

Tesla

AI system involved

Autopilot

9 source articles · read the reporting →

Grab's surge pricing algorithm always adds fees, investigation finds

An investigation by the Pulitzer Center found that Grab's ride-hailing app in the Philippines always includes surge fees in its fares, even when wait times are long. Customers like Rica Torres reported rising costs without explanation. A complaint filed with the LTFRB in 2022 remains unresolved, and the regulator has not determined how the algorithm sets surge rates.

Company involved

Grab

AI system involved

Grab app

8 source articles · read the reporting →

Waymo drives through farmer’s market in Denver, no one injured - Washington Times

The Waymo autonomous vehicle drove through a farmer's market, potentially endangering attendees.

Company involved

Waymo

AI system involved

Waymo

1 source article · read the reporting →

Uber apologises for surge pricing during Sydney siege

During the 2014 Sydney Martin Place siege, Uber's surge pricing algorithm automatically raised fares by up to four times the normal rate as people tried to leave the area. The company faced heavy criticism on social media and later apologised, refunding affected customers and offering free rides. Uber acknowledged that not stopping surge pricing immediately was the wrong decision.

Company involved

Uber

7 source articles · read the reporting →

Beep automated bus rear-ended by Tesla in Washington D.C.

An automated bus operated by Beep was rear-ended by a Tesla driver making an illegal lane change in Washington D.C. on 18 January 2026. The bus, part of a pilot programme with the U.S. Department of Transportation, had a human backup driver and sustained minor cosmetic damage. Service was temporarily paused but resumed after a review found the bus operated appropriately. No police were called and the incident was not reported to NHTSA as the damage was under $1,000 and the other driver was at fault.

Company involved

Beep

AI system involved

Beep automated bus

2 source articles · read the reporting →

San Francisco officials complain of robotaxi interference with emergency responders

San Francisco's fire chief and police union have complained that Waymo and Cruise robotaxis are interfering with emergency responders, including driving into active scenes and blocking fire trucks. The fire chief says it is only a matter of time before a catastrophic injury or death occurs. The police union asked the California Public Utilities Commission to postpone a vote on expanding robotaxi services. The companies say they are working with officials and that most encounters are without issue.

Company involved

Waymo and Cruise

4 source articles · read the reporting →

Waymo Self-Driving Taxis Crash into Backwards-Towed Pickup in Phoenix

On 11 December 2023, two Waymo autonomous vehicles collided with a pickup truck being towed backwards in Phoenix, Arizona. The vehicles were operating without passengers and sustained minor damage, with no injuries reported. Waymo attributed the crashes to a software error that mispredicted the towed vehicle's motion and subsequently deployed a fix. The company filed a voluntary recall report with the NHTSA.

Company involved

Waymo

AI system involved

Waymo Driver

3 source articles · read the reporting →

Xiaomi SU7 fatal crash in Chengdu raises EV safety concerns

A Xiaomi SU7 electric vehicle crashed in Chengdu, China, and caught fire. Bystanders were unable to open the electronic door handles to rescue an occupant, and a 31-year-old male driver died. The incident follows an earlier fatal SU7 crash that raised questions about the vehicle's smart driving features, and has intensified scrutiny on electronic door handle safety.

Company involved

Xiaomi

AI system involved

SU7

3 source articles · read the reporting →

Ring's AI pet-search feature draws privacy backlash after Super Bowl ad

Ring, an Amazon company, promoted its Search Party feature in a Super Bowl advertisement, saying its AI helps reunite dog owners with missing pets. Critics said the feature, which is enabled by default, turns Ring doorbells into a mass surveillance network and could easily be adapted to track people. Ring said it had reunited 99 dogs in the US in 90 days and that customers keep full control of their data.

Company involved

Ring

AI system involved

Search Party

5 source articles · read the reporting →

Google's Waze App Blocks Wildfire Escape Route in Los Gatos, Residents Say

Residents of historic neighborhoods in Los Gatos, California, allege that Google's Waze navigation app is routing heavy beach traffic through their streets, blocking the only wildfire evacuation route. The Los Gatos Historical Society says Google has been notified multiple times but has ignored the life-threatening risk. A U.S. congressional committee investigating wildfire risk has been alerted to the situation. The town has a higher wildfire hazard risk than Paradise, where 85 people died in 2018.

Company involved

Google

AI system involved

Waze

2 source articles · read the reporting →

Tesla Full Self-Driving involved in fatal pedestrian crash in Arizona

In November 2023, a Tesla Model Y with Full Self-Driving engaged struck and killed 71-year-old Johna Story on Interstate 17 in Arizona. The system failed to brake due to sun glare, according to footage obtained by Bloomberg. The victim's family has sued Tesla and the driver. NHTSA is investigating whether the system poses an unacceptable safety risk.

Company involved

Tesla

AI system involved

Full Self-Driving

7 source articles · read the reporting →

Waymo Self-Driving Taxis Proactively Report to Police After Discovering Passenger Hiding a Gun

قامت سيارات الأجرة ذاتية القيادة من شركة Waymo بإبلاغ الشرطة بشكل استباقي بعد اكتشافها أن أحد الركاب كان يخفي مسدساً.…

A Waymo self-driving taxi in San Francisco detected two teenage passengers carrying an unidentified AR-15 rifle loaded with ammunition. The vehicle automatically stopped and alerted police, leading to the arrest of the two teenagers. Waymo stated that its system can identify violations of its terms of service, including carrying weapons, and will proactively notify police.

Company involved

Waymo

1 source article · read the reporting →

Oakland County investigating alleged misuse of Flock camera data by former deputy - WXYZ 7 News Detroit

The Flock camera system collected and stored license plate data, and a former deputy allegedly accessed it without authorization.

Company involved

Oakland County Sheriff's Office

AI system involved

Flock camera

1 source article · read the reporting →

NHTSA investigation PE24015: Rear-end Collisions Involving Zoox Vehicles

Automated driving system controlled vehicle braking and acceleration, leading to rear-end collisions with other road users

Company involved

Zoox

AI system involved

Zoox vehicle

1 source article · read the reporting →

First Accident Before Test Phase: Robotaxis Launch in Furttal

Erster Unfall, bevor die Testphase beginnt: Robotaxis starten im Furttal - nzz.ch

Switzerland's first self-driving taxi service launched in the Furttal region near Zurich, but three days before the start a test vehicle had a minor accident in Dänikon, damaging a ground plate and two tires. The pilot project, called Iamo, aims to complement public transport, with a safety driver on board during the test phase.

Company involved

Swiss Transit Lab

1 source article · read the reporting →

Waymo autonomous vehicle allegedly kills beloved San Francisco bodega cat KitKat

A Waymo autonomous vehicle allegedly struck and killed KitKat, a 9-year-old cat that was a beloved mascot of Randa's Market in San Francisco's Mission District, on the night of October 27, 2025. Witnesses reported that the vehicle swerved and did not stop after hitting the cat, which later died at a veterinary hospital. The incident has sparked mourning among neighbors and calls for accountability, with a 311 complaint filed alleging the car 'did not even try to stop.' Waymo did not respond to requests for comment.

Company involved

Waymo

AI system involved

Waymo

7 source articles · read the reporting →

Ezemvelo KZN Wildlife's AI cameras fail to detect poachers, four rhinos killed

Four dehorned rhinos were found killed at Hluhluwe-iMfolozi Park after AI-powered infra-red cameras failed to detect the poachers. The cameras, which are designed to identify people and alert the operations centre, did not pick up the intruders. Ezemvelo KZN Wildlife spokesperson Musa Mntambo explained that the park is vast and the cameras cannot monitor every area simultaneously. The incident has raised concerns about the effectiveness of the technology in preventing rhino poaching.

Company involved

Ezemvelo KZN Wildlife

1 source article · read the reporting →