Where automated decisions went wrong

NHTSA investigation PE24016: Unexpected ADS behavior

Waymo's automated driving system exhibited unexpected behavior during driving.

Company involved

Waymo

1 source article · read the reporting →

Consumer Reports finds Tesla Autopilot operates without driver

Consumer Reports engineers tested a Tesla Model Y on a closed track and found that the Autopilot system could operate the vehicle without anyone in the driver's seat, failing to warn or detect the absence of a driver. The test highlighted a safety flaw as investigators probed a fatal crash in Texas where two men died in a Tesla Model S believed to be operating without a driver. Tesla did not respond to a request for comment.

Company involved

Tesla

AI system involved

Autopilot

10 source articles · read the reporting →

Tesla Driver Passed Out Drunk on Bay Bridge Claims Autopilot Was in Control

On 19 January 2018, a Tesla driver was found passed out behind the wheel on the San Francisco Bay Bridge with a blood alcohol content over twice the legal limit. The driver told California Highway Patrol officers that the vehicle had been set on Autopilot. The driver was arrested and charged with driving under the influence, and the car was towed. No accident or injuries occurred, and Tesla did not confirm whether Autopilot was engaged.

Company involved

Tesla

AI system involved

Autopilot

10 source articles · read the reporting →

Cruise Self-Driving Car Rear-Ended by Human-Driven Cruise Vehicle in San Francisco

On 11 June 2018, a self-driving Cruise Chevrolet Bolt was rear-ended by another Cruise vehicle driven by a human employee on Bryant Street in San Francisco. The collision caused only minor scuffs and no injuries. Cruise reported the incident to the California DMV as required. The incident highlights the challenges of autonomous vehicles interacting with human drivers.

Company involved

Cruise

2 source articles · read the reporting →

AI-powered tram crashes into pedestrians in St Petersburg test run

An AI-powered 'Smart Tram' on a test run in St Petersburg, Russia, crashed into pedestrians after its AI system turned off and brakes failed. The driver, who was present, attempted to use emergency brakes but they did not fully work. Several people were injured, including a 45-year-old woman trapped under the tram. The city authorities have launched an investigation and a criminal case has been opened.

Company involved

St Petersburg city authorities

AI system involved

Smart Tram

1 source article · read the reporting →

Ola drivers file GDPR lawsuit over algorithmic management and data access

Two ride-hailing drivers are taking Ola to court in Amsterdam, alleging the company violated GDPR data access rights by not providing full personal data, including GPS and ratings information. They argue this hinders their ability to challenge algorithmic decisions, such as a driver who had pay docked after Ola's system incorrectly flagged trips as invalid. The case, supported by the App Drivers & Couriers Union, seeks to compel Ola to comply with data protection law and faces fines for non-compliance. Ola says it has not received notification and complies with all applicable laws.

Company involved

Ola

AI system involved

Guardian

1 source article · read the reporting →

Google and Delphi Self-Driving Cars Involved in Minor Accidents in California

Four self-driving cars in California have been involved in accidents since September 2014, when the state began issuing testing permits. Two occurred while the cars were in autonomous mode, and two while a human was driving. Google and Delphi, the companies operating the vehicles, stated that their cars were not at fault and that the collisions were minor, with no injuries. The accidents have raised concerns about transparency, as details are kept confidential under state law.

Company involved

Google Inc. and Delphi Automotive

10 source articles · read the reporting →

Tesla Model X on Autopilot hits crash attenuator in Mountain View, killing driver

On 23 March 2018, a Tesla Model X P100D operating with Autopilot hit a damaged crash attenuator on US Highway 101 in Mountain View, California, killing the driver. The NTSB found that Autopilot steered the vehicle into a highway gore area because of system limitations, and that the driver was distracted and over-reliant on the partial driving automation. Two other vehicles were struck; one driver sustained minor injuries.

Company involved

Tesla

AI system involved

Autopilot

9 source articles · read the reporting →

Tesla Model Y crash, 1 killed, Autopilot verified: Pedestrian hit on shoulder

A Tesla Model Y struck and killed a pedestrian changing a tire on the shoulder of a highway.

Company involved

Tesla

AI system involved

Autopilot

1 source article · read the reporting →

Waymo drives through farmer’s market in Denver, no one injured - Washington Times

The Waymo autonomous vehicle drove through a farmer's market, potentially endangering attendees.

Company involved

Waymo

AI system involved

Waymo

1 source article · read the reporting →

Beep automated bus rear-ended by Tesla in Washington D.C.

An automated bus operated by Beep was rear-ended by a Tesla driver making an illegal lane change in Washington D.C. on 18 January 2026. The bus, part of a pilot programme with the U.S. Department of Transportation, had a human backup driver and sustained minor cosmetic damage. Service was temporarily paused but resumed after a review found the bus operated appropriately. No police were called and the incident was not reported to NHTSA as the damage was under $1,000 and the other driver was at fault.

Company involved

Beep

AI system involved

Beep automated bus

2 source articles · read the reporting →

Waymo pulls over, calls cops on riders with a ghost gun - The Verge

The vehicle pulled over and alerted police to riders in possession of a firearm

Company involved

Waymo

AI system involved

Waymo robotaxi

1 source article · read the reporting →

San Francisco officials complain of robotaxi interference with emergency responders

San Francisco's fire chief and police union have complained that Waymo and Cruise robotaxis are interfering with emergency responders, including driving into active scenes and blocking fire trucks. The fire chief says it is only a matter of time before a catastrophic injury or death occurs. The police union asked the California Public Utilities Commission to postpone a vote on expanding robotaxi services. The companies say they are working with officials and that most encounters are without issue.

Company involved

Waymo and Cruise

4 source articles · read the reporting →

Waymo Self-Driving Taxis Crash into Backwards-Towed Pickup in Phoenix

On 11 December 2023, two Waymo autonomous vehicles collided with a pickup truck being towed backwards in Phoenix, Arizona. The vehicles were operating without passengers and sustained minor damage, with no injuries reported. Waymo attributed the crashes to a software error that mispredicted the towed vehicle's motion and subsequently deployed a fix. The company filed a voluntary recall report with the NHTSA.

Company involved

Waymo

AI system involved

Waymo Driver

3 source articles · read the reporting →

VIDEO: Wayward Waymo runs red light at major intersection near Universal Orlando - WKMG

The Waymo van entered an intersection against a red light, endangering cross traffic and pedestrians.

Company involved

Waymo

1 source article · read the reporting →

Tesla driver charged with vehicular homicide after autopilot crash kills motorcyclist

A Tesla driver using autopilot mode was charged with vehicular homicide after his vehicle struck and killed a motorcyclist in Washington. The driver told police the car lurched forward and accelerated into the motorcycle. The motorcyclist, 28-year-old Jeff Nissen, died at the scene. The driver was using his phone at the time and was charged with vehicular homicide.

Company involved

Tesla

AI system involved

Tesla Autopilot (Full Self-Driving Supervised)

5 source articles · read the reporting →

Austin Fire Department Faces Another Bizarre Standoff With an Empty Waymo - MotorBiscuit

A driverless car failed to yield to an emergency vehicle, blocking its path

Company involved

Waymo

AI system involved

Waymo

1 source article · read the reporting →

A Waymo Drove Into a Denver Farmers Market. Colorado Police Had No Mechanism to Ticket It. - Gadget Review

A driverless Waymo entered a Denver farmers market past barricades and a stop sign, disrupting vendors and staff.

Company involved

Waymo

AI system involved

Waymo

1 source article · read the reporting →

Tesla in self-drive mode crashes into Fullerton police car

A Tesla operating in self-drive mode crashed into a parked Fullerton police car while the officer was outside directing traffic at a fatal crash scene. The officer jumped out of the way and was not injured, but the patrol vehicle sustained major damage. The driver admitted to using a cellphone at the time. It is unclear if the driver will be cited.

AI system involved

Tesla Full Self-Driving (Supervised)

1 source article · read the reporting →

California Slaps Robotaxis With $10,000 Fines for Blocking First Responders - Hoodline

Autonomous vehicles blocked emergency vehicles, delaying response times for first responders.

Company involved

Waymo

AI system involved

Waymo

1 source article · read the reporting →

Waymo Robotaxi Blocks Fire Truck Responding To Austin Hotel Alarm - AOL.com

An empty Waymo robotaxi blocked and delayed a fire truck responding to an Austin hotel alarm.

Company involved

Waymo

AI system involved

Waymo robotaxi

1 source article · read the reporting →

Buena Vista Crash Killed 82-Year-Old — Tesla Records Show Driver-Assist Was On - Hoodline

The Tesla driver-assist system ran a stop sign and struck a Honda Civic, killing its driver, Stephen Field.

Company involved

Tesla

1 source article · read the reporting →

Tesla Full Self-Driving involved in fatal pedestrian crash in Arizona

In November 2023, a Tesla Model Y with Full Self-Driving engaged struck and killed 71-year-old Johna Story on Interstate 17 in Arizona. The system failed to brake due to sun glare, according to footage obtained by Bloomberg. The victim's family has sued Tesla and the driver. NHTSA is investigating whether the system poses an unacceptable safety risk.

Company involved

Tesla

AI system involved

Full Self-Driving

7 source articles · read the reporting →

Waymo Self-Driving Taxis Proactively Report to Police After Discovering Passenger Hiding a Gun

قامت سيارات الأجرة ذاتية القيادة من شركة Waymo بإبلاغ الشرطة بشكل استباقي بعد اكتشافها أن أحد الركاب كان يخفي مسدساً.…

A Waymo self-driving taxi in San Francisco detected two teenage passengers carrying an unidentified AR-15 rifle loaded with ammunition. The vehicle automatically stopped and alerted police, leading to the arrest of the two teenagers. Waymo stated that its system can identify violations of its terms of service, including carrying weapons, and will proactively notify police.

Company involved

Waymo

1 source article · read the reporting →