Where automated decisions went wrong

Lawsuit says Tesla Autopilot missed a curve before a fatal Model X crash

The Autopilot system missed a curve, causing a fatal crash for the driver.

Company involved

Tesla

AI system involved

Autopilot

1 source article · read the reporting →

NHTSA investigation PE25012: Traffic safety violations while Full Self Driving ("FSD") is engaged

The system made driving decisions that violated traffic safety laws, affecting other road users.

Company involved

Tesla, Inc.

AI system involved

Full Self Driving (FSD)

1 source article · read the reporting →

Tesla Full Self-Driving involved in fatal pedestrian crash in Arizona

In November 2023, a Tesla Model Y with Full Self-Driving engaged struck and killed 71-year-old Johna Story on Interstate 17 in Arizona. The system failed to brake due to sun glare, according to footage obtained by Bloomberg. The victim's family has sued Tesla and the driver. NHTSA is investigating whether the system poses an unacceptable safety risk.

Company involved

Tesla

AI system involved

Full Self-Driving

7 source articles · read the reporting →

Waymo Self-Driving Taxis Proactively Report to Police After Discovering Passenger Hiding a Gun

قامت سيارات الأجرة ذاتية القيادة من شركة Waymo بإبلاغ الشرطة بشكل استباقي بعد اكتشافها أن أحد الركاب كان يخفي مسدساً.…

A Waymo self-driving taxi in San Francisco detected two teenage passengers carrying an unidentified AR-15 rifle loaded with ammunition. The vehicle automatically stopped and alerted police, leading to the arrest of the two teenagers. Waymo stated that its system can identify violations of its terms of service, including carrying weapons, and will proactively notify police.

Company involved

Waymo

1 source article · read the reporting →

Tesla driver dies after vehicle stops on highway while using FSD/Autopilot

一名特斯拉司機在高速公路上因車輛突然停下而身亡，當時他正在使用FSD/自動駕駛系統 - Moomoo

A 43-year-old Tesla Model 3 driver was killed when his vehicle stopped in a travel lane on Interstate 4 and was rear-ended by a semi-truck. Florida Highway Patrol could not explain why the car stopped, but Tesla’s own filing confirms that the driver-assistance system was engaged and the car was stopped at 0 mph. Tesla has redacted key data, including the software version and event details, in its report to NHTSA.

Company involved

Tesla, Inc.

AI system involved

Autopilot / Full Self-Driving (FSD)

1 source article · read the reporting →

Hackers Extract Encryption Keys from Flock Surveillance Cameras, Revealing Person Detection Alongside Vehicle Tracking - finance.biggo.com

The system detected and recorded images of vehicles and people, affecting individuals in public spaces.

Company involved

Flock Safety

AI system involved

Flock Safety ALPR cameras

1 source article · read the reporting →

Woman Blindly Trusts Driver Assistance, Rear-Ends Lane-Changing Bus on Highway

女子轻信辅助驾驶放任不管 高速上直接追尾变道大巴车 - 驱动之家

On August 28, on the Shanghai-Kunming Expressway, a woman driving an SUV with driver assistance engaged rear-ended a bus that suddenly changed lanes. She assumed the system would brake automatically and took no action. The bus driver was found at fault for the lane change, but police warned against over-reliance on driver assistance.

1 source article · read the reporting →

Investigation probes autopilot failure after fatal Tesla accident in the United States

Investigação apura falha no piloto automático após acidente fatal com Tesla nos Estados Unidos - dfmobilidade.com.br

A fatal crash involving a Tesla Model 3 occurred at an intersection in Buena Vista Township, New Jersey. Authorities are investigating a possible failure of the Autopilot system and whether it was active at the time, adding to scrutiny of Tesla's driver-assistance technology.

Company involved

Tesla

AI system involved

Autopilot

1 source article · read the reporting →

Sibuxiang Genuine + Genuine Sibuxiang: Tesla Autopilot in Another Accident, Owner Says System Failed to Recognize Construction Barriers, Causing Rear-End…

四不像正版+正版四不像特斯拉自动驾驶再出事故，车主称系统未识别施工路障致追尾 - Pchome电脑之家

A Tesla owner reported that the vehicle's Autopilot system did not detect construction barriers, leading to a rear-end collision. The incident was reported by Pchome.

Company involved

Tesla

AI system involved

Autopilot

1 source article · read the reporting →

Ford BlueCruise crash in Philadelphia kills two, injures one

On March 3, 2024, a Ford Mustang Mach-E operating in hands-free partial automation mode (BlueCruise) collided with stationary vehicles in a work zone on I-95 in Philadelphia. The system recorded a speed of 72.4 mph with no braking or steering before impact. The drivers of the two stationary vehicles died, and the Ford driver was injured. The NTSB determined the probable cause to be driver impairment and distraction due to overreliance on automation, as well as Ford's inadequate integration of speed management with the automation system.

Company involved

Ford Motor Company

AI system involved

BlueCruise

9 source articles · read the reporting →

Tesla Model 3 crash, 1 killed, Autopilot verified: Multi-event collision with truck on highway

The Tesla Model 3, with Autopilot possibly engaged, struck an overturned semi-truck and a pedestrian, killing the driver and injuring the pedestrian.

Company involved

Tesla

AI system involved

Autopilot

1 source article · read the reporting →

Tesla Model 3 crash, 1 killed, Autopilot verified: Tesla hits pedestrian

Tesla Model 3 struck a pedestrian on the highway

AI system involved

Tesla Model 3

1 source article · read the reporting →

Tesla Lawsuit Claims Autopilot Accelerated So Violently the Tires Burst - Autoblog

The vehicle accelerated violently, causing the tires to burst, affecting the driver.

Company involved

Tesla

AI system involved

Autopilot

1 source article · read the reporting →

First Accident Before Test Phase: Robotaxis Launch in Furttal

Erster Unfall, bevor die Testphase beginnt: Robotaxis starten im Furttal - nzz.ch

Switzerland's first self-driving taxi service launched in the Furttal region near Zurich, but three days before the start a test vehicle had a minor accident in Dänikon, damaging a ground plate and two tires. The pilot project, called Iamo, aims to complement public transport, with a safety driver on board during the test phase.

Company involved

Swiss Transit Lab

1 source article · read the reporting →

Tesla's New Autopilot Crashes, Driver's Hands Off Wheel Entire Time

so米体育app下载特斯拉新款自动驾驶出事故，车主全程双手离方向盘 - Pchome电脑之家

A Tesla vehicle with the new autopilot system was involved in an accident. The driver kept both hands off the steering wheel throughout the incident. The news was reported by Pchome.

Company involved

Tesla

1 source article · read the reporting →

NHTSA investigates Tesla's 'Mad Max' driving mode

Tesla introduced a 'Mad Max' mode for its Full Self-Driving system that speeds and weaves through traffic. The National Highway Traffic Safety Administration is seeking information from the company after multiple reports of Teslas running stop signs and crossing into oncoming traffic. A lawyer who won a wrongful death lawsuit against Tesla alleges the company deliberately programmes cars to exceed speed limits and drive aggressively. The investigation is ongoing.

Company involved

Tesla

AI system involved

Full Self-Driving (FSD) 'Mad Max' mode

5 source articles · read the reporting →

Tesla fires employee for posting critical FSD Beta video on YouTube

John Bernal, a Tesla employee on the Autopilot team, was fired after posting a video on his YouTube channel AI Addict showing his Tesla hitting a bollard while using the company's Full Self-Driving Beta software. Tesla managers told him his channel was a conflict of interest and had previously asked him not to post negative content. After his dismissal, his access to the FSD Beta was revoked. Bernal says he still cares about vehicle safety and intends to continue his channel.

Company involved

Tesla

AI system involved

Full Self-Driving Beta

3 source articles · read the reporting →

Tesla on Autopilot hits Florida Highway Patrol cruiser in Orlando

A Tesla using its partially automated Autopilot system struck a Florida Highway Patrol cruiser on an interstate near Orlando on August 28, 2021. The trooper had activated emergency lights and was assisting a disabled vehicle. The Tesla driver and the driver of the disabled vehicle suffered minor injuries; the trooper was unhurt. The National Highway Traffic Safety Administration had previously opened an investigation into Tesla's Autopilot after similar collisions with emergency vehicles.

Company involved

Tesla

AI system involved

Autopilot

10 source articles · read the reporting →

Tesla Autopilot fatally strikes motorcyclist in Japan, lawsuit alleges

In April 2018, a Tesla Model X in Autopilot mode struck a group of parked motorcyclists in Kanagawa, Japan, killing one. The driver had dozed off, and the lawsuit alleges that multiple Autopilot sensors failed, causing the car to accelerate into the group. The victim's family is suing Tesla in California, claiming the system was defective and that Tesla ignored safety recommendations.

Company involved

Tesla

AI system involved

Autopilot

4 source articles · read the reporting →

Tesla Employee Dies in Full Self-Driving Crash in Colorado Mountains

Hans von Ohain, a Tesla recruiter, died when his Model 3 crashed into a tree and caught fire while using Tesla's Full Self-Driving feature on a mountain road in Colorado. His widow, Nora Bass, criticised Tesla for lack of response, saying they were treated like 'guinea pigs'. Investigators noted that Ohain's blood alcohol was over three times the legal limit, but the crash did not show typical signs of drunk driving, suggesting the driver-assistance feature was active. Tesla has not commented on the incident.

Company involved

Tesla

AI system involved

Full Self-Driving

10 source articles · read the reporting →

Paris taxi company G7 suspends Tesla Model 3 after fatal crash

A Paris taxi company, G7, suspended Tesla Model 3 cars from its fleet after an off-duty driver's car was involved in a collision that killed one person and injured 20 others. It was not clear whether Tesla's Autopilot was active. G7 said Tesla's initial inquiry found no malfunction but that the car appeared to accelerate when the driver tried to brake. The company said it would pay affected drivers' lost wages while a police investigation continues.

Company involved

G7

AI system involved

Tesla Model 3

7 source articles · read the reporting →

Worker dies after Tesla on autopilot collides with repair truck in Taoyuan

A Tesla with Autopilot engaged collided with a highway maintenance truck on National Highway 3 near Daxi, Taoyuan. The driver admitted he was not paying attention while Autopilot was active. A worker putting out traffic cones after the crash was struck and killed by a BMW 20 seconds later. Police are investigating the two drivers for public safety offenses.

AI system involved

Autopilot

2 source articles · read the reporting →

Uber operator pleads guilty in fatal self-driving crash

In 2018, an Uber self-driving car struck and killed Elaine Herzberg as she crossed a road in Tempe, Arizona. The vehicle's operator, Rafaela Vasquez, was charged with negligent homicide but pleaded guilty to endangerment in July 2023 and received probation. The National Transportation Safety Board found that Uber's inadequate safety culture and the operator's distraction contributed to the crash. Vasquez alleged she was monitoring Uber's Slack, not watching a show, but accepted responsibility.

Company involved

Uber

10 source articles · read the reporting →

Tesla Autopilot involved in 13 fatal crashes, US regulator finds

The US National Highway Traffic Safety Administration (NHTSA) found that Tesla's Autopilot driver-assistance system was involved in at least 13 fatal crashes. The regulator stated that Tesla's weak driver engagement system and permissive operating capabilities created a critical safety gap, and that the name 'Autopilot' may mislead drivers about its capabilities. Tesla issued a recall for 2.03 million vehicles to improve driver monitoring, but the NHTSA opened a second investigation to assess whether the recall adequately addressed the safety concerns.

Company involved

Tesla

AI system involved

Autopilot

2 source articles · read the reporting →