11 Crashes from 'System Errors' in Pilot Self-Driving Cars; Investigations Lag, Cause Classification Inadequate
시범 자율주행차 '시스템 오류' 사고 11건…조사는 '늑장' 원인 분류도 '부실' - v.daum.net
Between 2023 and 2025, 11 traffic accidents were caused by autonomous driving system errors in vehicles with temporary operation permits, resulting in four minor injuries. In the first half of this year alone, 27 accidents occurred, but only two have had their causes confirmed, while the remaining 25 are still under investigation. The Korea Transportation Safety Authority has not disclosed company-specific accident details, citing trade secrets, and uses only broad categories for causes.
1 source article · read the reporting →
Baidu's 'Apollo Go' robotaxi hits jaywalking pedestrian in Wuhan
On 7 July 2024, a Baidu 'Apollo Go' autonomous taxi struck a pedestrian at an intersection in Wuhan, China. The vehicle was starting at a green light when it made slight contact with the pedestrian, who was crossing against the red light. Baidu stated it cooperated with police and accompanied the injured person to hospital for observation. The incident has raised questions about liability for driverless cars under China's evolving regulations.
- Company involved
- Baidu
- AI system involved
- 萝卜快跑
1 source article · read the reporting →
Tesla Autopilot fails to centre car, shredding tyres in Redmond
In summer 2017, Eric Horvitz activated the Autopilot function of his Tesla sedan while driving in Redmond, Washington. The system failed to centre the car properly, causing both driver's side tyres to hit a raised curb and shred instantly. Horvitz regained control and watched his car being towed away. He reported the incident to Tesla, which he says was more concerned about liability than addressing ethical issues with its AI.
- Company involved
- Tesla
- AI system involved
- Autopilot
2 source articles · read the reporting →
Life or Death in Two Seconds: The Smart Driving Chaos Behind the Fatal Xiaomi Car Crash—Every Owner Must Read!
生死两秒：小米汽车致命车祸背后的智驾迷局，所有车主必看！ - 汽车之家
A Xiaomi SU7 was traveling at 116 km/h on the Dezhou-Shanghai Expressway on March 29 when its NOA system issued an obstacle warning, leaving the driver only about two seconds to react. The vehicle hit a cement barrier at 97 km/h and caught fire, killing three university students inside. The article criticizes the short warning time, the gap between marketing claims and driver responsibility disclaimers, and calls for stricter smart driving safety standards.
- Company involved
- Xiaomi
- AI system involved
- NOA
1 source article · read the reporting →
Tesla recalls 11,704 vehicles over Full Self-Driving Beta software error
Tesla voluntarily recalled 11,704 vehicles after identifying a software error in its Full Self-Driving Beta system that could cause false forward-collision warnings or unexpected automatic emergency braking. The error, a communication disconnect between two on-board chips, was introduced in an over-the-air update on October 23, 2021. Tesla stated it was not aware of any crashes or injuries and issued a software fix. The National Highway Traffic Safety Administration (NHTSA) is monitoring the situation.
- Company involved
- Tesla
- AI system involved
- Full Self-Driving Beta
10 source articles · read the reporting →
Tesla Staged 2016 Self-Driving Video, Deposition Reveals
In October 2016, Tesla published a video titled 'Full Self-Driving Hardware on all Teslas' showing a car driving itself without human input. A deposition by Tesla's director of Autopilot software, Ashok Elluswamy, revealed that the video was staged on a predetermined route and that the car crashed into a fence during filming. The production version of Autopilot at the time did not have the capability to respond to traffic lights as shown. Tesla and Elon Musk did not disclose that the video was edited to portray the system more favourably.
- Company involved
- Tesla
- AI system involved
- Full Self-Driving Hardware
6 source articles · read the reporting →
TuSimple self-driving truck crashes in Arizona after system executes old command
In April 2022, a TuSimple autonomous semi-trailer truck travelling on I-10 in Tucson, Arizona, suddenly veered left and struck a concrete divider. The company initially blamed human error, but an internal report revealed that the crash occurred because an operator failed to properly reboot the system, causing it to execute a 2.5-minute-old left-turn command. No injuries or property damage were reported. Federal regulators, including the FMCSA and NHTSA, have launched investigations into the incident.
- Company involved
- TuSimple
2 source articles · read the reporting →
Tesla Model S on Autopilot crashes into stationary van on Swiss highway
A Tesla Model S owner in Switzerland was using the car's Autopilot and Adaptive Cruise Control on a highway when the system failed to detect a stationary van after the preceding vehicle changed lanes. The car accelerated slightly and collided with the van, causing front-end damage that required replacement of parts including a parking sensor and a steel beam. The driver acknowledged they could have reacted sooner but had grown to trust the system after many successful trips. Tesla's manual warns that the system may not brake for stationary vehicles in such situations.
- Company involved
- Tesla
- AI system involved
- Tesla Autopilot
3 source articles · read the reporting →
Tesla Autopilot crashes into parked police SUV in Laguna Beach
On 29 May 2018, Dave Key's Tesla Model S, operating in Autopilot mode, drifted out of its lane and collided with a parked police SUV in Laguna Beach, California. No one was injured, but the car was written off. The crash was attributed to a known limitation of Tesla's AI, which struggled to distinguish stationary objects from the background. Key, a former software entrepreneur, later bought three more Teslas and used the Full Self-Driving beta.
- Company involved
- Tesla
- AI system involved
- Autopilot
4 source articles · read the reporting →
Volvo self-driving car blinded by snow in Sweden test
During testing in Jokkmokk, Sweden, a Volvo self-driving XC90 SUV's radar sensors became caked with frozen flakes, rendering the vehicle unable to read the road. The incident highlighted the challenge of snow blindness for autonomous vehicles. Ford engineers later proposed a solution using lidar to create high-definition 3-D maps.
- Company involved
- Volvo Cars
- AI system involved
- self-driving XC90 SUV
4 source articles · read the reporting →
Tesla's Full Self-Driving Beta fails to stop for pedestrian at crosswalk
A video posted online shows a Tesla vehicle using the Full Self-Driving Beta software detecting a pedestrian at a crosswalk in San Francisco but not stopping, only slowing from 26 mph to 24 mph after passing. The incident highlights the system's failure to comply with California law requiring drivers to yield to pedestrians. Tesla has faced regulatory scrutiny from NHTSA over unsafe behaviours around intersections, and this video adds to concerns about the safety of the unfinished software being tested on public roads.
- Company involved
- Tesla
- AI system involved
- Full Self-Driving Beta
5 source articles · read the reporting →
Truck drivers sue Motive over AI dash cam mass surveillance
Truck drivers Adam Dean and Christian Erickson filed a class action lawsuit against Motive Technologies Inc., alleging that the company's AI-powered dash cams with automatic license plate readers collected license plate data and tracked drivers' movements without consent. The lawsuit, filed in California federal court, claims violations of California's ALPR Law and seeks damages of $2,500 per class member. The drivers allege privacy invasion and emotional distress from the surreptitious surveillance.
- Company involved
- Motive Technologies Inc.
1 source article · read the reporting →
Xpeng P7 Collides with Truck During NGP Assisted Driving, Driver Injured
On 23 September 2021, a Xpeng P7 owner in China was using the NGP automatic navigation assisted driving system when the vehicle failed to identify a flatbed truck ahead and collided with its rear. The driver, who trusted the system to brake automatically, sustained slight injuries. Xpeng Motors stated that the system's functions were operating normally before the crash and that further analysis would be conducted. Experts suggested the visual camera may have misjudged the distance due to the truck's unusual shape.
- Company involved
- Xpeng Motors
- AI system involved
- NGP automatic navigation assisted driving system
1 source article · read the reporting →
Tesla Recalls 360,000 Vehicles with Full Self-Driving Beta Over Crash Risk
Tesla is recalling more than 360,000 vehicles equipped with its Full Self-Driving Beta software after a government regulator found that the cars could proceed through intersections without stopping, posing a crash risk. The company said it would issue a remote software update to remedy the problem. The recall is a significant setback for Tesla's autonomous driving ambitions.
- Company involved
- Tesla
- AI system involved
- Full Self-Driving Beta
10 source articles · read the reporting →
Alcon Entertainment sues Tesla and Elon Musk over AI-generated Blade Runner 2049 imagery
Alcon Entertainment, the producer of Blade Runner 2049, has sued Tesla, Elon Musk, and Warner Bros Discovery, alleging they used AI to create promotional imagery based on the film for Tesla's robotaxi launch event on 10 October 2024. Alcon claims it had specifically denied a request from Warner Bros to use material from the film. The lawsuit alleges misappropriation and false endorsement, and describes the financial magnitude of the misappropriation as substantial.
- Company involved
- Tesla
6 source articles · read the reporting →
Tesla's Full Self-Driving beta includes assertive mode that may perform rolling stops
Tesla's Full Self-Driving beta software includes driving profiles, including an 'Assertive' mode that may perform rolling stops, follow cars more closely, and change lanes more frequently. The feature was included in an October 2021 update that was briefly pulled due to issues with left turns. In November 2021, a Tesla using FSD mode was involved in a crash that left the vehicle severely damaged.
- Company involved
- Tesla
- AI system involved
- Full Self-Driving (FSD) beta
5 source articles · read the reporting →
Cruise autonomous vehicle briefly evades police traffic stop
On April 2, 2022, a driverless Cruise taxi in San Francisco was pulled over by police for driving without headlights. The vehicle initially drove away before stopping at a safe location, confusing officers. Cruise stated the car yielded as programmed and no citation was issued. The incident highlights challenges of autonomous vehicles interacting with law enforcement.
- Company involved
- Cruise
5 source articles · read the reporting →
Hackers trick Tesla Autopilot into veering into oncoming lane
Researchers at Tencent's Keen Security Lab demonstrated that stickers placed on a road could trick a Tesla Model S's Autopilot system into veering into the oncoming traffic lane. The attack exploited the system's reliance on camera-based computer vision to recognise lanes. Tesla responded that this vulnerability was unrealistic because a driver can easily override Autopilot, while saying a separate steering-wheel hijack had been fixed in a software update.
- Company involved
- Tesla
- AI system involved
- Autopilot
9 source articles · read the reporting →
Researchers demonstrate drone attack on Renault Captur ADAS with fake road signs
Security researchers from Ben Gurion University demonstrated a vulnerability in the Mobileye ADAS system used in a Renault Captur. They used drones to project fake road signs for 100 milliseconds, too quick for human perception but detectable by the system. The attack could trick the car into making unsafe maneuvers, though the system is only Level 0 (advisory). The researchers published their findings in a paper titled 'MobilBye: Attacking ADAS with Camera Spoofing'.
- AI system involved
- Mobileye ADAS
7 source articles · read the reporting →
Nvidia releases AI-powered Eye Contact feature for Broadcast app
Nvidia has released a beta version of Eye Contact, an AI-powered feature for its Broadcast app that replaces a user's eyes in video streams with simulated eyeballs that always face the camera. The feature is intended to maintain eye contact even when the user looks away, but social media response has been largely negative. A former TV news anchor said that unbroken eye contact can be regarded as creepy or uncomfortable.
- Company involved
- Nvidia
- AI system involved
- Eye Contact
8 source articles · read the reporting →
Delphi denies near-miss between its self-driving car and Google's autonomous vehicle
Delphi Automotive initially reported a near-miss between its self-driving Audi Q5 SUV and a Google Lexus RX400h in Palo Alto, California. A Delphi executive claimed Google's car cut off the Delphi vehicle, which took evasive action. However, Delphi later denied the incident, stating the vehicles were a lane width apart and that both cars performed correctly. Google also confirmed the cars behaved appropriately in an ordinary driving scenario.
- Company involved
- Delphi Automotive
- AI system involved
- Delphi self-driving car (Audi Q5 SUV)
7 source articles · read the reporting →
Tesla Autopilot stops for phantom stop sign on digital billboard
Researchers at Ben-Gurion University demonstrated that Tesla's Autopilot could be tricked into stopping for a split-second phantom stop sign displayed on a digital billboard. The system falsely detected the depthless object and applied the brakes in the middle of the road. The researchers disclosed their findings to Tesla and Mobileye. No actual accident occurred during the test.
- Company involved
- Tesla
- AI system involved
- Tesla Autopilot
9 source articles · read the reporting →
NHTSA investigates Tesla Autopilot over 11 crashes with first responder vehicles
The US National Highway Traffic Safety Administration (NHTSA) has opened an investigation into Tesla's Autopilot system after identifying 11 crashes in which Tesla vehicles collided with first responder vehicles. In each incident, the vehicles were confirmed to have Autopilot or Traffic-Aware Cruise Control engaged. Most crashes occurred after dark and involved scene control measures such as lights, flares, and road cones. NHTSA will assess the system's ability to monitor driver engagement and handle emergency scenes.
- Company involved
- Tesla Inc.
- AI system involved
- Autopilot
10 source articles · read the reporting →
Tesla Model X on Autopilot crashes into police car, injuring five officers
On February 27, 2021, a Tesla Model X operating on Autopilot crashed into a stationary police vehicle on a highway near Montgomery County, Texas, injuring five officers and hospitalizing the subject of a traffic stop. The driver was intoxicated and the system had alerted him 150 times to apply force to the steering wheel. The National Highway Traffic Safety Administration (NHTSA) is investigating the incident, along with 15 other similar crashes involving Teslas and emergency vehicles.
- Company involved
- Tesla
- AI system involved
- Autopilot
10 source articles · read the reporting →
