Cruise autonomous vehicle briefly evades police traffic stop
On April 2, 2022, a driverless Cruise taxi in San Francisco was pulled over by police for driving without headlights. The vehicle initially drove away before stopping at a safe location, confusing officers. Cruise stated the car yielded as programmed and no citation was issued. The incident highlights challenges of autonomous vehicles interacting with law enforcement.
- Company involved
- Cruise
5 source articles · read the reporting →
Hackers trick Tesla Autopilot into veering into oncoming lane
Researchers at Tencent's Keen Security Lab demonstrated that stickers placed on a road could trick a Tesla Model S's Autopilot system into veering into the oncoming traffic lane. The attack exploited the system's reliance on camera-based computer vision to recognise lanes. Tesla responded that this vulnerability was unrealistic because a driver can easily override Autopilot, while saying a separate steering-wheel hijack had been fixed in a software update.
- Company involved
- Tesla
- AI system involved
- Autopilot
9 source articles · read the reporting →
Researchers demonstrate drone attack on Renault Captur ADAS with fake road signs
Security researchers from Ben Gurion University demonstrated a vulnerability in the Mobileye ADAS system used in a Renault Captur. They used drones to project fake road signs for 100 milliseconds, too quick for human perception but detectable by the system. The attack could trick the car into making unsafe maneuvers, though the system is only Level 0 (advisory). The researchers published their findings in a paper titled 'MobilBye: Attacking ADAS with Camera Spoofing'.
- AI system involved
- Mobileye ADAS
7 source articles · read the reporting →
Delphi denies near-miss between its self-driving car and Google's autonomous vehicle
Delphi Automotive initially reported a near-miss between its self-driving Audi Q5 SUV and a Google Lexus RX400h in Palo Alto, California. A Delphi executive claimed Google's car cut off the Delphi vehicle, which took evasive action. However, Delphi later denied the incident, stating the vehicles were a lane width apart and that both cars performed correctly. Google also confirmed the cars behaved appropriately in an ordinary driving scenario.
- Company involved
- Delphi Automotive
- AI system involved
- Delphi self-driving car (Audi Q5 SUV)
7 source articles · read the reporting →
Tesla Autopilot tricked into speeding by electrical tape on sign
Researchers from McAfee Labs demonstrated that a two-inch piece of black electrical tape stuck on a 35 mph speed limit sign can cause the Traffic Aware Cruise Control in older Tesla Model S and X vehicles to misread the sign as 85 mph and automatically accelerate. The vulnerability affects vehicles using the MobilEye EyeQ3 camera system (Hardware Pack 1). The driver can override by braking. Tesla and MobilEye were informed but did not commit to fixing the issue.
- Company involved
- Tesla
- AI system involved
- Tesla Autopilot (Hardware Pack 1 with MobilEye EyeQ3)
8 source articles · read the reporting →
Tesla Autopilot stops for phantom stop sign on digital billboard
Researchers at Ben-Gurion University demonstrated that Tesla's Autopilot could be tricked into stopping for a split-second phantom stop sign displayed on a digital billboard. The system falsely detected the depthless object and applied the brakes in the middle of the road. The researchers disclosed their findings to Tesla and Mobileye. No actual accident occurred during the test.
- Company involved
- Tesla
- AI system involved
- Tesla Autopilot
9 source articles · read the reporting →
AnyVision patents drone facial recognition technology
AnyVision, an Israeli private surveillance company, filed a US patent for drone technology that uses facial recognition to identify people on the ground. The patent describes adaptive positioning of drones to find optimal angles for accurate facial recognition. The technology is not yet in production, but the company's CEO stated it will become a reality soon. Potential privacy concerns have been raised.
- Company involved
- AnyVision
- AI system involved
- Adaptive positioning of drones for enhanced facial recognition
9 source articles · read the reporting →
NHTSA investigates Tesla Autopilot over 11 crashes with first responder vehicles
The US National Highway Traffic Safety Administration (NHTSA) has opened an investigation into Tesla's Autopilot system after identifying 11 crashes in which Tesla vehicles collided with first responder vehicles. In each incident, the vehicles were confirmed to have Autopilot or Traffic-Aware Cruise Control engaged. Most crashes occurred after dark and involved scene control measures such as lights, flares, and road cones. NHTSA will assess the system's ability to monitor driver engagement and handle emergency scenes.
- Company involved
- Tesla Inc.
- AI system involved
- Autopilot
10 source articles · read the reporting →
ViaQuatro's facial recognition system in São Paulo metro challenged in court
In April 2018, ViaQuatro installed the Digital Interactive Doors System, developed by AdMobilize, on the São Paulo metro's yellow line. The system used cameras to detect passengers' faces and claimed to infer their emotion, age, and gender in order to target advertisements. The Brazilian Institute of Consumer Protection (IDEC) filed a public civil action alleging that the system violated consumer and data protection laws by processing biometric data without consent and making pseudoscientific and discriminatory inferences. A judge ordered the cameras removed in August 2018, and the case is pending a final ruling.
- Company involved
- ViaQuatro
- AI system involved
- Digital Interactive Doors System (DID system)
10 source articles · read the reporting →
Tesla Model X on Autopilot crashes into police car, injuring five officers
On February 27, 2021, a Tesla Model X operating on Autopilot crashed into a stationary police vehicle on a highway near Montgomery County, Texas, injuring five officers and hospitalizing the subject of a traffic stop. The driver was intoxicated and the system had alerted him 150 times to apply force to the steering wheel. The National Highway Traffic Safety Administration (NHTSA) is investigating the incident, along with 15 other similar crashes involving Teslas and emergency vehicles.
- Company involved
- Tesla
- AI system involved
- Autopilot
10 source articles · read the reporting →
California DMV suspends Cruise LLC driverless and deployment permits
The California Department of Motor Vehicles suspended Cruise LLC's autonomous vehicle deployment and driverless testing permits, saying the vehicles were not safe for public operation and that Cruise had misrepresented safety information. The suspension took effect immediately under California regulations. Cruise may apply to reinstate the permits once it satisfies the department's requirements. Its permit for testing with a safety driver was not affected.
- Company involved
- Cruise LLC
10 source articles · read the reporting →
Cruise alleged to rely on human operators for autonomous driving
A New York Times article alleged that Cruise's autonomous vehicles rely on human operators to achieve autonomous driving, with 1.5 workers per vehicle intervening every 2.5 to five miles. Cruise CEO Kyle Vogt responded on Hacker News, stating that remote assistance is used 2-4% of the time and that many sessions are resolved by the vehicle itself. The company declined an interview request from the NYT.
- Company involved
- Cruise
- AI system involved
- Cruise AV
9 source articles · read the reporting →
Tesla Model 3 crash with truck kills driver using partial automation
At 6:17 a.m. on March 1, 2019, a Tesla Model 3 operating with partial driving automation struck a semitrailer in Delray Beach, Florida, killing the 50-year-old driver. The NTSB determined the probable cause was the truck driver's failure to yield right of way combined with the car driver's inattention due to overreliance on the automation, with Tesla's system design allowing disengagement as a contributing factor.
- Company involved
- Tesla
10 source articles · read the reporting →
Tesla's Full Self-Driving involved in fatal Bay Bridge crash
On Thanksgiving 2022, a Tesla vehicle using its Full Self-Driving feature was involved in a fatal accident on the San Francisco Bay Bridge. Footage published by The Intercept showed the system braking abruptly. The incident highlights safety concerns with autonomous driving technology.
- Company involved
- Tesla
- AI system involved
- Full Self-Driving
7 source articles · read the reporting →
Driver relying on "smart driving" on highway crashes after system suddenly disengages near truck
司机依赖“智驾 ”跑高速，逼近大货车时“智驾”突然退出引发车祸 - 手机新浪网
A driver was using a smart driving system on an expressway. As the vehicle approached a large truck, the system unexpectedly disengaged, leading to a crash.
1 source article · read the reporting →
Tesla Autopilot accidents and data leak suggest larger problems
Tesla is facing numerous lawsuits over its Autopilot driver assistance system, which has been involved in accidents. Internal data leaked to Handelsblatt suggests the company has larger technical problems than previously known. The US National Highway Traffic Safety Administration is investigating. Tesla has dismissed the data leak as theft.
- Company involved
- Tesla
- AI system involved
- Autopilot
9 source articles · read the reporting →
NHTSA investigation PE24016: Unexpected ADS behavior
Waymo's automated driving system exhibited unexpected behavior during driving.
- Company involved
- Waymo
1 source article · read the reporting →
Consumer Reports finds Tesla Autopilot operates without driver
Consumer Reports engineers tested a Tesla Model Y on a closed track and found that the Autopilot system could operate the vehicle without anyone in the driver's seat, failing to warn or detect the absence of a driver. The test highlighted a safety flaw as investigators probed a fatal crash in Texas where two men died in a Tesla Model S believed to be operating without a driver. Tesla did not respond to a request for comment.
- Company involved
- Tesla
- AI system involved
- Autopilot
10 source articles · read the reporting →
Tesla on Autopilot Crashes into Overturned Truck on Taiwan Highway
On 1 June 2020, a Tesla sedan with Autopilot engaged failed to detect an overturned delivery truck on National Highway 1 in Taiwan and collided with it. The driver, a 53-year-old man, had set the system to 110 km/h and assumed it would avoid obstacles, but he braked too late. The truck driver, who was standing on the highway after his own accident, narrowly avoided being struck. No serious injuries were reported, and police are investigating the cause, with possible factors including the truck's unusual shape and the system's limitation to detecting only moving vehicles.
- AI system involved
- Autopilot
3 source articles · read the reporting →
Tesla Full Self-Driving system fails to recognise train, driver intervenes
A Tesla owner in Ohio reported that his car in Full Self-Driving mode attempted to drive into a crossing train on May 8, 2024. The driver took control and avoided a collision, but the car hit the crossing arm causing damage. The driver said it was the second such incident in six months. Tesla did not respond to requests for comment.
- Company involved
- Tesla
- AI system involved
- Full Self-Driving (FSD)
8 source articles · read the reporting →
Cruise Self-Driving Car Rear-Ended by Human-Driven Cruise Vehicle in San Francisco
On 11 June 2018, a self-driving Cruise Chevrolet Bolt was rear-ended by another Cruise vehicle driven by a human employee on Bryant Street in San Francisco. The collision caused only minor scuffs and no injuries. Cruise reported the incident to the California DMV as required. The incident highlights the challenges of autonomous vehicles interacting with human drivers.
- Company involved
- Cruise
2 source articles · read the reporting →
Tesla Autopilot Failed to Brake, Killing 15-Year-Old, Family Alleges
In August 2019, a Tesla Model 3 on Autopilot rear-ended a Ford Explorer pickup on a California freeway, killing 15-year-old Jovani Maldonado. Data shows neither the system nor the driver braked until a fraction of a second before impact. The family has filed a lawsuit alleging Autopilot is defective and failed to react to traffic. Tesla has denied responsibility, blaming the driver's inattention.
- Company involved
- Tesla
- AI system involved
- Autopilot
2 source articles · read the reporting →
Father Sues Tesla After Son's Fatal Autopilot Crash in China
In January 2016, Gao Yaning died when his Tesla Model S crashed into a road sweeper on a highway in Hebei, China. His father, Gao Jubin, alleges the car was using Autopilot, which failed to detect the stationary truck and did not brake. Jubin filed a lawsuit against Tesla, accusing the company of exaggerating Autopilot's capabilities, and the case remains ongoing. Tesla has stated that it is cooperating with the investigation and that the system functioned as designed.
- Company involved
- Tesla
- AI system involved
- Autopilot
4 source articles · read the reporting →
Google and Delphi Self-Driving Cars Involved in Minor Accidents in California
Four self-driving cars in California have been involved in accidents since September 2014, when the state began issuing testing permits. Two occurred while the cars were in autonomous mode, and two while a human was driving. Google and Delphi, the companies operating the vehicles, stated that their cars were not at fault and that the collisions were minor, with no injuries. The accidents have raised concerns about transparency, as details are kept confidential under state law.
- Company involved
- Google Inc. and Delphi Automotive
10 source articles · read the reporting →
