Waymo robotaxi and Serve delivery robot collide in Los Angeles
On December 27, 2024, a Waymo robotaxi and a Serve Robotics sidewalk delivery robot collided at an intersection in West Hollywood, Los Angeles. The Waymo vehicle was making a right turn when the Serve bot, under remote supervision, moved into the turning lane. The Waymo Driver system classified the bot as an inanimate object and applied hard braking, but made contact at 4 mph. Neither vehicle was damaged, and both companies stated they are working to avoid similar incidents in the future.
- Company involved
- Waymo and Serve Robotics
- AI system involved
- Waymo Driver and Serve delivery robot
1 source article · read the reporting →
Tesla Autopilot crashes into parked police SUV in Laguna Beach
On 29 May 2018, Dave Key's Tesla Model S, operating in Autopilot mode, drifted out of its lane and collided with a parked police SUV in Laguna Beach, California. No one was injured, but the car was written off. The crash was attributed to a known limitation of Tesla's AI, which struggled to distinguish stationary objects from the background. Key, a former software entrepreneur, later bought three more Teslas and used the Full Self-Driving beta.
- Company involved
- Tesla
- AI system involved
- Autopilot
4 source articles · read the reporting →
Volvo self-driving car blinded by snow in Sweden test
During testing in Jokkmokk, Sweden, a Volvo self-driving XC90 SUV's radar sensors became caked with frozen flakes, rendering the vehicle unable to read the road. The incident highlighted the challenge of snow blindness for autonomous vehicles. Ford engineers later proposed a solution using lidar to create high-definition 3-D maps.
- Company involved
- Volvo Cars
- AI system involved
- self-driving XC90 SUV
4 source articles · read the reporting →
Tesla's Full Self-Driving Beta fails to stop for pedestrian at crosswalk
A video posted online shows a Tesla vehicle using the Full Self-Driving Beta software detecting a pedestrian at a crosswalk in San Francisco but not stopping, only slowing from 26 mph to 24 mph after passing. The incident highlights the system's failure to comply with California law requiring drivers to yield to pedestrians. Tesla has faced regulatory scrutiny from NHTSA over unsafe behaviours around intersections, and this video adds to concerns about the safety of the unfinished software being tested on public roads.
- Company involved
- Tesla
- AI system involved
- Full Self-Driving Beta
5 source articles · read the reporting →
Xpeng P7 Collides with Truck During NGP Assisted Driving, Driver Injured
On 23 September 2021, a Xpeng P7 owner in China was using the NGP automatic navigation assisted driving system when the vehicle failed to identify a flatbed truck ahead and collided with its rear. The driver, who trusted the system to brake automatically, sustained slight injuries. Xpeng Motors stated that the system's functions were operating normally before the crash and that further analysis would be conducted. Experts suggested the visual camera may have misjudged the distance due to the truck's unusual shape.
- Company involved
- Xpeng Motors
- AI system involved
- NGP automatic navigation assisted driving system
1 source article · read the reporting →
Uber suspends autonomous tests after Arizona crash
Uber suspended its self-driving car tests in Arizona and paused operations in Pittsburgh after one of its autonomous Volvo SUVs was involved in a high-impact crash in Tempe, Arizona. The crash occurred when another vehicle failed to yield, causing the Uber car to flip on its side. No injuries were reported, and the Uber vehicle was not at fault. Uber is investigating the incident.
- Company involved
- Uber Technologies Inc.
8 source articles · read the reporting →
Tesla's Full Self-Driving beta includes assertive mode that may perform rolling stops
Tesla's Full Self-Driving beta software includes driving profiles, including an 'Assertive' mode that may perform rolling stops, follow cars more closely, and change lanes more frequently. The feature was included in an October 2021 update that was briefly pulled due to issues with left turns. In November 2021, a Tesla using FSD mode was involved in a crash that left the vehicle severely damaged.
- Company involved
- Tesla
- AI system involved
- Full Self-Driving (FSD) beta
5 source articles · read the reporting →
Uber's Self-Driving Cars Ran Red Lights in San Francisco, Internal Documents Reveal
In late 2016, Uber's autonomous vehicles were recorded running red lights during a pilot program in San Francisco, endangering bicyclists. Uber publicly blamed human error and suspended a driver, but internal documents and employee statements indicated the cars were in autonomous mode at the time. The company declined to apply for a state testing permit, which would have required disengagement reports, and subsequently moved its self-driving operations to Arizona.
- Company involved
- Uber
9 source articles · read the reporting →
Cruise Recalls Self-Driving Software After San Francisco Crash Injures Two
In June 2022, a Cruise autonomous vehicle operating in San Francisco attempted an unprotected left turn and braked hard after incorrectly predicting an oncoming car would turn right. The other car continued straight and struck the Cruise vehicle, injuring a Cruise employee and a person in the other car. Cruise subsequently recalled the software on 80 vehicles, temporarily restricted left turns, and issued a software update to improve predictions. The US National Highway Traffic Safety Administration oversaw the recall.
- Company involved
- Cruise
7 source articles · read the reporting →
Waymo self-driving taxi blocks road after confusion with traffic cones
A Waymo self-driving taxi in Phoenix, Arizona, became confused by traffic cones closing a lane and stopped, blocking the road. The vehicle repeatedly drove away from roadside assistance dispatched to help, before a human driver took over. Waymo acknowledged the incident, stating that its remote guidance team provided incorrect instructions and that it has since improved its operational process. The passenger, who documented the event, said such mistakes are rare.
- Company involved
- Waymo
- AI system involved
- Waymo Driver
3 source articles · read the reporting →
Tesla Full Self-Driving confused by truck carrying traffic lights
A Tesla vehicle using Full Self-Driving mode encountered a truck loaded with traffic lights. The system identified the traffic lights on the truck but was unable to determine the correct action, creating a confusing edge case. The incident was shared on social media, highlighting a limitation of the autonomous driving system.
- Company involved
- Tesla
- AI system involved
- Full Self-Driving
2 source articles · read the reporting →
Cruise autonomous vehicle briefly evades police traffic stop
On April 2, 2022, a driverless Cruise taxi in San Francisco was pulled over by police for driving without headlights. The vehicle initially drove away before stopping at a safe location, confusing officers. Cruise stated the car yielded as programmed and no citation was issued. The incident highlights challenges of autonomous vehicles interacting with law enforcement.
- Company involved
- Cruise
5 source articles · read the reporting →
Hackers trick Tesla Autopilot into veering into oncoming lane
Researchers at Tencent's Keen Security Lab demonstrated that stickers placed on a road could trick a Tesla Model S's Autopilot system into veering into the oncoming traffic lane. The attack exploited the system's reliance on camera-based computer vision to recognise lanes. Tesla responded that this vulnerability was unrealistic because a driver can easily override Autopilot, while saying a separate steering-wheel hijack had been fixed in a software update.
- Company involved
- Tesla
- AI system involved
- Autopilot
9 source articles · read the reporting →
Delphi denies near-miss between its self-driving car and Google's autonomous vehicle
Delphi Automotive initially reported a near-miss between its self-driving Audi Q5 SUV and a Google Lexus RX400h in Palo Alto, California. A Delphi executive claimed Google's car cut off the Delphi vehicle, which took evasive action. However, Delphi later denied the incident, stating the vehicles were a lane width apart and that both cars performed correctly. Google also confirmed the cars behaved appropriately in an ordinary driving scenario.
- Company involved
- Delphi Automotive
- AI system involved
- Delphi self-driving car (Audi Q5 SUV)
7 source articles · read the reporting →
Tesla Autopilot stops for phantom stop sign on digital billboard
Researchers at Ben-Gurion University demonstrated that Tesla's Autopilot could be tricked into stopping for a split-second phantom stop sign displayed on a digital billboard. The system falsely detected the depthless object and applied the brakes in the middle of the road. The researchers disclosed their findings to Tesla and Mobileye. No actual accident occurred during the test.
- Company involved
- Tesla
- AI system involved
- Tesla Autopilot
9 source articles · read the reporting →
California threatens legal action against Uber over self-driving cars without permit
Uber launched a pilot program of self-driving taxis in San Francisco without obtaining a required permit from the California DMV. Two of the vehicles were observed running red lights on the first day. The California attorney general threatened legal action unless Uber removes the cars from the road. Uber argued that its cars are not fully autonomous and therefore not subject to the permit requirement.
- Company involved
- Uber
10 source articles · read the reporting →
Answer.AI tests Devin and reports 14 failures in 20 tasks
Answer.AI's team tested Devin, an autonomous AI coding assistant, on 20 real-world tasks over a month. Devin succeeded in only 3 tasks, failed 14, and was inconclusive in 3. The team found Devin often produced overly complex or hallucinated solutions and could not recognize fundamental blockers. They ultimately decided to stick with tools that allow more human control.
- AI system involved
- Devin
5 source articles · read the reporting →
Cruise alleged to rely on human operators for autonomous driving
A New York Times article alleged that Cruise's autonomous vehicles rely on human operators to achieve autonomous driving, with 1.5 workers per vehicle intervening every 2.5 to five miles. Cruise CEO Kyle Vogt responded on Hacker News, stating that remote assistance is used 2-4% of the time and that many sessions are resolved by the vehicle itself. The company declined an interview request from the NYT.
- Company involved
- Cruise
- AI system involved
- Cruise AV
9 source articles · read the reporting →
Tesla Model 3 crash with truck kills driver using partial automation
At 6:17 a.m. on March 1, 2019, a Tesla Model 3 operating with partial driving automation struck a semitrailer in Delray Beach, Florida, killing the 50-year-old driver. The NTSB determined the probable cause was the truck driver's failure to yield right of way combined with the car driver's inattention due to overreliance on the automation, with Tesla's system design allowing disengagement as a contributing factor.
- Company involved
- Tesla
10 source articles · read the reporting →
Study finds driverless car AI worse at detecting darker-skinned pedestrians
A study by researchers from Georgia Tech found that facial recognition systems used in autonomous cars are up to 12% less accurate at detecting pedestrians with darker skin tones. The bias stems from a lack of diverse training data in the BDD100K database. The researchers warn that this could lead to higher accident rates for people with darker skin if not addressed.
- AI system involved
- BDD100K
9 source articles · read the reporting →
Tesla's Full Self-Driving involved in fatal Bay Bridge crash
On Thanksgiving 2022, a Tesla vehicle using its Full Self-Driving feature was involved in a fatal accident on the San Francisco Bay Bridge. Footage published by The Intercept showed the system braking abruptly. The incident highlights safety concerns with autonomous driving technology.
- Company involved
- Tesla
- AI system involved
- Full Self-Driving
7 source articles · read the reporting →
Waymo driverless car hits bicyclist in San Francisco intersection
On 6 February 2024, a Waymo driverless car collided with a bicyclist at a four-way intersection in San Francisco. The cyclist was obscured by a truck and the vehicle braked but could not stop in time. The cyclist sustained minor scratches and left the scene. Waymo stated it is contacting relevant authorities.
- Company involved
- Waymo
4 source articles · read the reporting →
Driver relying on "smart driving" on highway crashes after system suddenly disengages near truck
司机依赖“智驾 ”跑高速，逼近大货车时“智驾”突然退出引发车祸 - 手机新浪网
A driver was using a smart driving system on an expressway. As the vehicle approached a large truck, the system unexpectedly disengaged, leading to a crash.
1 source article · read the reporting →
Tesla Cybertruck on Full Self-Driving swerves into oncoming traffic
A Tesla Cybertruck with Full Self-Driving (FSD) activated attempted to turn left into a driveway while an SUV approached from the opposite direction, nearly causing a head-on collision. The driver seized control and avoided the crash. The owner reported the incident on a forum, expressing concern about using FSD again. No injuries or damage occurred.
- Company involved
- Tesla
- AI system involved
- Full Self-Driving (FSD)
3 source articles · read the reporting →
