Where automated decisions went wrong

NHTSA investigation PE24016: Unexpected ADS behavior

Waymo's automated driving system exhibited unexpected behavior during driving.

Company involved

Waymo

1 source article · read the reporting →

Consumer Reports finds Tesla Autopilot operates without driver

Consumer Reports engineers tested a Tesla Model Y on a closed track and found that the Autopilot system could operate the vehicle without anyone in the driver's seat, failing to warn or detect the absence of a driver. The test highlighted a safety flaw as investigators probed a fatal crash in Texas where two men died in a Tesla Model S believed to be operating without a driver. Tesla did not respond to a request for comment.

Company involved

Tesla

AI system involved

Autopilot

10 source articles · read the reporting →

Tesla on Autopilot Crashes into Overturned Truck on Taiwan Highway

On 1 June 2020, a Tesla sedan with Autopilot engaged failed to detect an overturned delivery truck on National Highway 1 in Taiwan and collided with it. The driver, a 53-year-old man, had set the system to 110 km/h and assumed it would avoid obstacles, but he braked too late. The truck driver, who was standing on the highway after his own accident, narrowly avoided being struck. No serious injuries were reported, and police are investigating the cause, with possible factors including the truck's unusual shape and the system's limitation to detecting only moving vehicles.

AI system involved

Autopilot

3 source articles · read the reporting →

San Jose is training AI to spot homeless encampments in first US pilot

San Jose has been running a pilot in which cameras on a municipal vehicle feed computer vision software being trained to detect lived-in vehicles and homeless encampments in District 10. The scheme, believed to be the first of its kind in the US, has prompted concern among housing advocates that it could be used to punish and displace unhoused residents. The city says the aim is to respond more efficiently to complaints about encampments and that the footage is not being actively monitored for law enforcement.

Company involved

City of San Jose

10 source articles · read the reporting →

PredictiveHire builds AI to predict job hopping from interviews

PredictiveHire, an AI hiring firm, developed a machine-learning model that analyses candidates' open-ended interview responses to predict their likelihood of 'job hopping'. The company used data from 45,899 applicants to build the 'flight risk' assessment, which it advertises as coming soon. Scholars warn that such tools can suppress wages by screening out workers who might seek better pay or conditions, continuing a historical trend of using personality tests to identify potential labour organisers.

Company involved

PredictiveHire

AI system involved

Phai

1 source article · read the reporting →

Tesla Full Self-Driving system fails to recognise train, driver intervenes

A Tesla owner in Ohio reported that his car in Full Self-Driving mode attempted to drive into a crossing train on May 8, 2024. The driver took control and avoided a collision, but the car hit the crossing arm causing damage. The driver said it was the second such incident in six months. Tesla did not respond to requests for comment.

Company involved

Tesla

AI system involved

Full Self-Driving (FSD)

8 source articles · read the reporting →

Senators demand review of VA's AI-driven contract cancellations

The Department of Veterans Affairs used an AI tool created by a Department of Government Efficiency employee to identify hundreds of contracts for cancellation. Senators Richard Blumenthal and Angus King have called for the VA Inspector General to investigate the use of AI in these decisions, alleging that the tool used flawed formulas and that the cancellations are harming veterans by cutting services. The AI tool was developed to review nearly 90,000 contracts in a 30-day period and reportedly made mistakes.

Company involved

Department of Veterans Affairs

8 source articles · read the reporting →

Cruise Self-Driving Car Rear-Ended by Human-Driven Cruise Vehicle in San Francisco

On 11 June 2018, a self-driving Cruise Chevrolet Bolt was rear-ended by another Cruise vehicle driven by a human employee on Bryant Street in San Francisco. The collision caused only minor scuffs and no injuries. Cruise reported the incident to the California DMV as required. The incident highlights the challenges of autonomous vehicles interacting with human drivers.

Company involved

Cruise

2 source articles · read the reporting →

AI-powered tram crashes into pedestrians in St Petersburg test run

An AI-powered 'Smart Tram' on a test run in St Petersburg, Russia, crashed into pedestrians after its AI system turned off and brakes failed. The driver, who was present, attempted to use emergency brakes but they did not fully work. Several people were injured, including a 45-year-old woman trapped under the tram. The city authorities have launched an investigation and a criminal case has been opened.

Company involved

St Petersburg city authorities

AI system involved

Smart Tram

1 source article · read the reporting →

Google and Delphi Self-Driving Cars Involved in Minor Accidents in California

Four self-driving cars in California have been involved in accidents since September 2014, when the state began issuing testing permits. Two occurred while the cars were in autonomous mode, and two while a human was driving. Google and Delphi, the companies operating the vehicles, stated that their cars were not at fault and that the collisions were minor, with no injuries. The accidents have raised concerns about transparency, as details are kept confidential under state law.

Company involved

Google Inc. and Delphi Automotive

10 source articles · read the reporting →

Google Cloud Used in CBP AI Virtual Border Wall Contract

The Intercept reported that U.S. Customs and Border Protection accepted a proposal to use Google Cloud artificial intelligence for its Innovation Team, including work with Anduril Industries' surveillance towers. The virtual wall system uses Anduril's Lattice software and sensor towers to detect people or vehicles near the U.S.-Mexico border and relay their locations to agents. Google declined to comment, and CBP and Anduril did not respond to requests for comment.

Company involved

U.S. Customs and Border Protection (CBP)

AI system involved

Google Cloud AI Platform with Anduril Lattice and Sentry Towers

1 source article · read the reporting →

Tesla Model X on Autopilot hits crash attenuator in Mountain View, killing driver

On 23 March 2018, a Tesla Model X P100D operating with Autopilot hit a damaged crash attenuator on US Highway 101 in Mountain View, California, killing the driver. The NTSB found that Autopilot steered the vehicle into a highway gore area because of system limitations, and that the driver was distracted and over-reliant on the partial driving automation. Two other vehicles were struck; one driver sustained minor injuries.

Company involved

Tesla

AI system involved

Autopilot

9 source articles · read the reporting →

Beep automated bus rear-ended by Tesla in Washington D.C.

An automated bus operated by Beep was rear-ended by a Tesla driver making an illegal lane change in Washington D.C. on 18 January 2026. The bus, part of a pilot programme with the U.S. Department of Transportation, had a human backup driver and sustained minor cosmetic damage. Service was temporarily paused but resumed after a review found the bus operated appropriately. No police were called and the incident was not reported to NHTSA as the damage was under $1,000 and the other driver was at fault.

Company involved

Beep

AI system involved

Beep automated bus

2 source articles · read the reporting →

San Francisco officials complain of robotaxi interference with emergency responders

San Francisco's fire chief and police union have complained that Waymo and Cruise robotaxis are interfering with emergency responders, including driving into active scenes and blocking fire trucks. The fire chief says it is only a matter of time before a catastrophic injury or death occurs. The police union asked the California Public Utilities Commission to postpone a vote on expanding robotaxi services. The companies say they are working with officials and that most encounters are without issue.

Company involved

Waymo and Cruise

4 source articles · read the reporting →

Waymo Self-Driving Taxis Crash into Backwards-Towed Pickup in Phoenix

On 11 December 2023, two Waymo autonomous vehicles collided with a pickup truck being towed backwards in Phoenix, Arizona. The vehicles were operating without passengers and sustained minor damage, with no injuries reported. Waymo attributed the crashes to a software error that mispredicted the towed vehicle's motion and subsequently deployed a fix. The company filed a voluntary recall report with the NHTSA.

Company involved

Waymo

AI system involved

Waymo Driver

3 source articles · read the reporting →

VIDEO: Wayward Waymo runs red light at major intersection near Universal Orlando - WKMG

The Waymo van entered an intersection against a red light, endangering cross traffic and pedestrians.

Company involved

Waymo

1 source article · read the reporting →

Austin Fire Department Faces Another Bizarre Standoff With an Empty Waymo - MotorBiscuit

A driverless car failed to yield to an emergency vehicle, blocking its path

Company involved

Waymo

AI system involved

Waymo

1 source article · read the reporting →

Waymo Robotaxi Blocks Fire Truck Responding To Austin Hotel Alarm - AOL.com

An empty Waymo robotaxi blocked and delayed a fire truck responding to an Austin hotel alarm.

Company involved

Waymo

AI system involved

Waymo robotaxi

1 source article · read the reporting →

NHTSA investigation PE25012: Traffic safety violations while Full Self Driving ("FSD") is engaged

The system made driving decisions that violated traffic safety laws, affecting other road users.

Company involved

Tesla, Inc.

AI system involved

Full Self Driving (FSD)

1 source article · read the reporting →

Waymo Self-Driving Taxis Proactively Report to Police After Discovering Passenger Hiding a Gun

قامت سيارات الأجرة ذاتية القيادة من شركة Waymo بإبلاغ الشرطة بشكل استباقي بعد اكتشافها أن أحد الركاب كان يخفي مسدساً.…

A Waymo self-driving taxi in San Francisco detected two teenage passengers carrying an unidentified AR-15 rifle loaded with ammunition. The vehicle automatically stopped and alerted police, leading to the arrest of the two teenagers. Waymo stated that its system can identify violations of its terms of service, including carrying weapons, and will proactively notify police.

Company involved

Waymo

1 source article · read the reporting →

Woman Blindly Trusts Driver Assistance, Rear-Ends Lane-Changing Bus on Highway

女子轻信辅助驾驶放任不管 高速上直接追尾变道大巴车 - 驱动之家

On August 28, on the Shanghai-Kunming Expressway, a woman driving an SUV with driver assistance engaged rear-ended a bus that suddenly changed lanes. She assumed the system would brake automatically and took no action. The bus driver was found at fault for the lane change, but police warned against over-reliance on driver assistance.

1 source article · read the reporting →

Investigation probes autopilot failure after fatal Tesla accident in the United States

Investigação apura falha no piloto automático após acidente fatal com Tesla nos Estados Unidos - dfmobilidade.com.br

A fatal crash involving a Tesla Model 3 occurred at an intersection in Buena Vista Township, New Jersey. Authorities are investigating a possible failure of the Autopilot system and whether it was active at the time, adding to scrutiny of Tesla's driver-assistance technology.

Company involved

Tesla

AI system involved

Autopilot

1 source article · read the reporting →

Sibuxiang Genuine + Genuine Sibuxiang: Tesla Autopilot in Another Accident, Owner Says System Failed to Recognize Construction Barriers, Causing Rear-End…

四不像正版+正版四不像特斯拉自动驾驶再出事故，车主称系统未识别施工路障致追尾 - Pchome电脑之家

A Tesla owner reported that the vehicle's Autopilot system did not detect construction barriers, leading to a rear-end collision. The incident was reported by Pchome.

Company involved

Tesla

AI system involved

Autopilot

1 source article · read the reporting →

NHTSA investigation PE24015: Rear-end Collisions Involving Zoox Vehicles

Automated driving system controlled vehicle braking and acceleration, leading to rear-end collisions with other road users

Company involved

Zoox

AI system involved

Zoox vehicle

1 source article · read the reporting →