Tesla Model S tricked into veering into wrong lane
Researchers from Tencent's Keen Security Lab adversarily tricked a Tesla Model S into driving into a lane of oncoming traffic. The exploit worked by using small, inconspicuous stickers that tricked the car's Enhanced Autopilot into detecting and then following a change in the current lane, raising questions about the safety and efficacy of Tesla's driver assistance system. As Technology Review 's Karen Hao noted , 'Tesla’s Autopilot is vulnerable because it recognizes lanes using computer vision. In other words, the system relies on camera data, analyzed by a neural network, to tell the vehicle how to keep centered within its lane.' Tesla welcomed the attack, adding that the adversarial attack was unrealistic 'given that a driver can easily override Autopilot at any time by using the steering wheel or brakes and should always be prepared to do so, and can manually operate the windshield wiper settings at all times.' System 🤖 Tesla Autopilot, Full-self Driving Operator: Keen Security Lab Developer: Tesla Country: USA Sector: Automotive Purpose: Automate steering, acceleration, braking Technology: Driver assistance system Issue: Accuracy/reliability; Safetyox
- Date it happened
- 2019-04-01
- Product, system or model
- Tesla Autopilot
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.