Microsoft Tay chatbot generates offensive tweets

Tay, an AI chatbot released by Microsoft that was programmed to respond to other Twitter users and caption photographs provided to it , was manipulated to generate racist, homophobic and anti-semitic tweets. D esigned by Microsoft to mimic a 19-year-old American girl, Tay was meant to improve itself by learning from interacting with human beings on Twitter . However, Twitter users started trolling the bot, resulting in it spewing tens of thousands of racist, homophobic and anti-semitic tweets. Microsoft deleted unsafe tweets and then suspended the bot's Twitter profile, saying it suffered from a 'coordinated attack by a subset of people' that had 'exploited a vulnerability in Tay.' A few days later, Microsoft accidentally re-released Tay on Twitter while testing it, only for it to get stuck in a repetitive loop, tweeting 'You are too fast, please take a rest'. Microsoft pulled the bot a few hours later and apologised . The fracas resulted in comm ents about the p oor quality of Tay ' s safety guardrails and questions about Microsoft's artificial intelligence capabilities. System 🤖 Microsoft Tay (Wayback machine) Documents 📃 Microsoft (2016). Learning from Tay’s introduction Operator: Microsoft Developer: Microsoft Country: USA Sector: Media/entertainment/sports/arts Purpose: Train language model Technology: Chatbot; NLP/text analysis; Deep learning; Machine learning Issue: Bias/discrimination - race, ethnicity , gender, religion; Safety Transparency: Governance; Black box

Date it happened

2016-03-01

Organisation involved

Microsoft

Product, system or model

Tay

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.