Google Allo Smart Reply gives offensive responses

Google Allo's AI-powered Smart Reply feature generated racist, offensive or insensitive responses, prompting concerns about its effectiveness and safety . Smart Reply, a feature central to Google's Allo instant messaging mobile app, was designed to provide quick, contextually relevant responses based on previous conversations. However, the feature sometimes suggested racist, sexist, or otherwise discriminatory replies. In one instance, it suggested sending a "person wearing turban" emoji in response to a message that included a gun emoji. It was also critici s ed for generating generic and sometimes irrelevant responses in various contexts, such as serious conversations about breakups or job firings. Users complained that while the feature can be convenient, it often fail ed to handle nuanced emotional situations effectively . This and other complaints raised concerns about Allo's tendency to perpetuate and amplify existing biases and prejudices - a problem seen to emanate from the system's underlying machine learning algorithms which p roduced unexpected and problematic suggestions due to biases in training data or random errors . Google shuttered Allo in March 2019 after complaints about loss of privacy and other issues. System 🤖 Google Smart Reply Operator: Developer: Alphabet/Googl e Country: Global Sector: Multiple Purpose: Automate conversation responses Technology: Chatbot; NLP/text analysis; Neural network; Deep learning; Machine learning Issue: Bias/disc rimination; Safety

Date it happened

2017-10-01

Organisation involved

Google Allo users

Product, system or model

Google Allo; Smart Reply

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.