Google sentiment analyser thinks being gay is bad
A Google sentiment analysis tool exhibited bias against LGBTQ+ identities, raising concerns that it may perpetuate harmful stereotypes and discrimination. What happened A Vice investigation found that Google Cloud's sentiment analysis system showed evidence of bias against queer terms, assigning more negative sentiment scores to sentences containing LGBTQ+ identities compared to non-queer identities. This bias could lead to representational and allocational harms for marginalised groups, such as misrepresentation of their views or unfair allocation of resources if the tool is used for triaging or decision-making processes. Why it happened The bias likely stems from the tool's training on existing text data, which may contain societal prejudices and negative portrayals of LGBTQ+ individuals. Google attempted to mitigate this by implementing an "ignore list" for certain identity terms, but this approach is incomplete and can lead to contextual misunderstandings. What it means For LGBTQ+ individuals, Google's tool may contribute to feelings of marginalisation and stigmatisation, particularly among youth who are already at higher risk of discrimination and mental health issues. For society at large, it underlines the need for greater awareness of AI biases and the importance of diverse representation in technology development in order to ensure fairer and more equitable treatment of all groups. System 🤖 Cloud Natural Language API Developer: Google Country: USA Sector: Multiple Purpose: Analyse sentiment Technology: NLP/text analysis Issue: Bias/discrimination
- Date it happened
- 2017-01-01
- Organisation involved
- Product, system or model
- Cloud Natural Language API
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.