Genderify AI identification tool accused of gender bias
AI gender identification tool Genderify faced accusations of gender bias before being quickly shut down. An AI-powered API service launched in July 2020, Genderify claimed to be able to determine a person's gender from their name, email address or username and was was marketed as a tool for businesses to personalise communications and target marketing. However, Genderify was critici s ed for producing biased and stereotypical results almost immediately after launch, including misgendering people with non-Western names and demonstrating clear biases, such as associating terms like "doctor" with male and "nurse" with female. In addition, the company's API returned probabilities for only binary male/female options, ignoring non-binary identities. Genderify was taken offline after a few days of intense criticism on social media, and the company's website and social media accounts deleted without explanation. The episode was seen as a cautionary tale about the need for diverse teams and extensive bias testing in the development of AI systems. System 🤖 Genderify Developer: Gen derify Country: USA Sector: Techno logy Purpose: Identify gender Technology: NLP/text analysis; Machine learning Issue: Accuracy/reliability; Fairness
- Date it happened
- 2020-07-01
- Product, system or model
- Genderify
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.