Study: Facial recognition software misidentifies dark-skinned women 35% of the time
Facial recognition technology exhibits significant gender and racial bias, with error rates up to 35 percent for dark-skinned women compared to less than 1 percent for light-skinned men, according to researchers. What happened MIT and Stanford University researchers Joy Buolamwini and Timnit Gebru conducted a study on commercial facial-analysis programmes, revealing large disparities in accuracy across different demographic groups. They found that error rates for gender classification were consistently higher for females than males, and for darker-skinned subjects than lighter-skinned ones. For darker-skinned women, error rates reached as high as 34.7 percent, while for light-skinned men, they never exceeded 0.8 percent. Why it happened Bias in facial recognition algorithms stems from inadequate diversity in the training datasets, with many datasets not sufficiently diverse to enable the algorithms to learn to correctly identify non-white faces. The problem is rooted in the historical underrepresentation of people of color and women in largescale datasets used to train and benchmark these systems. Additionally, there has been a widespread lack of attention paid by model designers to measuring performance disparities for historically marginalised groups. What it means B ias in facial recognition technology are seen to perpetuate and amplify existing racial and gender inequalities, with a real potential impact on law enforcement, immigration and other sectors, where misidentification can lead to wrongful arrests, lengthy detentions, and even deadly police violence . System 🤖 Cognitive Services Face API Face++ 🔗 Watson Visual Recognition API Developer: IBM; Megvii; Microsoft Country: USA Sector: Govt - police ; Govt - immigration Purpose: Identify identity Technology: Facial recognition Issue: Accuracy/reliability; Bias/discrim ination
- Product, system or model
- Cognitive Services Face API; Face++; Watson Visual Recognition API
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.