SimCLR, iGPT image generation systems found to contain racial bias
Prominent image generation algorithms, including Google’s SimCLR and OpenAI’s iGPT, are biased and prone to negative stereotyping, according to a research study . Using a new diagnostic tool called the Image Embedding Association Test (iEAT) to audit two state-of-the-art computer vision models: SimCLR (developed by Google) and iGPT (developed by OpenAI), r esearchers Ryan Steed and Aylin Caliskan revealed that both models, which learn by scanning millions of unlabeled images from the internet, did not just learn to recognise objects; they absorbed social prejudices. Specifically, the models exhibited: Racial bias: Stronger associations between "White" faces and "pleasant" attributes compared to "Black" faces. Gender bias: Stereotypical associations of "women" with "home" and "arts," while "men" were more strongly linked to "science" and "business." The researchers showed iGPT a head shot of prominent US politician Alexandria Ocasio-Cortez ('AOC') wearing business attire, only for the software to recreate her mutiple times in a bikini or low-cut top. Intersectional harms: The biases were compounded for individuals at the intersection of marginalized identities, such as Black women. These findings were significant because unsupervised (or self-supervised) models were previously thought to be "cleaner" than supervised models since they don't rely on human-labeled data - which is a known source of bias. Meantime, the researchers were criticised for including AI generated images of AOC in the pre-print paper, thereby exposing her to additional potential sexualisation and other possible abuses. System 🤖 SimCLR 🔗 Image GPT 🔗 Developer: Alphabet/Google; OpenAI Country: USA Sector: Multiple; Research/academia Purpose: Generate images Technology: Image generation; Neural network; Deep learning; Machine learning Issue: Accuracy/reliability; Bias/discrimination
- Date it happened
- 2021-01-01
- Organisation involved
- Google; OpenAI
- Product, system or model
- iGPT; SimCLR
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.