ChatGPT reproduces recommendation letter gender bias
AI tools such as ChatGPT and Alpaca contain 'significant' gender biases when asked to produce recommendation letters for hypothetical employees , according to University of California researchers. Asked to produce recommendation letters for hypothetical models, the researchers found that large language model chatbots ChatGPT and Stanford University's Alpaca used 'very different' language to describe imaginary male and female workers. For men, ChatGPT used nouns such as 'expert' and 'integrity', while calling women a 'beauty' or 'delight.' Alpaca described men as 'listeners' and 'thinkers,' while women had 'grace' and 'beauty.' The bias is thought likely to reflect historial records, which tended to be written by men and depicted them as 'active workers' as opposed to women, who were often seen as 'passive objects'. It may also reflect the fact that LLMs have been trained on data from the internet, where men spend more time than women, according to the ITU. System 🤖 ChatGPT Operator: Yixin Wan, George Pu, Jiao Sun, Aparna Garimella, Kai-Wei Chang, Nanyun Peng Developer: OpenAI; Stanford University Country: USA Sector: Business/professional services Purpose: Generate text Technology: Chatbot; Generative AI; Machine learning Issue: Bias/discrimination - gender
- Date it happened
- 2023-11-01
- Organisation involved
- Yixin Wan, George Pu, Jiao Sun, Aparna Garimella, Kai-Wei Chang, Nanyun Peng
- Product, system or model
- Alpaca; ChatGPT
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.