Study: ChatGPT "systematically" amplifies global inequalities

An audit of 20 million queries found that ChatGPT systematically attributes positive traits such as "intelligence" and "safety" to wealthy Western countries while ranking low-income nations at the bottom, thereby reinforcing cultural stereotypes. What happened Researchers from the Oxford Internet Institute (University of Oxford) and University of Kentucky analysed over 20 million English language ChatGPT queries to see how the model responds to comparisons about people, places, and quality-of-life attributes. They found consistent patterns where higher-income regions such as the United States, Western Europe and parts of East Asia were presented more positively (e.g., “smarter,” “happier” or “safer”), while many countries in Africa, the Middle East, and parts of Asia and Latin America tended to be ranked at the bottom. The research visualised these disparities on world maps and neighbourhood-level comparisons (e.g., in cities like London and Rio de Janeiro) showing that ChatGPT’s outputs aligned more with existing socioeconomic and racial divides than with substantive measures of quality or merit. Why it happened According to the researchers, these outcomes don’t reflect random error but stem from how fundamental biases in the training data - shaped by centuries of unequal information production, digital visibility, and power hierarchies - get amplified by large language models. Because ChatGPT learns from massive datasets that disproportionately represent Western, high-income areas and English-language sources, it tends to mirror that imbalance in its answers. The study describes this as a “silicon gaze” bias, a worldview shaped by the priorities of the developers, platform owners, and training data that trained the model. What it means For society: Treating AI as a neutral source of knowledge risks "laundering" old prejudices through a high-tech interface. If a student or business leader asks an AI for advice on where to invest or which culture is "innovative," the AI functions as a feedback loop that rewards existing wealth and punishes historical disadvantage. For policymakers: There is an urgent need for mandatory public bias audits. Policy frameworks (like those in the EU and US) must shift from viewing AI as a "tool" to viewing it as a "situated" entity shaped by power. Governments may need to mandate "provenance" for AI-generated claims about specific cultures or regions. For the public: The study warns against "taking responses at face value." Users must be educated in digital literacy to understand that AI offers a skewed map of the world, not a mirror of reality. System 🤖 ChatGPT Developer: OpenAI Country: Multiple Sector: Multiple Purpose: Assess intelligence, happiness, quality of life Technology: Generative AI Issue: Bias/discrimination; Representation

Date it happened

2025-01-01

Product, system or model

ChatGPT

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.