Study: AI chatbots fail disabled voters
High-profile chatbots provide inaccurate and misleading information about how disabled people can vote in the US, potentially dissuading disabled people from voting and sowing doubts about electoral integrity. Out of 77 prompts tested across five different chatbots - Mixtral 8x7B v0.1, Gemini 1.5 Pro, ChatGPT -4 , Claude 3 Opus, and Llama 2 70b - 61 percent of responses exhibited at least one insufficiency. Notably, over one third contained incorrect information, which ranged from minor issues such as broken links to significant misinformation about voter registration deadlines and voting procedures. Approximately 25 percent of the responses could potentially dissuade or hinder users from exercising their voting rights. This included inaccuracies about available voting methods, particularly concerning internet and curbside voting. In addition, all chatbots demonstrated a tendency to "hallucinate," generating entirely fabricated information about laws and organizations related to voting and disabilities. And many responses lacked necessary nuance, failing to clarify when polling places are accessible or misinterpreting key terms associated with voting methods. When asked for authoritative information, nearly half of the answers were incorrect. Furthermore, the chatbots showed susceptibility to misinformation tactics, sometimes providing misleading information in response to simulated queries from bad actors. The report underscores the urgent need for improved accuracy and reliability in AI-generated information regarding voting, especially for vulnerable populations like those with disabilities. Hallucination (artificial intelligence) In the field of artificial intelligence (AI), a hallucination or artificial hallucination (also called bullshitting, confabulation or delusion) is a response generated by AI that contains false or misleading information presented as fact . Source: Wikipedia 🔗 System 🤖 ChatGPT -4 Claude 3 Opus Gemini 1.5 Pro Llama 2 70b Mixtral 8x7B v0.1 🔗 Operator: Developer: Anthropic; Google; Meta; Mistral; OpenAI Country: USA Sector: Politics Purpose: Provide disab led voter information Technology: Chatbot; Machine learning Issue: Accuracy/reliability; Mis/disinformation
- Date it happened
- 2024-07-01
- Product, system or model
- Mixtral 8x7B v0.1; Gemini 1.5 Pro; ChatGPT-4; Claude 3 Opus; Llama 2 70b
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.