Top AI models generate misleading US election information 30 percent of the time

High-profile generative AI models produce inaccurate and misleading information about US Vice President Kamala Harris' candidacy for the presidency much of the time, according to research. Proof News tested five leading models - Meta’s Llama 3, Anthropic’s Claude 3, OpenAI’s GPT-4 , Mistral’s Mixtral 8, and Google’s Gemini 1.5 - on their ability to address common misinformation about Harris. The results showed that while these models provided accurate answers 70 percent of the time, they also produced misleading information that could confuse voters. Notably, Mixtral was the least reliable, with nearly half of its responses being incorrect. Issues arose particularly around Harris's eligibility for office and her racial background. For instance, Gemini provided a convoluted answer regarding her eligibility, suggesting complexity where there is none, as legal experts agree she is a natural-born citizen. Additionally, GPT-4 inaccurately attributed a Medicare cut claim to Harris, despite her not being a senator at the time of the vote in question. The research highlights the broader implications of AI-generated misinformation, especially in a politically charged environment leading up to the 2024 election, and notes that AI technologies can amplify false narratives, as seen with misleading AI-generated content shared by public figures. System 🤖 Claude Gemini GPT-4 Llama Mixtral Operator: Proof Developer: Google; Anthropic; Meta; Mistral; OpenAI Country: USA Sector: Politics Purpose: Generate electoral information Technology: Generative AI Issue: Accuracy/reliability; Mis/disinformation

Date it happened

2024-01-01

Organisation involved

Proof

Product, system or model

Claude; Gemini; GPT-4; Llama; Mixtral

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.