Lemonade use of emotion recognition to assess insurance claims prompts backlash

New York-based digital insurance firm Lemonade sparked an ethics and privacy controversy after boasting that its AI analysed "non-verbal cues" in customer videos to detect fraud, a move that critics denounced as the modern-day equivalent of "digital phrenology." Lemonade Inc claimed on Twitter that its 'AI Jim' claims bot uses emotion recognition to automate the process of assessing and denying insurance claims based on "non-verbal cues" and "1,600 data points" in videos its customers must shoot to explain what had happened. The claim triggered intense criticism of the company, with people accusing it of potential bias and of conducting phrenology and other forms of discredited pseudoscience. Lemonade was forced to deny using AI or emotion recognition to 'automatically decline claims', going on to add that 'harmful concepts like phrenology and physiognomy has never, and will never, be used at Lemonade.' The company had previously stated in a S-1 filing that its system collects approximately 1,700 customer data points from customers. It later said that this is 'about 100 times more data than traditional data carriers'. System 🤖 AI Jim Developer: Lemonade Inc Country: USA Sector: Banking/financia l services Purpose: Assess and process insurance claims Technology: Facial recognition; Emotion recognition; Machine learning Issue: Accuracy/reliability; Fairness; Privacy/surveillance; Transparency Timeline ⏰ May 24, 2021: Lemonade posts a Twitter thread bragging that its AI analyses "non-verbal cues" and "1,600 data points" in user-submitted claim videos to detect fraud. May 26, 2021: The thread goes viral, drawing intense criticism from AI ethics researchers (e.g., Timnit Gebru) who label it "digital phrenology." May 27, 2021: Lemonade deletes the "awful" tweets and issues a blog post clarification, claiming "non-verbal cues" was a poor choice of words for identity verification. July 15, 2021: A class-action lawsuit is filed in Illinois (Clarke v. Lemonade Inc.), alleging the company collected biometric data without consent, violating the Illinois Biometric Information Privacy Act (BIPA). August 2021: A second major class-action lawsuit (Pruden v. Lemonade Inc.) is filed in New York, alleging deceptive trade practices regarding biometric privacy. May 17, 2022: Lemonade files a memo in court proposing a USD 4 million settlement t o resolve the consolidated biometric privacy lawsuits. June 2022: The settlement website goes live. Lemonade agrees to delete all previously collected biometric data (faceprints) and ceases the use of facial geometry for fraud detection. August 25, 2022: A final approval hearing is held for the $4 million settlement, covering over 110,000 customers nationwide. September 2023: A new class-action investigation begins regarding Lemonade’s alleged disclosure of user health data to third-party trackers (Meta/TikTok). April 3, 2024: A new lawsuit (La Febre v. Lemonade Inc.) is filed, targeting the sharing of "protected health information" from life insurance applications with third parties. October 2024: Preliminary court approval is granted for a USD 4.99 million settlement regarding these broader privacy violations. January 21, 2025: Deadline for class members to file claims in the USD 4.99M privacy settlement. February 2, 2025: The EU AI Act begins prohibiting "Unacceptable Risk" AI practices, including certain emotion recognition systems in workplaces and education—a move heavily influenced by the 2021 Lemonade controversy. May 16, 2025: Extended deadline for claims in the La Febre privacy settlement. June 10, 2025: Final approval hearing for the USD 4.99 million privacy settlement in New York. Resource s 📃 Lemonade 'non-verbal cue' tweets Lemonade. Lemonade’s Claim Automation Lemonade. S-1 SEC filing

Date it happened

2021-05-01

Organisation involved

Lemonade Inc

Product, system or model

AI Jim

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.