Thomson Reuters Fraud Detect 'incorrectly' identifies fraud
A fraud detection system developed by Thomson Reuters generated false fraud alerts, leaving hundreds of thousands of legitimate claimants without access to public benefits, according to a legal complaint. Based on a three-year investigation, a complaint (pdf) filed by privacy non-profit organisation EPIC alleged that Thomson Reuters unlawfully acquired data, including from social media, and used 'harmful AI practices' to build and operate Fraud Detect, an automated tool used to detect and prevent welfare and healthcare insurance fraud in at least 42 US states. The complaint also alleged that Fraud Detect regularly incorrectly flagged legitimate public benefits claims as fraudulent, leading to the wrongful reduction, denial, and recollection of public benefits for eligible recipients. Used by California's Employment Development Department during the COVID-19 pandemic to detect welfare fraud, Fraud Detect led to the suspension of 1.1 million claims, of which at least 600,000 were discovered to be legitimate. The complaint also stated that Thomson Reuters maintained direct c ontrol of Fraud Detect, including its source code, operation and maintenance, under many of its contracts. It also accused the company of witholding key information about the design, evaluation and operation of the system from government agencies and the general public. System 🤖 Fraud Detect 🔗 Operator: California Employment Development Department; Iowa Workforce Development Developer: Thomson Reuters Country: USA Sector: Govt - welfare Purpose: Detect and prevent fraud Technology: Risk assessment algorithm; Machine learning Issue: Accuracy/reliability; Accountability; Human/civil rights Regulation ⚖️ US Federal Trade Commission Act US Fair Credit Reporting Act Legal, regulatory 👩🏼⚖️ EPIC (2024). FTC Thomson Reuters complaint (pdf)
- Date it happened
- 2020-12-01
- Organisation involved
- California Employment Development Department; Iowa Workforce Development
- Product, system or model
- Fraud Detect
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.