Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.
Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.
236 incidents closest to “FRISS Fraud Detection at Claims” · matched on meaning · public reporting
Generated a legal brief with hallucinated case law, filed in court.
1 source article · read the reporting →
The AI system generated fabricated legal citations that were included in a court filing, misleading the court and opposing counsel.
1 source article · read the reporting →
A hospital is suing Blue Cross and Blue Shield of Kansas City, alleging the insurer's AI flagged physicians' diagnoses as clinically invalid and led to the denial of about $2 million in claims.
1 source article · read the reporting →
A class action, settled for $30 million, alleged that Equifax automatically rejected consumers' disputes of hard inquiries on their credit reports, sending a form letter instead of investigating.
1 source article · read the reporting →
The AI-generated voice deceived the chairman into authorizing transfers of about 95 million euros.
1 source article · read the reporting →
Police falsely arrested an innocent man due to a facial recognition misidentification
1 source article · read the reporting →
The system excluded a transgender woman from a women-only social networking app based on her facial appearance.
1 source article · read the reporting →
Denied post-acute care coverage to elderly Medicare Advantage patients
1 source article · read the reporting →
A generative AI tool drafted a temporary restraining order in Jackson Federation of Teachers v. Fitch containing fabricated parties, declarations, and statutory language, affecting the state of Mississippi and the plaintiffs.
1 source article · read the reporting →
Identified Christopher Gatlin as a suspect in an assault, leading to his arrest and 17 months in jail.
1 source article · read the reporting →
Mr Craig Hadley was wrongly accused of being a fraudster at a Rotherham Sports Direct shop using facial recognition technology supplied by Facewatch. The system flagged him as a known fraudster, but the alert was later attributed to human error. Big Brother Watch, a privacy campaign group, has raised concerns about the lack of due process in such systems.
2 source articles · read the reporting →
The Allegheny Family Screening Tool, an algorithm used to screen child neglect reports, has been found to flag a disproportionate number of Black children for mandatory investigation compared to white children. A technical glitch also presented wrong risk scores for over two years. County officials say social workers can override the algorithm and that the glitch has been fixed, but critics argue the tool hardens racial disparities and lacks transparency.
10 source articles · read the reporting →
Pro Se Litigant | D. Arizona | State(s): USA. AI tool: Clearpoint. Problem: Fabricated: Case Law. Consequence: Bar Referral.
1 source article · read the reporting →
Law enforcement in Akron, Ohio, used the AI tool Cybercheck to place two defendants at a murder scene. Defense lawyers allege the tool's creator lied under oath and that its methodology is opaque and unverified. Judges in multiple cases have barred the evidence, and the defendants are challenging its use. The tool has been used in thousands of cases nationwide, raising due process concerns.
3 source articles · read the reporting →
The AI generated fake legal citations that were included in an appellate brief, affecting the court and the client.
1 source article · read the reporting →
The AI generated false legal citations that were included in a court filing, misleading the court.
1 source article · read the reporting →
A child protection worker at the Department of Families, Fairness and Housing (DFFH) used ChatGPT to draft a Protection Application Report for the Children’s Court, entering sensitive personal information about a child. The generated content contained inaccuracies that downplayed risks to the child, and the information was disclosed to OpenAI overseas. An investigation by the Office of the Victorian Information Commissioner found DFFH failed to ensure accuracy and protect personal information, contravening IPPs 3.1 and 4.1. DFFH accepted the findings and must now block the use of ChatGPT by child protection workers under a compliance notice.
9 source articles · read the reporting →
A middle-aged woman in Indore, India, lost her entire savings of Rs 97,500 after a fraudster used AI voice cloning to impersonate her cousin, a police officer. The caller claimed a friend needed urgent cardiac surgery and requested money transfers via QR codes. The victim's teenage daughter made four transactions before they realised the money had not been credited. Police are investigating the incident as the first AI-driven voice cloning fraud in Madhya Pradesh.
3 source articles · read the reporting →
ProPublica's analysis of auto insurance premiums in California, Illinois, Texas and Missouri found that major insurers such as Geico, Allstate and Liberty Mutual charge minority neighborhoods up to 30% more than white areas with similar accident costs. The disparity is alleged to result from pricing algorithms that may inadvertently use zip code as a proxy for race. Otis Nash, a 26-year-old Chicagoan, pays $190.69 per month for Geico insurance while a white driver in a safer neighborhood pays $54.67. The insurers deny discrimination, stating they do not collect race data.
5 source articles · read the reporting →
In 2022, Juliette, a single mother on welfare in France, was flagged by CNAF's secretive fraud detection algorithm. A fraud investigator later determined she owed thousands of euros, which were deducted from her monthly payments. The algorithm, which scores half of France's population, is accused of discriminating against vulnerable people by using factors like single parenthood and low income.
10 source articles · read the reporting →
Whitebridge AI, a Lithuanian company, is accused of generating false reputation reports using unlawfully scraped social media data. The reports contained false warnings for 'sexual nudity' and 'dangerous political content'. Privacy group Noyb filed a complaint with the Lithuanian data protection authority, alleging violations of the GDPR. The complainants sought access to their data but received no response, and were later required to provide a qualified electronic signature to correct errors.
6 source articles · read the reporting →
In May 2015, Poland's Ministry of Labor and Social Policy introduced an algorithmic system to profile unemployed people into three categories based on their readiness to work, location, and disabilities. The system is accused of being discriminatory and lacking transparency, as it uses default criteria that may lead to social exclusion. A report by the Panoptykon Foundation in October 2015 highlighted these issues and recommended reforms, including granting the right to appeal.
10 source articles · read the reporting →
Upstart Holdings, Inc., a cloud-based AI lending platform, is accused of making false and misleading statements about its AI model's ability to account for macroeconomic factors. The lawsuit, filed on May 13, 2022, alleges that the company's positive statements about its business were materially false. Investors suffered losses when Upstart's stock price fell 56% on May 9, 2022, after the company reduced its fiscal 2022 guidance.
10 source articles · read the reporting →
The Oregon Department of Human Services announced it will stop using its Safety at Screening Tool algorithm, which helped hotline workers decide which families to investigate for child abuse and neglect. The decision followed concerns about racial bias and disparities, as the algorithm was inspired by a Pennsylvania tool that had flagged a disproportionate number of Black children. The agency will replace the algorithm with a new Structured Decision Making model.
10 source articles · read the reporting →
