The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

151 incidents closest to “HMS coordination of benefits / third party liability services” · matched on meaning · public reporting

WF-DFMQ9J1 Mar 2024

Insurer drops California homeowner after aerial photo analysis

Cindy Picos was dropped by her home insurer after the company used aerial photos to assess her roof. The insurer refused to let her see the photos, and she believes her roof is in fine shape. The incident highlights the growing use of drones and aerial imagery by insurers to inspect properties without in-person visits.

2 source articles · read the reporting →

Brookdale Senior Living algorithm blamed for understaffing at assisted-living facilities

Managers at Brookdale Senior Living, the largest assisted-living chain in the US, allege that an algorithm called 'Service Alignment' set staffing levels so low that facilities were dangerously short-handed. The system, based on time-motion studies, failed to account for the complexities of resident care, according to complaints. Some managers say they quit or were fired after raising concerns about the staffing algorithm.

Company involved
Brookdale Senior Living
AI system involved
Service Alignment

1 source article · read the reporting →

WF-E5X8GK1 Jan 2017

University of Chicago Medical Center sued over sharing patient data with Google

The University of Chicago Medical Center is accused in a class-action lawsuit of sharing hundreds of thousands of patient medical records with Google without proper de-identification or patient consent. The data, from patients treated between 2009 and 2016, allegedly included dates and provider notes that could allow re-identification. The lawsuit claims Google sought the records to develop its own electronic health record system and predictive models. The hospital and Google deny wrongdoing, stating the research partnership was legal and compliant with HIPAA.

Company involved
University of Chicago Medical Center

1 source article · read the reporting →

iTutorGroup to Pay $365,000 to Settle EEOC Discriminatory Hiring Suit

The system automatically rejected tutor applicants aged 55+ (women) or 60+ (men).

Company involved
iTutorGroup

1 source article · read the reporting →

WF-IWL4TH1 Jan 2018

US Secret Service bought access to cellphone location data

The US Secret Service signed a contract to access Locate X, a service that aggregates location data from phone apps and allows law enforcement to track devices without a warrant. The contract with Babel Street was worth about $36,000 and ran from 2017 to 2018. A former employee said the Secret Service used Locate X in 2018 to seize illegal credit card skimmers at petrol stations. Lawmakers and civil liberties advocates criticised the practice, and Senator Ron Wyden introduced a bill to ban such purchases.

Company involved
United States Secret Service
AI system involved
Locate X

1 source article · read the reporting →

WF-CBSEF51 Jan 2017

VA's Suicide Prevention Algorithm Favours Men, Ignores Sexual Trauma

The Department of Veterans Affairs' REACH VET algorithm, designed to identify veterans at high risk of suicide, reportedly prioritises white men and excludes factors like military sexual trauma that disproportionately affect women. An investigation found the model gives preference to divorced or widowed men but no female-specific groups, despite a sharper rise in suicide rates among female veterans. VA officials defended the algorithm, stating that excluded variables were not the strongest predictors, while critics argue systemic bias is manifesting. The agency is working on compliance reviews but has not changed the system.

Company involved
Department of Veterans Affairs
AI system involved
REACH VET

2 source articles · read the reporting →

WF-DC7W6F1 Aug 2022

Hospital Denies Pain Medication to Patient Based on Narx Score

Elizabeth Amirault, a 34-year-old chronic pain patient, was denied narcotic pain medication during a hospital visit in Fort Wayne, Indiana, in August 2022. A nurse practitioner told her that her Narx Score, generated by Bamboo Health's NarxCare platform, was too high to prescribe opioids. The algorithm uses prescription drug monitoring data to assess risk of misuse, but critics say it can harm patients by cutting off needed care. Amirault believes her score was unfairly high due to her multiple surgeries and doctors.

AI system involved
NarxCare

3 source articles · read the reporting →

WF-DYR3NM1 Jun 2021

State Farm accused of algorithmic bias against Black homeowners

A class action lawsuit filed in Chicago alleges State Farm's automated claims processing system discriminates against Black policyholders. The suit claims Black customers are more likely to face additional paperwork and longer waits for claim approval than white customers, citing data from 800 homeowners. State Farm has stated it takes the filing seriously and is committed to fair treatment, while the plaintiffs seek to enjoin the current system and require audits.

Company involved
State Farm
AI system involved
automated claims processing system

2 source articles · read the reporting →

New Meta smart glasses class action gives employers a reason to write the policy they've been puttin - hcamag.com

The smart glasses recorded bystanders without their consent and sent the footage to Meta for AI training.

Company involved
Meta Platforms
AI system involved
Meta AI-enabled Ray-Ban and Oakley smart glasses

1 source article · read the reporting →

Abbott issues correction for FreeStyle Libre 3 sensors over incorrect glucose readings

Abbott initiated a medical device correction for certain FreeStyle Libre 3 and Libre 3 Plus sensors after internal testing found they might give incorrect low glucose readings. The company says the issue stems from one production line and could lead to improper treatment decisions for people with diabetes. Abbott is replacing affected sensors at no charge and has reported severe adverse events and deaths potentially associated with the issue.

Company involved
Abbott
AI system involved
FreeStyle Libre 3 and FreeStyle Libre 3 Plus sensors

2 source articles · read the reporting →

WF-9D8L0R21 Aug 2026

Capital Standard, LLC v. U.S. Bank National Association (CA Florida (2d)): AI-hallucinated content in court filing, Monetary Sanction; Adverse Costs…

The AI generated false legal citations that were included in a court filing, misleading the court.

1 source article · read the reporting →

WF-QIPOTI21 May 2026

Jackie L. Miller v. Regions Bank (N.D. Alabama): AI-hallucinated content in court filing, DQ from case and court (6 months);…

Attorney H. Gregory Harp was sanctioned with a public reprimand, disqualification from the case, a six-month court suspension, and a bar referral for filing AI-hallucinated legal citations and deleting evidence.

Company involved
Regions Bank
AI system involved
ChatGPT

1 source article · read the reporting →

WF-JJP7IB30 Mar 2026

Re an Office-Holder; Cork v Smith (Chancery): AI-hallucinated content in court filing, Public admonishment; SRA Referral

The AI system generated a non-existent insolvency rule that was submitted to the court, misleading the court.

Company involved
Pinsent Masons LLP

1 source article · read the reporting →

DFFH breaches privacy by using ChatGPT in child protection report

A child protection worker at the Department of Families, Fairness and Housing (DFFH) used ChatGPT to draft a Protection Application Report for the Children’s Court, entering sensitive personal information about a child. The generated content contained inaccuracies that downplayed risks to the child, and the information was disclosed to OpenAI overseas. An investigation by the Office of the Victorian Information Commissioner found DFFH failed to ensure accuracy and protect personal information, contravening IPPs 3.1 and 4.1. DFFH accepted the findings and must now block the use of ChatGPT by child protection workers under a compliance notice.

Company involved
Department of Families, Fairness and Housing
AI system involved
ChatGPT

9 source articles · read the reporting →

Geico and other insurers charge higher premiums in minority neighborhoods than in white areas with similar risk

ProPublica's analysis of auto insurance premiums in California, Illinois, Texas and Missouri found that major insurers such as Geico, Allstate and Liberty Mutual charge minority neighborhoods up to 30% more than white areas with similar accident costs. The disparity is alleged to result from pricing algorithms that may inadvertently use zip code as a proxy for race. Otis Nash, a 26-year-old Chicagoan, pays $190.69 per month for Geico insurance while a white driver in a safer neighborhood pays $54.67. The insurers deny discrimination, stating they do not collect race data.

Company involved
Geico

5 source articles · read the reporting →

WF-QMVDDA26 Jan 2017

Mother of robodebt suicide victim tells inquiry of government stonewalling

Jennifer Miller, whose son Rhys Cauzzo took his own life in 2017 while facing Centrelink debts of about $17,000, told the robodebt royal commission that the government stonewalled her for years. The commission heard that Cauzzo's debt was unlawfully calculated using income averaging and that no vulnerability indicator was on his file despite his mental health conditions. Miller said she felt vindicated by the commission's findings, while the former human services minister Alan Tudge had claimed the department acted appropriately. The robodebt scheme, which ran from 2015 to 2019, ended in a $1.8bn settlement after a class action.

Company involved
Department of Human Services (Services Australia)
AI system involved
Robodebt

10 source articles · read the reporting →

WF-MO5BND1 Jan 2014

FTC orders WW International and Kurbo to delete children's data and pay $1.5m penalty

The US Federal Trade Commission alleged that WW International, formerly Weight Watchers, and its subsidiary Kurbo illegally collected personal and health data from children under 13 through the Kurbo by WW weight-loss app without parental consent. The complaint said the signup process encouraged children to lie about their age and that the companies retained the data indefinitely. The companies settled, agreeing to delete the data, destroy any algorithms derived from it, and pay a $1.5 million penalty.

Company involved
WW International, Inc. (formerly Weight Watchers) and Kurbo, Inc.
AI system involved
Kurbo by WW

10 source articles · read the reporting →

WF-2HR7AX1 Jan 2014

Post Office Horizon scandal falsely accused subpostmasters of theft

The Post Office's Horizon IT system, supplied by Fujitsu, falsely reported financial shortfalls at many branches across England and Wales. Subpostmasters like Pete Murray were wrongly accused of theft and prosecuted, leading to loss of livelihood, reputational damage, and in some cases suicide. A group litigation, Bates v Post Office, settled, but many victims are still seeking compensation and justice. The scandal remains under investigation by the Post Office Horizon IT Inquiry.

Company involved
Post Office
AI system involved
Horizon

10 source articles · read the reporting →

WF-BZ7M5L1 Jan 2016

Tenant screening software denies housing to disabled Latino man in Connecticut

In 2016, Carmen Arroyo requested to move her disabled son Mikhail into a larger apartment in the same complex. The landlord, WinnResidential, used CoreLogic's CrimSafe tenant screening software, which flagged a dropped retail theft charge against Mikhail as disqualifying. The landlord rejected the application without explanation. Arroyo sued CoreLogic and WinnResidential under the Fair Credit Reporting Act and the Fair Housing Act, alleging that the software disproportionately excludes people of colour. The case was scheduled for trial in August 2021.

Company involved
WinnResidential
AI system involved
CrimSafe

10 source articles · read the reporting →

WF-FH40TB1 Jan 2022

Scottish Prison Service freed eight life prisoners due to risk assessment system error

The Scottish Prison Service's Level of Service and Case Management System failed to update risk assessments when new information was entered, leading to incorrect risk scores for 495 closed cases and 285 open cases. As a result, eight prisoners serving life for murder or rape were granted temporary release without accurate risk assessments. Justice Secretary Keith Brown acknowledged the error and stated that the cases would be reviewed. Opposition politicians called for an apology, claiming the releases put the public at risk.

Company involved
Scottish Prison Service
AI system involved
Level of Service and Case Management System

10 source articles · read the reporting →

WF-XFAIXB1 Jan 2014

Dutch Tax Authority discriminated against dual nationality applicants for child care benefits using automated risk system.

The Dutch Data Protection Authority (AP) found that the Benefits Department of the Tax and Customs Administration (Belastingdienst) unlawfully processed the dual nationality of 1.4 million applicants for child care benefits. The data was used as an indicator in an automated risk-scoring system that flagged applications as high-risk, which was unnecessary and discriminatory. The AP concluded that the processing violated the GDPR and was discriminatory. The AP completed its fact-finding and will consider imposing a sanction.

Company involved
Belastingdienst

10 source articles · read the reporting →

WF-YYSM1421 Jul 2020

Software flaw misclassified women as children on Tui flight

A software flaw in Tui's reservation system misclassified 38 adult female passengers as children due to a language difference where 'Miss' is used for children in the programming country. This led to incorrect weight calculations for take-off on a flight from Birmingham to Majorca on 21 July 2020. The AAIB report stated that the safe operation of the aircraft was not compromised, and Tui corrected the fault.

Company involved
Tui
AI system involved
Reservation system

10 source articles · read the reporting →

WF-X0QTXJ9 Feb 2021

NFL accused of using race in concussion disability assessments to deny black players benefits

The NFL Concussion Settlement disability assessments allegedly use race as a factor to lower baseline cognitive function scores for black and brown players, making it harder for them to qualify for disability benefits. Physicians are reportedly pressured to use this scale. A petition demands the practice stop and that demographic data be released. The settlement is overseen by the United States District Court for the Eastern District of Pennsylvania.

Company involved
NFL

10 source articles · read the reporting →

WF-O519OD1 Mar 2023

BetterHelp settles FTC charges over sharing mental health data with advertisers

Mozilla's review of mental health apps found that many, including BetterHelp, had poor privacy and security practices. BetterHelp was accused of improperly sharing customers' sensitive data with advertisers like Facebook and Snapchat after promising to keep it private. The company agreed to pay $7.8 million to settle FTC charges. The incident highlights ongoing privacy concerns in the mental health app market.

Company involved
BetterHelp
AI system involved
BetterHelp

4 source articles · read the reporting →

← Newerpage 6 of 7Older →