The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

157 incidents closest to “HMS coordination of benefits / third party liability services” · matched on meaning · public reporting

WF-BZ7M5L1 Jan 2016

Tenant screening software denies housing to disabled Latino man in Connecticut

In 2016, Carmen Arroyo requested to move her disabled son Mikhail into a larger apartment in the same complex. The landlord, WinnResidential, used CoreLogic's CrimSafe tenant screening software, which flagged a dropped retail theft charge against Mikhail as disqualifying. The landlord rejected the application without explanation. Arroyo sued CoreLogic and WinnResidential under the Fair Credit Reporting Act and the Fair Housing Act, alleging that the software disproportionately excludes people of colour. The case was scheduled for trial in August 2021.

Company involved
WinnResidential
AI system involved
CrimSafe

10 source articles · read the reporting →

WF-FH40TB1 Jan 2022

Scottish Prison Service freed eight life prisoners due to risk assessment system error

The Scottish Prison Service's Level of Service and Case Management System failed to update risk assessments when new information was entered, leading to incorrect risk scores for 495 closed cases and 285 open cases. As a result, eight prisoners serving life for murder or rape were granted temporary release without accurate risk assessments. Justice Secretary Keith Brown acknowledged the error and stated that the cases would be reviewed. Opposition politicians called for an apology, claiming the releases put the public at risk.

Company involved
Scottish Prison Service
AI system involved
Level of Service and Case Management System

10 source articles · read the reporting →

WF-XFAIXB1 Jan 2014

Dutch Tax Authority discriminated against dual nationality applicants for child care benefits using automated risk system.

The Dutch Data Protection Authority (AP) found that the Benefits Department of the Tax and Customs Administration (Belastingdienst) unlawfully processed the dual nationality of 1.4 million applicants for child care benefits. The data was used as an indicator in an automated risk-scoring system that flagged applications as high-risk, which was unnecessary and discriminatory. The AP concluded that the processing violated the GDPR and was discriminatory. The AP completed its fact-finding and will consider imposing a sanction.

Company involved
Belastingdienst

10 source articles · read the reporting →

WF-YYSM1421 Jul 2020

Software flaw misclassified women as children on Tui flight

A software flaw in Tui's reservation system misclassified 38 adult female passengers as children due to a language difference where 'Miss' is used for children in the programming country. This led to incorrect weight calculations for take-off on a flight from Birmingham to Majorca on 21 July 2020. The AAIB report stated that the safe operation of the aircraft was not compromised, and Tui corrected the fault.

Company involved
Tui
AI system involved
Reservation system

10 source articles · read the reporting →

WF-X0QTXJ9 Feb 2021

NFL accused of using race in concussion disability assessments to deny black players benefits

The NFL Concussion Settlement disability assessments allegedly use race as a factor to lower baseline cognitive function scores for black and brown players, making it harder for them to qualify for disability benefits. Physicians are reportedly pressured to use this scale. A petition demands the practice stop and that demographic data be released. The settlement is overseen by the United States District Court for the Eastern District of Pennsylvania.

Company involved
NFL

10 source articles · read the reporting →

WF-O519OD1 Mar 2023

BetterHelp settles FTC charges over sharing mental health data with advertisers

Mozilla's review of mental health apps found that many, including BetterHelp, had poor privacy and security practices. BetterHelp was accused of improperly sharing customers' sensitive data with advertisers like Facebook and Snapchat after promising to keep it private. The company agreed to pay $7.8 million to settle FTC charges. The incident highlights ongoing privacy concerns in the mental health app market.

Company involved
BetterHelp
AI system involved
BetterHelp

4 source articles · read the reporting →

Call Vendors Skirt Wiretap Suit Over AI Transcription Tool - Law360

The system recorded and analyzed patient phone calls without their consent.

Company involved
Heartland Dental LLC

1 source article · read the reporting →

WF-A7H7JC28 May 2019

‘You’re denying what you did to my son’: mother of robodebt victim told she is ineligible for compensation - The…

The system determined Jarrad Madgwick owed a debt of $1,795.85 and would not receive his Newstart allowance.

Company involved
Services Australia
AI system involved
Robodebt scheme

1 source article · read the reporting →

WF-KBFGPD12 Oct 2023

Tesla Autopilot Lawsuit Filed in Death of William A. Hensley - Car Complaints

The Autopilot system failed to detect a curve and continued straight, causing the vehicle to veer off the road and crash, killing the driver.

Company involved
Tesla, Inc.
AI system involved
Autopilot

1 source article · read the reporting →

“Listen Up” if Your AI Policy Does Not Cover AI Recording Issues –Another Class Action Lawsuit Filed Over Third Party…

AI system recorded and analyzed calls between a dental company and its patients

Company involved
dental company

1 source article · read the reporting →

Talkdesk Hit With Class Suit Over Customer Service Software - news.bloomberglaw.com

The system secretly recorded and analyzed customer service conversations of callers to Patagonia and Zumiez without consent.

Company involved
Talkdesk Inc.
AI system involved
CX Cloud

1 source article · read the reporting →

Class action claims Navy Federal allows AI co. to monitor, record calls - Top Class Actions

The system intercepted, recorded, and analyzed customer service calls without consent, affecting the privacy of callers.

Company involved
Navy Federal Credit Union
AI system involved
Verint Real-Time Agent Assist

1 source article · read the reporting →

State Farm hit with lawsuit over AI bias, discrimination and unpaid insurance claims - beinsure.com

Denied or canceled insurance policies and delayed claim payments to policyholders, disproportionately affecting Black and nonwhite, disabled, and abuse survivor policyholders.

Company involved
State Farm

1 source article · read the reporting →

← Newerpage 7 of 7