The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

191 incidents closest to “Public Sector Solutions Benefit Management” · matched on meaning · public reporting

WF-AAEOVA1 May 2022

Lawsuit Says UnitedHealth Used AI With a 90% Error Rate to Deny Care - Yahoo

Denied post-acute care coverage to elderly Medicare Advantage patients

Company involved
UnitedHealth Group
AI system involved
nH Predict

1 source article · read the reporting →

WF-FFWZTS1 Jan 2016

Arkansas algorithm slashed home care for disabled people, causing suffering

In Arkansas and other states, algorithms were used to allocate home care hours for disabled and elderly people, replacing nurse judgments. The system assigned hours based on a scoring tool that failed to account for specific disabilities, leading to drastic cuts. Many suffered severe harm, including bed sores, missed meals, and one death. After lawsuits, Arkansas's system was thrown out in 2018, and Idaho's was declared unconstitutional.

Company involved
Arkansas state government

1 source article · read the reporting →

WF-HQIBSH16 Nov 2021

South Korean local governments build AI surveillance systems using CCTV data without consent

Several local governments in South Korea, including Bucheon, are developing AI systems that use CCTV footage to track individuals for purposes such as disease control, child abuse detection, and personal protection. The systems are being built without obtaining consent from the people captured in the footage, raising concerns about privacy and surveillance. Bucheon plans to launch a smart epidemiological investigation system in January 2022 that will analyze footage from thousands of crime-prevention cameras to track COVID-19 cases and contacts. Critics question the adequacy of de-identification measures and warn of potential misuse of biometric data.

Company involved
Bucheon City Government
AI system involved
Smart epidemiological investigation system

10 source articles · read the reporting →

DFFH breaches privacy by using ChatGPT in child protection report

A child protection worker at the Department of Families, Fairness and Housing (DFFH) used ChatGPT to draft a Protection Application Report for the Children’s Court, entering sensitive personal information about a child. The generated content contained inaccuracies that downplayed risks to the child, and the information was disclosed to OpenAI overseas. An investigation by the Office of the Victorian Information Commissioner found DFFH failed to ensure accuracy and protect personal information, contravening IPPs 3.1 and 4.1. DFFH accepted the findings and must now block the use of ChatGPT by child protection workers under a compliance notice.

Company involved
Department of Families, Fairness and Housing
AI system involved
ChatGPT

9 source articles · read the reporting →

WF-QMVDDA26 Jan 2017

Mother of robodebt suicide victim tells inquiry of government stonewalling

Jennifer Miller, whose son Rhys Cauzzo took his own life in 2017 while facing Centrelink debts of about $17,000, told the robodebt royal commission that the government stonewalled her for years. The commission heard that Cauzzo's debt was unlawfully calculated using income averaging and that no vulnerability indicator was on his file despite his mental health conditions. Miller said she felt vindicated by the commission's findings, while the former human services minister Alan Tudge had claimed the department acted appropriately. The robodebt scheme, which ran from 2015 to 2019, ended in a $1.8bn settlement after a class action.

Company involved
Department of Human Services (Services Australia)
AI system involved
Robodebt

10 source articles · read the reporting →

WF-GNY7M91 May 2015

Poland's Ministry of Labor profiles unemployed with discriminatory algorithm

In May 2015, Poland's Ministry of Labor and Social Policy introduced an algorithmic system to profile unemployed people into three categories based on their readiness to work, location, and disabilities. The system is accused of being discriminatory and lacking transparency, as it uses default criteria that may lead to social exclusion. A report by the Panoptykon Foundation in October 2015 highlighted these issues and recommended reforms, including granting the right to appeal.

Company involved
Ministry of Labor and Social Policy

10 source articles · read the reporting →

Oregon child welfare agency stops using algorithm to screen families for investigations

The Oregon Department of Human Services announced it will stop using its Safety at Screening Tool algorithm, which helped hotline workers decide which families to investigate for child abuse and neglect. The decision followed concerns about racial bias and disparities, as the algorithm was inspired by a Pennsylvania tool that had flagged a disproportionate number of Black children. The agency will replace the algorithm with a new Structured Decision Making model.

Company involved
Oregon Department of Human Services
AI system involved
Safety at Screening Tool

10 source articles · read the reporting →

WF-2HR7AX1 Jan 2014

Post Office Horizon scandal falsely accused subpostmasters of theft

The Post Office's Horizon IT system, supplied by Fujitsu, falsely reported financial shortfalls at many branches across England and Wales. Subpostmasters like Pete Murray were wrongly accused of theft and prosecuted, leading to loss of livelihood, reputational damage, and in some cases suicide. A group litigation, Bates v Post Office, settled, but many victims are still seeking compensation and justice. The scandal remains under investigation by the Post Office Horizon IT Inquiry.

Company involved
Post Office
AI system involved
Horizon

10 source articles · read the reporting →

WF-BZ7M5L1 Jan 2016

Tenant screening software denies housing to disabled Latino man in Connecticut

In 2016, Carmen Arroyo requested to move her disabled son Mikhail into a larger apartment in the same complex. The landlord, WinnResidential, used CoreLogic's CrimSafe tenant screening software, which flagged a dropped retail theft charge against Mikhail as disqualifying. The landlord rejected the application without explanation. Arroyo sued CoreLogic and WinnResidential under the Fair Credit Reporting Act and the Fair Housing Act, alleging that the software disproportionately excludes people of colour. The case was scheduled for trial in August 2021.

Company involved
WinnResidential
AI system involved
CrimSafe

10 source articles · read the reporting →

WF-FH40TB1 Jan 2022

Scottish Prison Service freed eight life prisoners due to risk assessment system error

The Scottish Prison Service's Level of Service and Case Management System failed to update risk assessments when new information was entered, leading to incorrect risk scores for 495 closed cases and 285 open cases. As a result, eight prisoners serving life for murder or rape were granted temporary release without accurate risk assessments. Justice Secretary Keith Brown acknowledged the error and stated that the cases would be reviewed. Opposition politicians called for an apology, claiming the releases put the public at risk.

Company involved
Scottish Prison Service
AI system involved
Level of Service and Case Management System

10 source articles · read the reporting →

WF-XFAIXB1 Jan 2014

Dutch Tax Authority discriminated against dual nationality applicants for child care benefits using automated risk system.

The Dutch Data Protection Authority (AP) found that the Benefits Department of the Tax and Customs Administration (Belastingdienst) unlawfully processed the dual nationality of 1.4 million applicants for child care benefits. The data was used as an indicator in an automated risk-scoring system that flagged applications as high-risk, which was unnecessary and discriminatory. The AP concluded that the processing violated the GDPR and was discriminatory. The AP completed its fact-finding and will consider imposing a sanction.

Company involved
Belastingdienst

10 source articles · read the reporting →

WF-X0QTXJ9 Feb 2021

NFL accused of using race in concussion disability assessments to deny black players benefits

The NFL Concussion Settlement disability assessments allegedly use race as a factor to lower baseline cognitive function scores for black and brown players, making it harder for them to qualify for disability benefits. Physicians are reportedly pressured to use this scale. A petition demands the practice stop and that demographic data be released. The settlement is overseen by the United States District Court for the Eastern District of Pennsylvania.

Company involved
NFL

10 source articles · read the reporting →

WF-47L7QK1 Jan 2021

Telangana algorithm denied food to thousands of poor families

The Telangana government's algorithmic system Samagra Vedika wrongfully denied subsidised food to thousands of poor families by incorrectly tagging them as ineligible. The system, developed by Posidex Technologies, misidentified individuals across databases, leading to false positives. Affected families, including 67-year-old widow Bismillah Bee, were forced to prove their eligibility, often unsuccessfully. A Supreme Court case ordered re-verification, but many remain excluded.

Company involved
Government of Telangana
AI system involved
Samagra Vedika

8 source articles · read the reporting →

WF-UDDP7B6 Dec 2017

Illinois DCFS ends unreliable predictive analytics program for child abuse risk

The Illinois Department of Children and Family Services ended its Rapid Safety Feedback program, which used data mining to predict child abuse risk, after the agency's director called the technology unreliable. The system, developed by Eckerd Connects and Mindshare Technology, assigned risk scores to children but failed to flag several high-profile child deaths. The program was also criticised for overwhelming caseworkers with alerts and for potential bias against poor children of colour. DCFS decided not to renew the contract.

Company involved
Illinois Department of Children and Family Services
AI system involved
Rapid Safety Feedback

10 source articles · read the reporting →

WF-W4TQ2K1 Oct 2022

DOJ examines Allegheny County child welfare AI screening tool for disability bias

The Justice Department is scrutinizing the Allegheny Family Screening Tool, an AI system used by Allegheny County, Pennsylvania, to assess risk for families reported for child welfare concerns. A father with an intellectual disability, represented by attorney Robin Frank, filed a complaint alleging the algorithm discriminates against families with disabilities by using disability-related data points. The county denies awareness of concerns and says the tool helps predict risk, but critics argue it hardens bias. The DOJ's Civil Rights Division is investigating.

Company involved
Allegheny County Department of Human Services
AI system involved
Allegheny Family Screening Tool

5 source articles · read the reporting →

WF-XV5P501 Oct 2015

Amazon Buy Box algorithm accused of overcharging UK consumers by £1 billion

Consumer champion Robert Hammond has filed a collective proceedings order application against Amazon in the UK Competition Appeal Tribunal. The claim alleges that Amazon's Buy Box algorithm manipulated search results to favour products more profitable for Amazon, obscuring cheaper alternatives and causing approximately 49.4 million UK consumers to overpay by more than £1 billion between October 2015 and June 2020. The algorithm determined the first-ranked item in the Buy Box, which consumers accepted over 80% of the time. The case is pending, and similar investigations by the Italian Competition Authority and the UK Competition and Markets Authority are ongoing.

Company involved
Amazon
AI system involved
Buy Box algorithm

5 source articles · read the reporting →

WF-RODNM31 Mar 2020

UK government's Spotlight algorithm denies funding to independent cinemas

The UK government's Culture Recovery Fund used an automated due diligence tool called Spotlight, designed by the Cabinet Office, to process grant applications. The algorithm is accused of excluding smaller, community-focused independent cinemas and arts organizations in favour of larger institutions, a practice the article describes as automated cultural elitism. The National Audit Office has confirmed the criteria, and Arts Council England is investigating cases of potential fraud, while many venues report struggling to survive.

Company involved
UK Government Cabinet Office
AI system involved
Spotlight

6 source articles · read the reporting →

WF-CB6PG11 Jan 2023

Paraná schools use European facial recognition to mark attendance, risking welfare for false absences

Since 2023, the Paraná state education department in Brazil has deployed facial recognition software from Slovak company Innovatrics in over 1,700 schools to automate attendance for nearly one million children. The system has made errors, marking present students as absent. Because attendance records generated by the algorithm affect eligibility for the Bolsa Família welfare program, false absences could threaten families' benefits. In April 2025, the state's public prosecutor filed a legal challenge alleging the system violates Brazilian law; the case is pending.

Company involved
Paraná state education department (Secretaria da Educação do Paraná)
AI system involved
LRCO Paraná

8 source articles · read the reporting →

WF-A7H7JC28 May 2019

‘You’re denying what you did to my son’: mother of robodebt victim told she is ineligible for compensation - The…

The system determined Jarrad Madgwick owed a debt of $1,795.85 and would not receive his Newstart allowance.

Company involved
Services Australia
AI system involved
Robodebt scheme

1 source article · read the reporting →

WF-Q7SX203 Jun 2019

Centrelink victim's mum reveals cruel twist years after son's $1,795 debt notice - Yahoo Finance Australia

The system determined Jarrad Madgwick owed a debt and made him ineligible for Centrelink payments.

Company involved
Services Australia
AI system involved
Robodebt

1 source article · read the reporting →

Talkdesk Hit With Class Suit Over Customer Service Software - news.bloomberglaw.com

The system secretly recorded and analyzed customer service conversations of callers to Patagonia and Zumiez without consent.

Company involved
Talkdesk Inc.
AI system involved
CX Cloud

1 source article · read the reporting →

Safaricom sued over AI use in customer service and M-PESA decisions - nation.africa

The AI system determined access to services, processed financial transactions, and handled customer complaints for Safaricom customers.

Company involved
Safaricom PLC

1 source article · read the reporting →

State Farm hit with lawsuit over AI bias, discrimination and unpaid insurance claims - beinsure.com

Denied or canceled insurance policies and delayed claim payments to policyholders, disproportionately affecting Black and nonwhite, disabled, and abuse survivor policyholders.

Company involved
State Farm

1 source article · read the reporting →

← Newerpage 8 of 8