The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

125 incidents closest to “PLUS Risk Workbench” · matched on meaning · public reporting

WF-BZ7M5L1 Jan 2016

Tenant screening software denies housing to disabled Latino man in Connecticut

In 2016, Carmen Arroyo requested to move her disabled son Mikhail into a larger apartment in the same complex. The landlord, WinnResidential, used CoreLogic's CrimSafe tenant screening software, which flagged a dropped retail theft charge against Mikhail as disqualifying. The landlord rejected the application without explanation. Arroyo sued CoreLogic and WinnResidential under the Fair Credit Reporting Act and the Fair Housing Act, alleging that the software disproportionately excludes people of colour. The case was scheduled for trial in August 2021.

Company involved
WinnResidential
AI system involved
CrimSafe

10 source articles · read the reporting →

Prisma Labs wins arbitration in Lensa biometric privacy lawsuit

Artificial intelligence company Prisma Labs is accused of violating Illinois' biometric privacy law by collecting facial geometry data from users of its Lensa app. A proposed class action was filed, but a California federal judge granted Prisma's motion to compel arbitration because users agreed to it during sign-up. The plaintiff alleges the app collected his facial data without consent.

Company involved
Prisma Labs Inc.
AI system involved
Lensa AI

8 source articles · read the reporting →

WF-UDDP7B6 Dec 2017

Illinois DCFS ends unreliable predictive analytics program for child abuse risk

The Illinois Department of Children and Family Services ended its Rapid Safety Feedback program, which used data mining to predict child abuse risk, after the agency's director called the technology unreliable. The system, developed by Eckerd Connects and Mindshare Technology, assigned risk scores to children but failed to flag several high-profile child deaths. The program was also criticised for overwhelming caseworkers with alerts and for potential bias against poor children of colour. DCFS decided not to renew the contract.

Company involved
Illinois Department of Children and Family Services
AI system involved
Rapid Safety Feedback

10 source articles · read the reporting →

WF-W4TQ2K1 Oct 2022

DOJ examines Allegheny County child welfare AI screening tool for disability bias

The Justice Department is scrutinizing the Allegheny Family Screening Tool, an AI system used by Allegheny County, Pennsylvania, to assess risk for families reported for child welfare concerns. A father with an intellectual disability, represented by attorney Robin Frank, filed a complaint alleging the algorithm discriminates against families with disabilities by using disability-related data points. The county denies awareness of concerns and says the tool helps predict risk, but critics argue it hardens bias. The DOJ's Civil Rights Division is investigating.

Company involved
Allegheny County Department of Human Services
AI system involved
Allegheny Family Screening Tool

5 source articles · read the reporting →

Talkdesk Hit With Class Suit Over Customer Service Software - news.bloomberglaw.com

The system secretly recorded and analyzed customer service conversations of callers to Patagonia and Zumiez without consent.

Company involved
Talkdesk Inc.
AI system involved
CX Cloud

1 source article · read the reporting →

← Newerpage 6 of 6