The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

44 incidents closest to “FRISS Compliance Screening” · matched on meaning · public reporting

ACLU Says in FTC Complaint That Aon’s AI Tools Discriminatory

The system screened out job applicants based on personality tests, video interviews, and cognitive assessments, affecting autistic, neurodivergent, and people of color.

AI system involved
ADEPT-15, gridChallenge

1 source article · read the reporting →

AI Hiring Platform Faces FCRA Class Action Over Data Use | Kistler et al. v. Eightfold AI Inc.

The AI platform screened job applicants, affecting their hiring prospects.

Company involved
Eightfold AI
AI system involved
Eightfold AI

1 source article · read the reporting →

Resume prompt injection tricks AI hiring - moneywise.com

AI screening system determined which job applicants to advance to the next stage of recruitment.

1 source article · read the reporting →

Class action claims Checkr misreported criminal records in background checks

Checkr's system misreported criminal records belonging to someone else on Natasha Davis's consumer report, impeding her ability to gain employment.

Company involved
Checkr Inc.

1 source article · read the reporting →

SafeRent Settles Class Action Over Tenant Screening for Voucher Holders

Mary Louis and Monica Douglas filed a class action lawsuit in 2022 alleging that SafeRent Solutions' SafeRent Score product discriminated against rental applicants in Massachusetts who held public housing vouchers, violating fair housing and consumer protection laws. SafeRent denied wrongdoing. The court approved a settlement in November 2024, and payments were distributed to eligible class members in 2025 and 2026.

Company involved
SafeRent Solutions, LLC
AI system involved
SafeRent Score

5 source articles · read the reporting →

WF-LE6SZV1 Jan 2017

Rotterdam's Welfare Fraud Algorithm Discriminated by Gender and Ethnicity

The city of Rotterdam deployed a machine learning algorithm built by Accenture to flag welfare recipients for fraud investigation. The system used personal data including gender, language, and subjective caseworker notes to generate risk scores, leading to investigations that disproportionately targeted women and migrants. An external review found the algorithm discriminatory and inaccurate, prompting the city to suspend its use in 2021. The system's opacity made it nearly impossible for those flagged to challenge the decisions.

Company involved
City of Rotterdam

6 source articles · read the reporting →

FTC Orders Evolv to Stop Overstating Effectiveness of AI Gun Detection Tech

Evolv, a company selling AI-powered weapons detection systems, has been ordered by the FTC to stop making misleading claims about its Evolv Express product. The FTC alleged that the technology is essentially a metal detector with unproven AI, and that Evolv falsely claimed it could detect guns while ignoring harmless items. A pilot in New York City subways resulted in over 100 false positives and no guns detected. The settlement requires Evolv to allow educational customers to cancel their contracts and prohibits further misrepresentations.

Company involved
Evolv
AI system involved
Evolv Express

2 source articles · read the reporting →

Home Office faces legal challenge over secret visa-sorting algorithm

The Home Office is using a secret algorithm to sort visa applicants into fast, slow, or full-check streams. The algorithm allegedly uses biased criteria like nationality, disadvantaging people of colour. Foxglove and the Joint Council for the Welfare of Immigrants have initiated legal action, serving a pre-action letter and seeking judicial review. The Home Office has not disclosed details, citing the Immigration Exemption to the Freedom of Information Act.

8 source articles · read the reporting →

WF-AYFJ2A1 Jan 2018

Schufa's Black-Box Scoring Unfairly Penalises Consumers with Positive Credit Data

An investigation by SPIEGEL and BR Data reveals that Schufa's credit scoring algorithm often assigns poor risk scores to consumers with only positive credit information. One consumer, Sven Drewert, was denied a credit card limit increase despite having no negative entries. The algorithm uses limited data, and its secret formula can lead to arbitrary categorisations, affecting access to loans, phone contracts, and housing. The system's opacity and potential biases raise concerns about fairness and accountability.

Company involved
Schufa Holding AG
AI system involved
Schufa Score

2 source articles · read the reporting →

WF-FSUUWV6 Apr 2018

Durham Police uses Experian Mosaic data in HART AI risk tool

Durham Constabulary developed the Harm Assessment Risk Tool (HART), a machine learning algorithm that assesses the recidivism risk of offenders. The tool uses 34 data categories including criminal history, age, gender and two types of postcode, one sourced from Experian's Mosaic marketing segmentation system. Big Brother Watch alleges that using such commercial consumer behaviour data to inform custody decisions risks prejudice and disproportionate targeting of deprived neighbourhoods. The force has stated it is refreshing the model with an aim to remove one of the postcode predictors.

Company involved
Durham Constabulary
AI system involved
Harm Assessment Risk Tool (HART)

9 source articles · read the reporting →

WF-YFC2FF29 Oct 2015

CFPB fines General Information Services for inaccurate background checks

The Consumer Financial Protection Bureau took action against General Information Services and its affiliate e-Background-checks.com for failing to ensure the accuracy of employment background screening reports. The companies allegedly provided inaccurate criminal history information to employers, potentially affecting job applicants' eligibility and causing reputational harm. The CFPB ordered the companies to provide $10.5 million in relief to harmed consumers and pay a $2.5 million penalty.

Company involved
General Information Services

10 source articles · read the reporting →

WF-IGS22L1 Jan 2012

FinTech and traditional lenders discriminate against minority borrowers in mortgage pricing

A study of mortgage lending from 2012-2018 found that Latinx and African-American borrowers were charged higher interest rates than white borrowers with similar credit risk. FinTech algorithms reduced the disparity by 40% but did not eliminate it. The discrimination costs minority borrowers an estimated $765 million per year in extra interest.

10 source articles · read the reporting →

WF-XHTPKC15 Oct 2024

France: CNAF's discriminatory risk-scoring algorithm must be stopped

Amnesty International and coalition partners filed a complaint with the Council of State against CNAF's risk-scoring algorithm used to detect benefit overpayments. The algorithm assigns risk scores based on criteria that discriminate against vulnerable groups, including those with disabilities, single parents, and low-income households. The complaint alleges the system violates human rights to equality and privacy. The EU AI Act's social scoring ban may apply, but its definition remains unclear.

Company involved
CNAF

2 source articles · read the reporting →

U.S. Border Patrol uses AI and ALPR to target drivers for pretext stops and asset seizures

The U.S. Border Patrol has built a nationwide dragnet driver-surveillance system using automated license plate readers and AI to flag suspicious travel patterns. Local police then pull over targeted drivers on pretexts, interrogate them, and seize cash and vehicles through civil asset forfeiture. The program has been kept secret, with details hidden from court documents and the public. The ACLU report highlights the abuse of innocent drivers and calls for congressional action.

Company involved
U.S. Border Patrol (CBP)
AI system involved
Automated license plate reader (ALPR) system with AI analytics

6 source articles · read the reporting →

WF-89GNUB1 Oct 2018

AMS algorithm lacks transparency and may discriminate against job seekers

The Austrian Public Employment Service (AMS) uses an algorithm to classify job seekers into categories A, B, and C, determining their access to benefits and training. Scientists from TU Wien, WU Wien, and University of Vienna have criticised the algorithm for lacking transparency, as only two of 96 model variants have been published. They allege that the system may discriminate against women and people with migration background, and that job seekers are not informed about how the algorithm works or given a chance to appeal.

Company involved
AMS (Arbeitsmarktservice Österreich)
AI system involved
AMS-Algorithmus

10 source articles · read the reporting →

Dutch police predictive policing project in Roermond faces discrimination allegations

Amnesty International's report alleges that the Dutch police's Sensing predictive policing project in Roermond uses algorithms that result in automated discrimination and mass surveillance. The system assesses crime risk for individuals and locations, potentially targeting marginalized communities. The report calls for accountability and human rights safeguards.

Company involved
Dutch police
AI system involved
Sensing project

10 source articles · read the reporting →

WF-HXS00H1 Jan 2020

Dutch government continued SyRI welfare fraud profiling after ban

The Dutch government deployed the SyRI algorithm to profile welfare recipients in low-income neighborhoods, despite a 2020 court ruling that the system violated EU human rights. The algorithm flagged thousands of parents for fraud based on illegal risk factors such as dual nationality, leading to false accusations, bankruptcy, and trauma. The government has announced an external review of the practices.

Company involved
Ministry of Social Affairs of the Netherlands
AI system involved
SyRI

10 source articles · read the reporting →

Titus Henderson files civil rights lawsuit against COMPAS

Titus Henderson, a prisoner, filed a civil rights lawsuit against COMPAS and other defendants in the U.S. District Court for the Western District of Wisconsin on July 16, 2018. The lawsuit alleges discrimination related to the use of the COMPAS risk assessment tool. The case is ongoing.

AI system involved
COMPAS

6 source articles · read the reporting →

Judge rules police search using Flock was mass surveillance

A judge ruled that a police search using Flock's automated license plate readers constituted a form of mass surveillance. The ruling concerns the deployment of the AI-based camera system by law enforcement, which the court found to be an invasive surveillance practice. No further details of the case were provided in the article.

AI system involved
Flock

4 source articles · read the reporting →

DeepScore markets facial and voice analysis app for trustworthiness scoring despite experts' doubts

DeepScore, a Tokyo-based company, is marketing an app that uses facial and voice recognition to score people's trustworthiness for lenders and insurers in Japan, Indonesia, Vietnam and the Philippines. The company says the app can detect deception with 70 per cent accuracy, but researchers and privacy advocates say there is no reliable scientific basis for such judgments and warn of discrimination and privacy harms. The chief executive said the system is only one part of lenders' and insurers' decision-making and that people can choose not to use it. Critics respond that an unequal balance of power makes consent difficult.

Company involved
DeepScore
AI system involved
DeepScore

6 source articles · read the reporting →

Wisconsin court used secret COMPAS algorithm to sentence Loomis to harsher term

In Loomis v. Wisconsin, a judge used a COMPAS risk score from Northpointe to sentence a defendant to a harsher punishment. The algorithm was kept secret as a trade secret, preventing the defendant from challenging its accuracy. The Wisconsin Supreme Court upheld the sentence, ruling that the score was only one part of the rationale. The case raises concerns about due process and the use of secret algorithms in criminal sentencing.

Company involved
State of Wisconsin
AI system involved
COMPAS

10 source articles · read the reporting →

WF-DYTSSY24 May 2021

Lemonade backtracks on tweet claiming AI scans customer faces for fraud

Lemonade, an insurance firm, posted a tweet claiming it uses AI to detect non-verbal cues in customer videos to deny claims. After backlash comparing the practice to phrenology, the company deleted the tweet and denied using AI to deny claims based on facial characteristics. Lemonade stated it uses facial recognition only to flag claims submitted under different identities, which are then reviewed by human investigators.

Company involved
Lemonade

10 source articles · read the reporting →

WF-BSSRTA30 Apr 2019

Danish law enables algorithm to profile unemployed for long-term risk

The Danish parliament passed a law in April 2019 that allows an algorithm to analyse personal data of all unemployed individuals to identify those at risk of long-term unemployment. The system uses data on ethnicity, age, health, and education history to produce a risk score for social workers. Critics, including legal experts and unemployment insurance funds, argue the profiling is stigmatising, lacks transparency, and may violate GDPR. The Danish Data Protection Authority has reopened its review of the law.

Company involved
Danish Ministry of Employment

10 source articles · read the reporting →

WF-J5IU7Z3 Dec 2024

FTC settles with IntelliVision over deceptive facial recognition claims

The Federal Trade Commission (FTC) took action against IntelliVision Technologies Corp. for making false, misleading, or unsubstantiated claims about its AI-powered facial recognition software. The company allegedly claimed its software had one of the highest accuracy rates on the market and was free of gender or racial bias, without supporting evidence. The FTC also alleged that IntelliVision did not train its software on millions of faces as claimed, but on images of approximately 100,000 individuals. Under a proposed consent order, IntelliVision is prohibited from making such misrepresentations without competent and reliable testing.

Company involved
IntelliVision Technologies Corp.
AI system involved
IntelliVision facial recognition software

9 source articles · read the reporting →

page 1 of 2Older →