France welfare fraud detection algorithm accused of exacerbating inequality

A welfare fraud detection algorithm used by France's social security agency, CNAF, was accused of exacerbating inequality and targeting the most vulnerable populations. Introduced in the late 2000s to detect fraud, the algorithm was found to disproportionately targets vulnerable groups, such as single parents, low-income individuals, and people with disabilities, who have no apparent connection to fraud, according to a dual investigation by Lighthouse Reports and Le Monde . It was used to score almost half of France's population . According to the investigation, t he algorithm's criteria we re poorly constructed, leading to arbitrary and potentially unfair flagging of individuals based on minor changes in behavio ur. People flagged by the algorithm were subjected to invasive investigations, including searches of their hom es and scrutiny of personal records , an experience described as distressing and stigmati s ing. Despite its public domain use, the algorithm remained unknown to French citizens and others, and its criteria for scoring remained largely undisclosed. Efforts by digital rights groups to obtain the source code of the algorithm faced significant resistance, with crucial variables redacted, limiting the ability to fully understand how the algorithm operates. ➕ October 2024. La Quadrature du Net and f ourteen coalition partners submitted a complaint (pdf) to France's Council of State demanding CNAF's risk-scoring algorithmic system is stopped. System 🤖 Operator: Caisse Nationale des Allocations Familiales (CNAF) Developer: Caisse Nationale des Allocations Familiales (CNAF) Country: France Sector: Govt - welfare Purpose: Detect fraud Technology: Risk assessment algorithm; Machine learning Issue: Accountability; Accuracy/reliability; Bias/discrimination - disability; Transparency Legal, regulatory 👩🏼‍⚖️ Le Quadrature du Net . Legal complaint (pdf)

Date it happened

2023-12-01

Organisation involved

Caisse Nationale des Allocations Familiales (CNAF)

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.