Report: Denmark AI welfare system "profoundly affects" mental health

Denmark's use of artificial intelligence to identify welfare fraud has resulted in an atmosphere of fear, deterioration of mental health and accusations of undue mass surveillance and potentially illegal social scoring, according to human rights campaigners. What happened Danish welfare authority, Udbetaling Danmark (UDK) extensively employs artificial intelligence (AI) tools to detect potential fraud in a wide range of areas, from pension payments to parental and sick leave, and student grants. However, analysis of four of the algorithms and r eports from individuals subjected to on going "mass surveillance" led Amnesty to accuse UDK of "mass surveillance" that infringes on individuals' privacy rights, discriminates against vulnerable groups, and has created a climate of fear and suspicion among those being tracked. It also argues that the system f unctions as a social scoring system under the EU 's Artificial Intelligence Act and should be banned . Why it happened The use of AI in Denmark's welfare system stems from a push for digitisation aimed at identifying fraudulent claims more efficiently and effectively. The programme has involved the collection of huge amounts of personal data - including residency status, citizenship, family relationships, travel history and health records - and the creation of up to 60 algorithmic models designed to detect social benefits fraud and flag individuals for further investigation. Amnesty argues that the amount of data collected and processed is "neither necessary nor proportionate". It also says t he algorithms used are built on existing societal biases, which exacerbate inequalities rather than alleviate them. What it means Amnesty's findings put pressure on Udbetaling Danmark to be consider carefully the workings and i mpact of its fraud detection systems, and to be more transparent and about how it uses AI. The report may also lead to discussion in Copenhagen and Brussels about what constitutes a social scoring system, which are banned under the EU AI Act. System 🤖 Udbetaling Danmark 🔗 Operator: Arbejdsmarkedets Tillægspension Developer: Arbejdsmarkedets Tillægspension ; NNIT Country: Denmark Sector: Education; Govt - welfare Purpose: Detect fraud Technology: Classification algorithm; Machine l earning Issue: Accuracy/reliability; Bias/discrimination; Privacy; Surveillance Regulation ⚖️ EU Artificial Intelligence Act - Recital 31

Date it happened

2024-11-01

Organisation involved

Udbetaling Danmark

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.