Facebook approved harmful ads targeting teens in TTP experiment
The Tech Transparency Project (TTP) submitted six test ads to Facebook promoting pill parties, anorexia, alcohol, smoking, gambling, and dating services, targeting users aged 13-17. Facebook approved all ads within hours, despite its policies prohibiting such content. The ads never ran as TTP did not allow them to be published. Facebook stated it has measures in place to review ads.
- Company involved
- Facebook
- AI system involved
- Facebook Ads
10 source articles · read the reporting →
UberEats algorithm change causes pay cuts and financial hardship for car drivers
A survey by the Transport Workers' Union found that UberEats car drivers experienced a 50% pay cut after the company changed its delivery allocation algorithm to preference cyclists and scooter riders. Drivers reported bankruptcy, homelessness, and inability to support their families. The TWU has called for regulation of the gig economy and delivered a letter to the NSW shadow minister.
- Company involved
- UberEats
- AI system involved
- UberEats platform
10 source articles · read the reporting →
Shipt's new V2 algorithm reduces pay for 41% of workers, study finds
A study by MIT Media Lab and Coworker.org found that Shipt's new V2 payment algorithm reduced pay for 41% of workers by an average of 11% per shop. The algorithm, rolled out in September 2020, replaced a transparent payment system with a black-box model. Workers pooled data to analyze the impact, revealing uneven distribution of earnings.
- Company involved
- Shipt
- AI system involved
- V2
10 source articles · read the reporting →
DWP's Universal Credit algorithm causes underpayment and hardship for claimants
The Department for Work and Pensions (DWP) uses an automated means-testing algorithm in its Universal Credit system that miscalculates earnings due to a design flaw, leading to underpayments and fluctuating benefit amounts. Claimants such as Janet R. received payments hundreds of pounds short, forcing them into debt and foodbank use. A Court of Appeal ruling in June 2020 ordered the DWP to fix the flaw, but the department had not yet fully implemented the remedy at the time of writing.
- Company involved
- Department for Work and Pensions
- AI system involved
- Universal Credit
10 source articles · read the reporting →
Dutch government continued SyRI welfare fraud profiling after ban
The Dutch government deployed the SyRI algorithm to profile welfare recipients in low-income neighborhoods, despite a 2020 court ruling that the system violated EU human rights. The algorithm flagged thousands of parents for fraud based on illegal risk factors such as dual nationality, leading to false accusations, bankruptcy, and trauma. The government has announced an external review of the practices.
- Company involved
- Ministry of Social Affairs of the Netherlands
- AI system involved
- SyRI
10 source articles · read the reporting →
LA's VI-SPDAT housing scoring system gives lower priority to Black and Latino people
The Los Angeles Homeless Services Authority uses the VI-SPDAT scoring system to prioritise unhoused people for subsidised permanent housing. An investigation by The Markup found that Black and Latino people consistently receive lower vulnerability scores than White people, leading to lower priority for housing. The agency has acknowledged the racial disparities and is working on a new tool, but continues to use the current system.
- Company involved
- Los Angeles Homeless Services Authority
- AI system involved
- VI-SPDAT
10 source articles · read the reporting →
Wisconsin’s dropout prediction algorithm labels students high risk with racial bias
Wisconsin’s Dropout Early Warning System (DEWS) uses machine learning to predict middle school students' likelihood of graduating on time, relying on factors including race and income. The Markup found the algorithm frequently mislabels students, with a higher false alarm rate for Black and Hispanic students. The Wisconsin Department of Public Instruction has not informed schools of the racial disparities and continues to use the system.
- Company involved
- Wisconsin Department of Public Instruction
- AI system involved
- Dropout Early Warning System (DEWS)
10 source articles · read the reporting →
Serbia's Social Card law could harm marginalized groups
Serbia's Social Card law, which entered into force on 1 March 2022, establishes a centralized government database to assess eligibility for social security support. Amnesty International and seven other rights organizations submitted a legal opinion to the Constitutional Court, alleging that the automated system is an intrusive surveillance system that could discriminate against Roma communities and people with disabilities. The system processes 130 categories of personal data and lacks transparency, potentially leading to errors and denial of benefits.
- Company involved
- Serbian government
- AI system involved
- Social Card system
10 source articles · read the reporting →
Jacobs files BIPA class action against Walgreen Company over fingerprint scanning
A class action lawsuit alleges that Walgreen Company (Walgreens) collected and stored employees' fingerprint data without obtaining informed consent, in violation of the Illinois Biometric Information Privacy Act (BIPA). The plaintiff, Jacobs, claims the company used fingerprint scanners for employee timekeeping without providing required disclosures or obtaining written consent. The case is pending in Illinois state court.
- Company involved
- Walgreen Company
- AI system involved
- Fingerprint time clock
7 source articles · read the reporting →
USPS algorithm RRECS causes pay cuts for two-thirds of rural mail carriers
The United States Postal Service (USPS) implemented a new algorithm, RRECS, to evaluate rural carrier routes and determine pay. Due to flaws in how carriers scanned packages, the algorithm underestimated route times, resulting in pay cuts for 66 per cent of rural carriers, some losing thousands of dollars annually. Carriers report that they were not adequately trained on the system and that the cuts are scheduled to take effect, though they have been postponed multiple times. The USPS stated that the system is the result of a nationally negotiated agreement.
- Company involved
- United States Postal Service
- AI system involved
- RRECS
9 source articles · read the reporting →
University of Chicago report finds US healthcare algorithms rife with bias
A University of Chicago report alleges that algorithms used by hospitals, insurers and other businesses across the US are biased along racial and economic lines. The algorithms help triage patients, predict diabetes and flag those needing extra care, but the report says flawed products were introduced with little oversight. It says inequitable treatment has continued for more than a decade.
7 source articles · read the reporting →
Denmark's Udbetaling Danmark welfare algorithm illegally surveilled millions
Udbetaling Danmark (UDK), Denmark's centralized welfare payment agency, used automated algorithms to check eligibility and detect fraud. The system collected personal data on millions of beneficiaries and their relatives, including non-beneficiaries, in what a think-tank called 'systematic surveillance'. A complaint to the Danish Data Protection Authority led to a ruling that the data collection violated GDPR, and UDK eventually deleted the illegally obtained data. The system also caused erroneous payments, including underpaying 300,000 pensioners by 94 million euros annually.
- Company involved
- Udbetaling Danmark
10 source articles · read the reporting →
Amazon expands palm-scanning payments to all Whole Foods stores amid privacy concerns
Amazon is expanding its Amazon One palm-scanning payment system to all Whole Foods stores by the end of 2023. The biometric technology, which creates a unique palm signature from vein patterns, is used for payments and identification. Privacy advocates and lawmakers have raised concerns about surveillance, data sharing, and security risks. Amazon faces a class action lawsuit under New York City's biometric privacy law for allegedly failing to provide adequate notice.
- Company involved
- Amazon
- AI system involved
- Amazon One
9 source articles · read the reporting →
EPIC lawsuit challenges USPS secret surveillance program using facial recognition
The United States Postal Service's law enforcement wing, the Postal Inspection Service, ran a secret surveillance program called the Internet Covert Operations Program (iCOP) that used facial recognition from Clearview AI and social media monitoring tools to infiltrate online communities and monitor protests. The Electronic Privacy Information Center (EPIC) sued the USPS for failing to conduct and publish a privacy impact assessment as required by the E-Government Act. A federal district court dismissed the lawsuit on standing grounds, ruling that EPIC did not suffer a cognizable injury from the agency's refusal to disclose information about the program.
- Company involved
- United States Postal Service
- AI system involved
- Internet Covert Operations Program (iCOP)
10 source articles · read the reporting →
California EDD's automated fraud detection wrongly suspended 600,000 legitimate unemployment claims
In January 2021, the California Employment Development Department used Thompson Reuters automated batch review software to flag 1.1 million unemployment claims as potentially fraudulent. EDD stopped payments on those claims without prior notice. Later, over 600,000 were confirmed as legitimate after claimants used ID.me to verify their identity. The incident highlights the trade-off between fraud prevention and timely benefit access.
- Company involved
- California Employment Development Department (EDD)
- AI system involved
- Thompson Reuters Automated Batch Review
7 source articles · read the reporting →
Amazon Fresh drops flawed AI-powered Just Walk Out checkout system
Amazon is discontinuing its 'Just Walk Out' technology from Amazon Fresh grocery stores after the system failed to work reliably. The system, which used computer vision and AI to automatically charge customers, actually required over 1,000 human reviewers in India to manually verify transactions. Reports indicate that human review was needed for 700 out of every 1,000 sales, far exceeding Amazon's target. Amazon will replace the system with self-checkout shopping carts.
- Company involved
- Amazon
- AI system involved
- Just Walk Out
10 source articles · read the reporting →
Swedish welfare agency's AI system flags marginalized groups for fraud investigations
Försäkringskassan, Sweden's Social Insurance Agency, uses an AI risk-scoring system to flag welfare applicants for fraud investigations. The system disproportionately targets women, individuals with foreign backgrounds, low-income earners, and those without university degrees, according to an investigation by Lighthouse Reports and Svenska Dagbladet. Amnesty International has called for the system to be discontinued, citing violations of the right to equality and non-discrimination.
- Company involved
- Försäkringskassan (Swedish Social Insurance Agency)
6 source articles · read the reporting →
Chicago class action challenges ShotSpotter's discriminatory stop-and-frisk deployments
ShotSpotter, a gunshot detection system used by the Chicago Police Department, generates alerts that lead to police deployments and stop-and-frisk stops. Analyses show over 90% of alerts are unfounded, and the system is deployed only in predominantly Black and Latinx districts. A class-action lawsuit against the City of Chicago alleges the system fuels unconstitutional and discriminatory policing, including false accusations against individuals like Michael Williams.
- Company involved
- City of Chicago
- AI system involved
- ShotSpotter
7 source articles · read the reporting →
Woman denied state welfare card groceries after face scan fails
A grandmother rode her motorcycle several kilometres to a shop in Sapphaya district, Chai Nat province, to use her state welfare card to buy groceries for her family. After selecting items, she failed repeated facial recognition scans because her ID card photo was years old and no longer matched her appearance. She left the shop empty-handed and in tears.
- Company involved
- Government of Thailand (state welfare card system)
2 source articles · read the reporting →
DWP algorithm mistakenly flags 200,000 Housing Benefit claimants for fraud review
The UK Department for Work and Pensions (DWP) uses an algorithm to flag Housing Benefit claimants for possible fraud or error. According to a Big Brother Watch investigation, the algorithm has mistakenly flagged 200,000 innocent people, subjecting them to intrusive reviews. Only one in three flagged cases actually had errors, compared to two in three during the pilot. The DWP has spent £4.4 million on these pointless checks.
- Company involved
- Department for Work and Pensions (DWP)
6 source articles · read the reporting →
Jacksons Food Stores sued over facial recognition in Portland
Jacksons Food Stores is accused of violating a Portland, Oregon, city ordinance by using facial recognition technology in its stores after the ban took effect. The lawsuit, filed in December 2022, alleges the system, supplied by Blue Line Technology, wrongly identifies people as criminals and disproportionately affects women and people of colour. Customers were required to look at a camera and be scanned before entering. The retailer has not responded to the allegations.
- Company involved
- Jacksons Food Stores
- AI system involved
- First Line
10 source articles · read the reporting →
DWP algorithm approved Kickstart gateways with no trading history or based abroad
An FE Week investigation found that the Department for Work and Pensions (DWP) approved dozens of companies as Kickstart gateways through automated due diligence checks using the Cabinet Office Spotlight Tool, although some had little or no trading history or were based abroad. The DWP said gateways were subject to stringent checks and later said human checks were also used. After the findings were shared with the Treasury and the DWP, the department stopped taking gateway applications and scrapped the requirement for small employers to use gateways from 3 February.
- Company involved
- Department for Work and Pensions
- AI system involved
- Cabinet Office Spotlight Tool
3 source articles · read the reporting →
Kroger's digital price tags investigated for surge pricing concerns
U.S. Senators Elizabeth Warren and Bob Casey sent a letter to Kroger CEO Rodney McMullen raising concerns about the company's use of Electronic Shelving Labels (ESLs) for dynamic pricing. The system, called EDGE shelf and developed with Microsoft, can change prices based on time of day or weather, potentially leading to price gouging. The senators warned that this could harm consumers by increasing grocery costs and requested information from Kroger about its use of the technology.
- Company involved
- Kroger
- AI system involved
- EDGE shelf (Electronic Shelving Labels)
10 source articles · read the reporting →
Aadhaar biometric glitch denies rations, leads to starvation death
Motka Manjhi, a low-income villager in Jharkhand, India, was repeatedly denied subsidised food rations because his thumbprint failed to register on the Aadhaar biometric system. Unable to travel to a centre to update his data, he went without food and died of starvation on 22 May. His family believes the denial of rations caused his death. Activists allege that similar Aadhaar glitches have led to other starvation deaths.
- Company involved
- Government of India
- AI system involved
- Aadhaar
10 source articles · read the reporting →
