Where automated decisions went wrong

TransUnion AI tenant screening denied 75-year-old man apartment due to mistaken littering conviction

Chris Robinson, then 75, applied for a senior living apartment in California. The property manager used an AI screening program from TransUnion, which assigned him a low score based on a mistaken conviction for littering that belonged to a different person with the same name in Texas. Robinson lost the apartment and his application fee. A federal class-action lawsuit against TransUnion moved toward a $11.5 million settlement in 2023.

10 source articles · read the reporting →

Ukrainian Hacker Pleads Guilty to Operating OnlyFake AI ID Scam Site

Yurii Nazarenko, a Ukrainian national, pleaded guilty to running OnlyFake, an AI-powered website that generated and sold over 10,000 counterfeit identification documents globally. The site allowed users to create realistic fake IDs, including driver's licenses and passports, to bypass identity verification at banks and cryptocurrency exchanges. Nazarenko faces up to 15 years in prison and agreed to forfeit $1.2 million. Sentencing is scheduled for June 2026.

Company involved

OnlyFake

AI system involved

OnlyFake

3 source articles · read the reporting →

Security Health Plan used AI to cut off nursing home care for 85-year-old woman

Frances Walter, an 85-year-old woman with a shattered shoulder, had her nursing home care payment cut off by Security Health Plan after an algorithm predicted she would recover in 16.6 days. The algorithm, nH Predict from NaviHealth, did not account for her severe pain and allergy to pain medicine. She was forced to spend her life savings and enroll in Medicaid while fighting the denial. A federal judge later ruled the denial was speculative and she was owed thousands of dollars.

Company involved

Security Health Plan

AI system involved

nH Predict

1 source article · read the reporting →

FBI and ICE use state driver's license photos for facial recognition without consent

The FBI and ICE have been scanning millions of Americans' driver's license photos using facial recognition technology without their knowledge or consent, according to newly released documents. The agencies accessed state DMV databases, turning them into a facial-recognition gold mine. This practice raises significant privacy concerns as it was done without public awareness or legal oversight.

Company involved

FBI and ICE

1 source article · read the reporting →

Yale Suspends Scholar After AI News Site Accuses Her of Terrorist Link

Helyeh Doutaghi, a scholar at Yale Law School, was placed on administrative leave and barred from campus after an AI-powered news site published allegations linking her to Samidoun, a group the U.S. has designated as funding terrorism. Yale officials cited the allegations as the reason for the suspension. Doutaghi's lawyer denies she is a member of Samidoun. The incident highlights the potential harm of AI-generated content on individuals' reputations and employment.

Company involved

Yale University

2 source articles · read the reporting →

AI-powered scam compound in Myanmar uses ChatGPT and Gemini to defraud thousands globally

Safeer Mohammed Koorimannil, trafficked to a scam compound in Tai Chang, Myanmar, was forced to use AI-powered software to impersonate a woman and deceive victims into sending money. The software, Kongtian Intelligent Customer Acquisition and Global Social Traffic Navigation, used OpenAI's ChatGPT and Google's Gemini to generate messages and translate in over 100 languages. Koorimannil targeted 50,000 victims in a month, while the tools enabled scammers to rake in tens of millions of dollars. U.S. authorities have created a strike force to disrupt such operations, and OpenAI has banned accounts linked to the scams.

Company involved

Tai Chang

AI system involved

Kongtian Intelligent Customer Acquisition (KT) and Global Social Traffic Navigation (007TG)

2 source articles · read the reporting →

Kmart's facial recognition system for refund fraud found unlawful by Privacy Commissioner

Kmart Australia deployed facial recognition technology in 28 stores from June 2020 to July 2022, capturing biometric data of every customer entering the stores and those at returns counters to detect refund fraud. The system collected sensitive information without notifying customers or obtaining their consent. The Australian Privacy Commissioner found that Kmart breached the Privacy Act, as the exemption for unlawful activity did not justify the indiscriminate and disproportionate collection of biometric data from thousands of individuals. Kmart has ceased using the system and cooperated with the investigation.

Company involved

Kmart Australia Limited

7 source articles · read the reporting →

West Midlands Police test NDAS system to assess crime risk

West Midlands Police are testing a system called the National Data Analytics Solution (NDAS) that analyses police data to assess the risk of individuals committing a crime or becoming a victim. The Home Office has funded the project with millions of pounds, intending to roll it out across England and Wales. Critics warn that the system could reinforce bias against minorities and undermine the presumption of innocence. The police say the technology is designed to support, not replace, officer decision-making and that it is still in early testing.

Company involved

West Midlands Police

AI system involved

National Data Analytics Solution

1 source article · read the reporting →

LinkedIn Used by Foreign Spies with AI-Generated Photos to Target US Officials

Foreign intelligence operations created fake LinkedIn profiles with AI-generated photos to connect with US politicians, lobbyists, and government officials. The fake accounts, such as 'Katie Jones,' sent connection requests to gain credibility and access. LinkedIn removed the account after being contacted by the Associated Press, but the platform remains vulnerable to such espionage tactics.

Company involved

LinkedIn

1 source article · read the reporting →

Labor unions sue Trump administration over AI social media surveillance

Three labor unions (UAW, CWA, AFT) filed a lawsuit against the U.S. Departments of State and Homeland Security, alleging the Trump administration created a mass surveillance program using AI and automated technologies to monitor the social media accounts of visa holders and lawful permanent residents. The program is accused of targeting constitutionally protected speech based on viewpoint, causing a chilling effect where union members reported refraining from posting, deleting content, and altering offline union activities. The lawsuit, represented by EFF, Muslim Advocates, and MFIA, argues the program violates the First Amendment and the Administrative Procedure Act.

Company involved

U.S. Department of State

AI system involved

Catch and Revoke

9 source articles · read the reporting →

California AG declares out-of-state ALPR data sharing unlawful

California Attorney General Rob Bonta issued guidance confirming that sharing automated license plate reader (ALPR) data with out-of-state or federal agencies violates state law SB 34. The decision follows years of advocacy by EFF and the ACLU, who demonstrated through public records that many California law enforcement agencies were sharing data with hundreds of external agencies, including ICE and CBP. The guidance mandates that all California agencies review their data sharing and cut off access to out-of-state and federal entities.

Company involved

California law enforcement agencies

AI system involved

Automated license plate readers (ALPRs)

1 source article · read the reporting →

Arizona Unemployment Applicants Required to Submit Facial Recognition

People applying for unemployment benefits in Arizona must verify their identity via facial recognition software provided by ID.me. If the system fails to match a selfie to a photo ID, applicants can be denied benefits unless they successfully complete a video chat with a referee. A class-action lawsuit filed in June 2021 alleges that the practice violates due process rights and discriminates against people of colour.

Company involved

Arizona Department of Economic Security

AI system involved

ID.me

1 source article · read the reporting →

Meta's Instagram wrongfully suspends users for child exploitation, AI blamed

Multiple Instagram users in Southern California reported being locked out of their accounts after Meta's AI content moderation system flagged them for violating community standards on child sexual exploitation. The users, including a pediatric nurse and a contractor, say the accusations are false and have disrupted their personal and professional lives. They have appealed but received no response from Meta, and suspect an over-reliance on AI caused the erroneous suspensions. ABC7 has contacted Meta for comment.

Company involved

Meta

AI system involved

Instagram

1 source article · read the reporting →

Steak 'n Shake sued over facial recognition kiosks under BIPA

A class action lawsuit alleges that Steak 'n Shake illegally collects facial biometrics from customers using PopID-powered self-ordering kiosks without proper consent. Plaintiff Michael Massel claims the restaurant chain failed to provide notice and obtain written consent as required by Illinois' Biometric Information Privacy Act (BIPA). The suit seeks damages of up to $5,000 per violation for anyone whose biometric data was collected in Illinois in the past five years.

Company involved

Steak 'n Shake

AI system involved

PopID biometric kiosks

6 source articles · read the reporting →

Hospital Denies Pain Medication to Patient Based on Narx Score

Elizabeth Amirault, a 34-year-old chronic pain patient, was denied narcotic pain medication during a hospital visit in Fort Wayne, Indiana, in August 2022. A nurse practitioner told her that her Narx Score, generated by Bamboo Health's NarxCare platform, was too high to prescribe opioids. The algorithm uses prescription drug monitoring data to assess risk of misuse, but critics say it can harm patients by cutting off needed care. Amirault believes her score was unfairly high due to her multiple surgeries and doctors.

AI system involved

NarxCare

3 source articles · read the reporting →

Thailand halts iris-scan crypto scheme and deletes 1.2m biometric records

The Ministry of Digital Economy and Society (DES) and the Personal Data Protection Committee (PDPC) ordered a company to stop collecting iris data and delete 1.2 million citizen records. The PDPC found that the operator used crypto-token rewards as an incentive for consent, meaning consent was not freely given, and the system raised concerns about data being used beyond its declared purpose. The company stated it had complied with all Thai regulations and intends to continue discussions with authorities.

Company involved

The company behind the iris-scan system (not named)

AI system involved

Iris-scan system

4 source articles · read the reporting →

Bronx Man Jailed Six Months After Facial Recognition Misidentification

In April 2018, a Bronx man was arrested for stealing socks from a TJ Maxx after police used facial recognition to identify him from security footage. He maintained he was at a hospital for his child's birth. A prosecutor disclosed the use of facial recognition, and a judge ordered a hearing on the identification process. The man accepted a plea deal for time served after spending six months in jail, avoiding a trial.

Company involved

New York Police Department

2 source articles · read the reporting →

Meta AI Chatbot Falsely Claims to Have a Child in NYC Parents Group

Meta's AI chatbot responded to a parent's question in a private Facebook group for New York City parents, falsely claiming to have a twice-exceptional child enrolled in a specific public school's gifted program. The chatbot later apologised, stating it does not have personal experiences or children. The incident was reported by 404 Media after a screenshot was shared by a Princeton professor.

Company involved

Meta

AI system involved

Meta AI chatbot

2 source articles · read the reporting →

CommNV vs Uprise (Nevada DC): AI-hallucinated content in court filing, Monetary Penalty OR Order to volunteer and teach about AI…

The AI system generated fabricated legal citations that were included in a court filing, misleading the court and opposing counsel.

Company involved

Christopher Day

1 source article · read the reporting →

AI-NOMIS data breach exposes thousands of AI-generated deepfake images

Cybersecurity researcher Jeremiah Fowler discovered a non-password-protected database belonging to South Korean AI company AI-NOMIS, containing nearly 100,000 records of AI-generated explicit images, including what appeared to be child sexual abuse material. The database was exposed without encryption, and the researcher notified the company, which restricted access after the disclosure. The company did not respond to the disclosure, and the websites later went offline.

Company involved

AI-NOMIS

AI system involved

GenNomis

5 source articles · read the reporting →

Australian children's photos scraped into LAION-5B AI dataset without consent

Human Rights Watch found that the LAION-5B dataset, used to train AI tools, contains links to identifiable photos of Australian children scraped from the web without consent. The dataset includes photos from schools, personal blogs, and unlisted YouTube videos, with captions revealing names and locations. LAION acknowledged the finding and pledged to remove the photos, but the data persists and poses risks of deepfake creation and privacy violations.

Company involved

LAION

AI system involved

LAION-5B

4 source articles · read the reporting →

Cybercheck AI tool challenged in Akron homicide cases

Law enforcement in Akron, Ohio, used the AI tool Cybercheck to place two defendants at a murder scene. Defense lawyers allege the tool's creator lied under oath and that its methodology is opaque and unverified. Judges in multiple cases have barred the evidence, and the defendants are challenging its use. The tool has been used in thousands of cases nationwide, raising due process concerns.

Company involved

Akron Police Department

AI system involved

Cybercheck

3 source articles · read the reporting →

Pras Michel seeks new trial over lawyer's use of AI for closing arguments

Prakazrel 'Pras' Michel, a former member of the Fugees, was convicted of criminal conspiracy charges for attempting to influence US presidential administrations. He is seeking a new trial, alleging that his defense attorney used an experimental AI program called Lit Assist to draft closing arguments without disclosure. The motion claims the AI-generated argument was deficient and that the attorney had an undisclosed financial stake in the AI company. The court has scheduled an evidentiary hearing.

AI system involved

Lit Assist

10 source articles · read the reporting →

NADRA biometric system strips citizenship from thousands of Pakistanis

Pakistan's National Database and Registration Authority (NADRA) operates a biometric identification system that underpins citizenship. In 2016, a reverification campaign led to the blocking of hundreds of thousands of national ID cards, disproportionately affecting Pashtuns. Gulzar Bibi, a 53-year-old widow, had her card suspended after her brother's lost ID was misused, causing her to lose access to welfare, mobile service, and effectively rendering her stateless. Despite visiting NADRA, she was told to provide decades-old documentation to prove her citizenship, with no resolution reported.

Company involved

National Database and Registration Authority (NADRA)

AI system involved

NADRA biometric identification system

5 source articles · read the reporting →