West Midlands Police test NDAS system to assess crime risk
West Midlands Police are testing a system called the National Data Analytics Solution (NDAS) that analyses police data to assess the risk of individuals committing a crime or becoming a victim. The Home Office has funded the project with millions of pounds, intending to roll it out across England and Wales. Critics warn that the system could reinforce bias against minorities and undermine the presumption of innocence. The police say the technology is designed to support, not replace, officer decision-making and that it is still in early testing.
- Company involved
- West Midlands Police
- AI system involved
- National Data Analytics Solution
1 source article · read the reporting →
Ola drivers file GDPR lawsuit over algorithmic management and data access
Two ride-hailing drivers are taking Ola to court in Amsterdam, alleging the company violated GDPR data access rights by not providing full personal data, including GPS and ratings information. They argue this hinders their ability to challenge algorithmic decisions, such as a driver who had pay docked after Ola's system incorrectly flagged trips as invalid. The case, supported by the App Drivers & Couriers Union, seeks to compel Ola to comply with data protection law and faces fines for non-compliance. Ola says it has not received notification and complies with all applicable laws.
- Company involved
- Ola
- AI system involved
- Guardian
1 source article · read the reporting →
Arizona Unemployment Applicants Required to Submit Facial Recognition
People applying for unemployment benefits in Arizona must verify their identity via facial recognition software provided by ID.me. If the system fails to match a selfie to a photo ID, applicants can be denied benefits unless they successfully complete a video chat with a referee. A class-action lawsuit filed in June 2021 alleges that the practice violates due process rights and discriminates against people of colour.
- Company involved
- Arizona Department of Economic Security
- AI system involved
- ID.me
1 source article · read the reporting →
VA's Suicide Prevention Algorithm Favours Men, Ignores Sexual Trauma
The Department of Veterans Affairs' REACH VET algorithm, designed to identify veterans at high risk of suicide, reportedly prioritises white men and excludes factors like military sexual trauma that disproportionately affect women. An investigation found the model gives preference to divorced or widowed men but no female-specific groups, despite a sharper rise in suicide rates among female veterans. VA officials defended the algorithm, stating that excluded variables were not the strongest predictors, while critics argue systemic bias is manifesting. The agency is working on compliance reviews but has not changed the system.
- Company involved
- Department of Veterans Affairs
- AI system involved
- REACH VET
2 source articles · read the reporting →
Refugee claimants strike out after Federal Court says their legal claim is AI fakery - National Post
The Federal Court rejected the refugee claimants' appeal due to AI-generated fake legal arguments.
- Company involved
- OWS Law
1 source article · read the reporting →
Hospital Denies Pain Medication to Patient Based on Narx Score
Elizabeth Amirault, a 34-year-old chronic pain patient, was denied narcotic pain medication during a hospital visit in Fort Wayne, Indiana, in August 2022. A nurse practitioner told her that her Narx Score, generated by Bamboo Health's NarxCare platform, was too high to prescribe opioids. The algorithm uses prescription drug monitoring data to assess risk of misuse, but critics say it can harm patients by cutting off needed care. Amirault believes her score was unfairly high due to her multiple surgeries and doctors.
- AI system involved
- NarxCare
3 source articles · read the reporting →
State Farm accused of algorithmic bias against Black homeowners
A class action lawsuit filed in Chicago alleges State Farm's automated claims processing system discriminates against Black policyholders. The suit claims Black customers are more likely to face additional paperwork and longer waits for claim approval than white customers, citing data from 800 homeowners. State Farm has stated it takes the filing seriously and is committed to fair treatment, while the plaintiffs seek to enjoin the current system and require audits.
- Company involved
- State Farm
- AI system involved
- automated claims processing system
2 source articles · read the reporting →
Sainsbury's ejects man misidentified by facial recognition software
Warren Rajah was told to leave a Sainsbury's supermarket in Elephant and Castle, London, after staff incorrectly identified him as an offender flagged by Facewatch facial recognition software. The error occurred at the human verification stage, not the technology itself. Sainsbury's apologised and offered a £75 shopping voucher, and Facewatch confirmed Rajah was not on its database. Rajah criticised the lack of explanation and recourse, and expressed concern for vulnerable customers.
- Company involved
- Sainsbury's
- AI system involved
- Facewatch
5 source articles · read the reporting →
Herbert Brooks v. Lowes Home Centers LLC (W.D. Louisiana): AI-hallucinated content in court filing, Monetary Sanction; CLE
Generated a legal brief with hallucinated case law, filed in court.
- AI system involved
- Claude
1 source article · read the reporting →
CommNV vs Uprise (Nevada DC): AI-hallucinated content in court filing, Monetary Penalty OR Order to volunteer and teach about AI…
The AI system generated fabricated legal citations that were included in a court filing, misleading the court and opposing counsel.
- Company involved
- Christopher Day
1 source article · read the reporting →
Malaysia tests AI court sentencing, lawyer challenges harsh sentence
In Sabah, Malaysia, a lawyer's client faced a minor drug possession charge and was sentenced with the help of an AI tool that recommended a harsh penalty. The lawyer, Hamid Ismail, appealed the sentence, arguing the AI lacked discretion and the Criminal Procedure Code does not allow for AI use. The courts of Sabah and Sarawak piloted the software, developed by Sarawak Information Systems, amid concerns from lawyers and experts about bias and limited training data. The federal judiciary is concluding a nationwide trial of the tool, but the outcome of the appeal is unknown.
- Company involved
- Sabah and Sarawak Courts
3 source articles · read the reporting →
Hospital sues insurer over use of AI to deny $2M in claims - HealthExec
A hospital is suing Blue Cross and Blue Shield of Kansas City, alleging the insurer's AI flagged physicians' diagnoses as clinically invalid and led to the denial of about $2 million in claims.
- Company involved
- Blue Cross and Blue Shield of Kansas City
- AI system involved
- Apixio
1 source article · read the reporting →
Italian bank (Fideuram / Intesa Sanpaolo) loses about 95 million euro to AI voice-clone scam
The AI-generated voice deceived the chairman into authorizing transfers of about 95 million euros.
1 source article · read the reporting →
Lawsuit Says UnitedHealth Used AI With a 90% Error Rate to Deny Care - Yahoo
Denied post-acute care coverage to elderly Medicare Advantage patients
- Company involved
- UnitedHealth Group
- AI system involved
- nH Predict
1 source article · read the reporting →
Mr Craig Hadley wrongly accused of fraud at Sports Direct shop using Facewatch facial recognition.
Mr Craig Hadley was wrongly accused of being a fraudster at a Rotherham Sports Direct shop using facial recognition technology supplied by Facewatch. The system flagged him as a known fraudster, but the alert was later attributed to human error. Big Brother Watch, a privacy campaign group, has raised concerns about the lack of due process in such systems.
- Company involved
- Sports Direct
- AI system involved
- Facewatch
2 source articles · read the reporting →
Sue Neill-Fraser: AI hallucinations in a parole decision (Tasmania)
The Parole Board imposed a condition preventing Susan Neill-Fraser from communicating with the media to claim innocence.
- Company involved
- Tasmania Parole Board
1 source article · read the reporting →
Allegheny County child neglect screening algorithm shows racial disparities
The Allegheny Family Screening Tool, an algorithm used to screen child neglect reports, has been found to flag a disproportionate number of Black children for mandatory investigation compared to white children. A technical glitch also presented wrong risk scores for over two years. County officials say social workers can override the algorithm and that the glitch has been fixed, but critics argue the tool hardens racial disparities and lacks transparency.
- Company involved
- Allegheny County Department of Human Services
- AI system involved
- Allegheny Family Screening Tool
10 source articles · read the reporting →
Pearl Gardner v. Nationstar Mortgage LLC (D. Arizona): AI-hallucinated content in court filing, Bar Referral
Pro Se Litigant | D. Arizona | State(s): USA. AI tool: Clearpoint. Problem: Fabricated: Case Law. Consequence: Bar Referral.
1 source article · read the reporting →
Arkansas algorithm slashed home care for disabled people, causing suffering
In Arkansas and other states, algorithms were used to allocate home care hours for disabled and elderly people, replacing nurse judgments. The system assigned hours based on a scoring tool that failed to account for specific disabilities, leading to drastic cuts. Many suffered severe harm, including bed sores, missed meals, and one death. After lawsuits, Arkansas's system was thrown out in 2018, and Idaho's was declared unconstitutional.
- Company involved
- Arkansas state government
1 source article · read the reporting →
Airbnb bans users in Australia using trustworthiness algorithm
Airbnb is accused of using an algorithm acquired from Trooly to score users' trustworthiness based on publicly available data, including social media and occupation. Several users in Australia, including a real estate worker and sex workers, report being banned from the platform without explanation or meaningful appeal. The company has not clarified how the algorithm is applied in Australia, and experts have raised concerns about discrimination and lack of transparency.
- Company involved
- Airbnb
- AI system involved
- Trooly
4 source articles · read the reporting →
Capital Standard, LLC v. U.S. Bank National Association (CA Florida (2d)): AI-hallucinated content in court filing, Monetary Sanction; Adverse Costs…
The AI generated false legal citations that were included in a court filing, misleading the court.
1 source article · read the reporting →
AI deepfake scam cons Los Angeles woman out of life savings
A South Los Angeles woman, Abigail, was deceived by scammers using AI-generated deepfake videos to impersonate 'General Hospital' actor Steve Burton. Believing she was in a relationship with the celebrity, she sent at least $81,304 in gift cards, cash and bitcoin, and sold her condo for $350,000 before her daughter intervened. The family has filed a lawsuit to reverse the home sale, and the real Steve Burton says hundreds of fans have been similarly targeted. The victim, who has bipolar disorder, is now in debt and facing bankruptcy.
3 source articles · read the reporting →
DFFH breaches privacy by using ChatGPT in child protection report
A child protection worker at the Department of Families, Fairness and Housing (DFFH) used ChatGPT to draft a Protection Application Report for the Children’s Court, entering sensitive personal information about a child. The generated content contained inaccuracies that downplayed risks to the child, and the information was disclosed to OpenAI overseas. An investigation by the Office of the Victorian Information Commissioner found DFFH failed to ensure accuracy and protect personal information, contravening IPPs 3.1 and 4.1. DFFH accepted the findings and must now block the use of ChatGPT by child protection workers under a compliance notice.
- Company involved
- Department of Families, Fairness and Housing
- AI system involved
- ChatGPT
9 source articles · read the reporting →
Consumer Reports calls for action against Meta over scam ad proliferation
Consumer Reports has called on the FTC and state attorneys general to take enforcement action against Meta for allowing billions of scam advertisements on its platforms. According to a Reuters investigation, Meta's own documents showed that the company delivered an estimated 15 billion scam ads per day in 2024, generating billions of dollars in revenue. Meta is accused of failing to identify and remove most scam ads, exposing users to fraudulent schemes and illegal products. Consumer Reports argues that Meta's practices constitute unfair business practices under the FTC Act and state laws.
- Company involved
- Meta
6 source articles · read the reporting →
