Where automated decisions went wrong

Uber set for €825mn Dutch fine over automating driver suspensions - Financial Times

The system automatically suspended Uber drivers.

Company involved

Uber

1 source article · read the reporting →

Fake AI-Generated Steven Bartlett Video Used in Investment Scam

Scammers circulated an AI-generated deepfake video on Instagram impersonating entrepreneur Steven Bartlett, falsely claiming he was offering stock tips with guaranteed 35% returns. The video directed viewers to a WhatsApp group, where fraudsters could harvest personal details and eventually solicit money. Bartlett's team confirmed the video was a scam and is working with platforms to remove it. No specific victims were identified, but the article warns that such scams could cause individuals to lose thousands of pounds.

1 source article · read the reporting →

Job seeker asks First Circuit to revive class action over AI interviewing tool

Massachusetts financial consultant Mozart Saint Cyr leads a proposed class of job applicants who say JPMorgan Chase's use of HireVue one-way video interviews amounted to a lie detector test banned by state law. After a federal judge dismissed the case, they asked the First Circuit to revive it.

Company involved

JPMorgan Chase

AI system involved

HireVue one-way video interview

1 source article · read the reporting →

Ottawa couple loses $177,000 in deepfake Elon Musk investment scam

Doug and Victoria Lloyd lost $177,000 after being deceived by a deepfake video of Elon Musk promoting a fraudulent investment scheme. The couple from Eastern Ontario shared their story as part of a scam awareness series. The Ontario Provincial Police provided tips on passwords and facial recognition to help prevent such fraud.

2 source articles · read the reporting →

Resume prompt injection tricks AI hiring - moneywise.com

AI screening system determined which job applicants to advance to the next stage of recruitment.

1 source article · read the reporting →

Bengaluru Man Loses Rs 7.66 Lakh In Investment Scam Using Celebrity Deepfakes - News Mobile

AI-generated deepfake videos of public figures falsely endorsed an investment scheme, causing a man to invest and lose money.

Company involved

One Two Markets

1 source article · read the reporting →

AI chatbots used to steal US college financial aid

Crime rings are deploying AI chatbots as 'ghost students' to enrol in online college courses and fraudulently collect US federal financial aid. Victims of identity theft, such as Heather Brady and Brittnee Nelson, discovered loans of over $9,000 and $5,000 respectively taken out in their names for colleges they never attended. The US Education Department introduced a temporary rule requiring government-issued ID for first-time aid applicants, while California community colleges reported losing at least $11.1 million to such scams.

Company involved

Delgado Community College

4 source articles · read the reporting →

How Your Shadow Credit Score Could Decide Whether You Get an Apartment - ProPublica

Tenant screening companies assign renters a score drawn from data far wider than credit history, an industry subject to less regulation than credit scoring agencies. ProPublica reports that experts warn these algorithms can decide who gets an apartment on the basis of information applicants cannot see or correct. Kim Fuller was denied a rental application on such a score.

Company involved

Habitat America

AI system involved

RentGrow

1 source article · read the reporting →

Goldman Sachs Apple Card Algorithm Accused of Gender Bias in Credit Limits

In late 2019, David Heinemeier Hansson alleged on Twitter that the Apple Card underwriting algorithm, operated by Goldman Sachs, gave him a higher credit limit than his wife despite similar financial profiles. The New York Department of Financial Services investigated and found no violation of fair lending laws, but critics argue the audit methodology was outdated and failed to detect proxy discrimination. Apple later updated its credit policy to allow spouses to combine credit files, acknowledging a lack of fairness in industry credit scoring.

Company involved

Goldman Sachs

AI system involved

Apple Card

6 source articles · read the reporting →

Uber's Secretive 'Upfront Fares' Algorithm Cuts Driver Pay in Many Cities

Uber introduced an opaque algorithm called 'Upfront Fares' to calculate driver pay in 24 U.S. cities, replacing the previous time-and-distance-based rate. Drivers report lower and unpredictable earnings, with Uber taking a larger cut of fares. The company claims the change gives drivers more control, but critics say the secrecy makes it impossible to verify fair pay. The algorithm may learn the lowest rate drivers will accept, potentially driving down wages.

Company involved

Uber

AI system involved

Upfront Fares

3 source articles · read the reporting →

Nomani Investment Scam Uses AI Video Testimonials to Steal Money and Data

Cybersecurity firm ESET reported that an investment scam campaign named Nomani used AI-generated video testimonials of famous people in social media adverts to lure victims worldwide to phishing websites. The scammers harvested personal information, then telephoned victims and manipulated them into investing in non-existent products, with some taking out loans or installing remote access apps. When victims sought payouts, they were told to pay extra fees and provide ID and credit-card details, after which the fraudsters disappeared with their money and data. South Korean authorities separately said they dismantled a related fake trading platform network that defrauded victims of nearly $6.3 million.

Company involved

Nomani

AI system involved

Nomani

1 source article · read the reporting →

CFPB Acts Against Hello Digit for Faulty Savings Algorithm Causing Overdrafts

Hello Digit, a fintech company, used an automated savings algorithm that made transfers from consumers' checking accounts, falsely guaranteeing no overdrafts. The algorithm caused customers to incur overdraft fees, and the company often denied reimbursement requests. The CFPB found that Hello Digit engaged in deceptive practices and ordered the company to pay redress to harmed consumers and a $2.7 million fine.

Company involved

Hello Digit, LLC

AI system involved

Hello Digit app

1 source article · read the reporting →

Pittsburg man loses $1,900 in AI-generated Trump investment scam

Wesley Skelton, a Marine Corps veteran from Pittsburg, California, was duped into buying 324 'golden eagles' for $2,500 after watching AI-generated videos of President Trump, Bank of America's CEO, and Elon Musk promoting the Golden Eagles Project. The videos, which appeared on Telegram, falsely claimed the collectible coins could be traded in for over $100,000 each at Bank of America. After discovering the items were worthless, Skelton received a partial refund of $600 from his credit card processor, but the seller did not respond. The scam highlights the use of AI-generated deepfakes to perpetrate financial fraud.

1 source article · read the reporting →

Schufa's Black-Box Scoring Unfairly Penalises Consumers with Positive Credit Data

An investigation by SPIEGEL and BR Data reveals that Schufa's credit scoring algorithm often assigns poor risk scores to consumers with only positive credit information. One consumer, Sven Drewert, was denied a credit card limit increase despite having no negative entries. The algorithm uses limited data, and its secret formula can lead to arbitrary categorisations, affecting access to loans, phone contracts, and housing. The system's opacity and potential biases raise concerns about fairness and accountability.

Company involved

Schufa Holding AG

AI system involved

Schufa Score

2 source articles · read the reporting →

Portland Water Bureau AI pilot offers discount to billionaire CEO

The Portland Water Bureau used a machine learning system in a randomised control trial to identify customers for its financial assistance programme. The system selected Columbia Sportswear CEO Tim Boyle, a billionaire and one of the city's largest residential water consumers, for a 40% water bill discount. Boyle declined the discount, stating it should go to someone who needs it. The bureau is testing whether the SERVUS algorithm can accurately identify customers' ability to pay.

Company involved

Portland Water Bureau

AI system involved

Smart Discount Program

1 source article · read the reporting →

FinTech and traditional lenders discriminate against minority borrowers in mortgage pricing

A study of mortgage lending from 2012-2018 found that Latinx and African-American borrowers were charged higher interest rates than white borrowers with similar credit risk. FinTech algorithms reduced the disparity by 40% but did not eliminate it. The discrimination costs minority borrowers an estimated $765 million per year in extra interest.

10 source articles · read the reporting →

West Virginia Couple Targeted in AI-Generated Real Estate Scam

Raegan Bartlo received an email on August 23, 2024, that appeared to be from her title company lawyer, instructing her to wire a $255,000 down payment for her new home. The email was part of a scam using generative AI to impersonate real estate professionals. Cybersecurity officials say such scams are occurring across the country, targeting homebuyers. The Bartlos were targets, but it is unclear if they lost any money.

1 source article · read the reporting →

DoNotPay sued for practicing law without a license in California

Jonathan Faridian, a paying customer of DoNotPay, alleges that the AI-powered legal service provided poorly drafted and unusable legal documents, including demand letters and court filings. He claims that DoNotPay is not a lawyer or law firm and engaged in unauthorized practice of law. The lawsuit, filed in California, seeks class action status and restitution for all paying customers in the state. DoNotPay denies the allegations, stating that the plaintiff had submitted successful cases and that the lawsuit is meritless.

Company involved

DoNotPay

AI system involved

DoNotPay

2 source articles · read the reporting →

RealPage's YieldStar algorithm pushes rents higher for tenants

RealPage's YieldStar algorithm uses private competitor data to recommend rent increases for apartment units. Landlords adopt up to 90% of the suggestions, leading to higher rents for tenants. Critics allege the software may facilitate indirect price-fixing in violation of antitrust law. The company denies wrongdoing and says the software helps eliminate collusion.

Company involved

RealPage

AI system involved

YieldStar

10 source articles · read the reporting →

UDR Sued Over Alleged Use of RealPage Algorithm to Set Rents in San Diego

A class action lawsuit alleges that UDR, Inc. used RealPage's YieldStar algorithmic pricing software to set rental rates and occupancy levels for its San Diego properties, in violation of a local ordinance. The suit claims the software relied on nonpublic competitor data, contributing to inflated rents and making housing less affordable. The lawsuit, filed in July 2026, seeks to represent all affected tenants. UDR has previously acknowledged using YieldStar as one tool among others in its decision-making.

Company involved

UDR, Inc.

AI system involved

RealPage YieldStar

1 source article · read the reporting →

SEC warns public of deep fake investment scams featuring Lance Gokongwei

The Securities and Exchange Commission (SEC) warned the public that scammers are using deep fake videos and audio of Lance Gokongwei to endorse fraudulent investment schemes. The manipulated media circulate on social media, deceiving people into investing in a platform registered in Cyprus. Victims are asked to provide credit card details and OTPs, then lose contact when attempting to withdraw funds. The SEC advises the public to verify investment offers with the agency.

2 source articles · read the reporting →

Barclays pilot of Sapience monitoring software causes employee stress

Barclays introduced a pilot of employee monitoring software from Sapience in its product control department at Canary Wharf. The system monitors computer activity and admonishes staff if they are not deemed active enough, recording breaks as "unaccounted activity". Employees reported significant stress and worry about taking breaks. Barclays acknowledged the pilot and said it would listen to feedback.

Company involved

Barclays

AI system involved

Sapience employee monitoring software

10 source articles · read the reporting →

Gradient app charges users unexpected subscription fees after free trial

The Gradient app, a celebrity lookalike app promoted by the Kardashians, charges users $19.99 per month after a three-day free trial without clear disclosure. Users complained on social media about unexpected credit card charges. The app's developer, Ticket to the Moon, has not addressed the billing complaints but denied collecting user data.

Company involved

Ticket to the Moon, Inc.

AI system involved

Gradient

9 source articles · read the reporting →

Google AI Overviews provide inaccurate finance information in 43% of searches

A study by The College Investor found that Google's AI Overviews provided misleading or inaccurate information in 43% of 100 personal finance-related searches. The AI-generated answers included outdated tax rules and incorrect student loan repayment plan details. One user believed she could convert her 529 plan to a Roth IRA in California, which is not allowed. Google has not responded to requests for comment.

Company involved

Google

AI system involved

AI Overviews

3 source articles · read the reporting →