Where automated decisions went wrong

Mumbai Cyber Police Bust Deepfake Share Trading Scam by Valueleaf

Mumbai Cyber Police arrested four individuals from Bengaluru-based advertising agency Valueleaf for allegedly circulating deepfake videos of stock market experts to deceive investors. The videos, created for Hong Kong-based First Bridge, were promoted despite knowledge of their falsity and potential financial harm. Meta flagged the content, but the accused evaded detection by increasing ad accounts and changing domain locations. The case, registered under the Bhartiya Nyaya Sanhita and Information Technology Act, is under investigation to determine the number of defrauded investors.

Company involved

Valueleaf

1 source article · read the reporting →

Knight Capital fined $12 million for 2012 trading glitch that caused $460 million loss

On August 1, 2012, Knight Capital Americas LLC's automated equity router malfunctioned due to a code deployment error, sending over 4 million erroneous orders into the market within 45 minutes. The firm suffered a loss of more than $460 million. The SEC charged Knight Capital with violating the market access rule and Regulation SHO, and the firm agreed to pay a $12 million penalty and retain an independent consultant.

Company involved

Knight Capital Americas LLC

AI system involved

Knight Capital's automated equity router

10 source articles · read the reporting →

CFPB fines General Information Services for inaccurate background checks

The Consumer Financial Protection Bureau took action against General Information Services and its affiliate e-Background-checks.com for failing to ensure the accuracy of employment background screening reports. The companies allegedly provided inaccurate criminal history information to employers, potentially affecting job applicants' eligibility and causing reputational harm. The CFPB ordered the companies to provide $10.5 million in relief to harmed consumers and pay a $2.5 million penalty.

Company involved

General Information Services

10 source articles · read the reporting →

Hong Kong tycoon sues Tyndaris over AI hedge fund losses

Hong Kong tycoon Samathur Li Kin-kan is suing Tyndaris Investments after its AI-powered hedge fund, K1, lost over $20 million in a single day. Li alleges that the system was not as sophisticated as claimed. The trial is set to begin in London in April 2020.

Company involved

Tyndaris Investments

AI system involved

K1

10 source articles · read the reporting →

Wells Fargo software error caused hundreds of faulty foreclosures

Wells Fargo disclosed that a software error in its mortgage modification underwriting tool caused incorrect denials of loan modifications for approximately 870 borrowers between 2010 and 2015. The error miscalculated attorneys' fees, leading to 545 foreclosures that should not have occurred. Wells Fargo set aside $8 million for remediation and is contacting affected customers to offer compensation and mediation.

Company involved

Wells Fargo

3 source articles · read the reporting →

Fraudsters use AI voice deepfake to trick UK energy firm CEO into transferring $243,000

In March 2019, criminals used commercially available AI voice-generation software to impersonate the CEO of a German parent company. They tricked the CEO of a UK-based energy firm into urgently wiring $243,000 to a Hungarian supplier. The fraud was discovered when the fraudsters attempted a second transfer, which the CEO refused. The company was insured and the loss was covered.

Company involved

Unnamed UK-based energy firm

AI system involved

Commercially available voice-generating AI software

10 source articles · read the reporting →

Deepfake video of Sam Bankman-Fried used in crypto phishing scam

A verified Twitter account posted a deepfake video of FTX founder Sam Bankman-Fried claiming to offer compensation to users of the collapsed exchange. The video directed viewers to a phishing website, ftxcompensation.com, that asked them to connect their crypto wallets and send funds with the promise of doubling them. The site's Ethereum address reportedly received over $1,000 in ETH. The account was later suspended.

10 source articles · read the reporting →

Allstate's proposed retention model charged big spenders more in Maryland

Allstate proposed an auto insurance rate adjustment plan in Maryland using a retention model algorithm that varied premium changes based on existing premiums. The algorithm would have increased rates up to 20% for customers already paying the highest premiums while capping increases at 5% for others, and would have denied meaningful discounts to thousands owed reductions. The Maryland Insurance Administration rejected the plan as discriminatory, but Allstate claims it was withdrawn and has continued proposing similar models elsewhere. The proposal never took effect in Maryland.

Company involved

Allstate Corporation

AI system involved

retention model

9 source articles · read the reporting →

AI deepfake fraud dupes man into transferring $612,000

A man surnamed Guo, a legal representative of a technology company in Fuzhou, received a video call from someone impersonating his friend using AI to mimic the friend's face and voice. The fake friend asked Guo to transfer 4.3 million yuan to a bank account, claiming it was for a bid. Guo transferred the money and later discovered the fraud when he contacted his real friend. Police stopped part of the transfer and are investigating.

10 source articles · read the reporting →

Mercadona fined €2.5 million for facial recognition pilot

Mercadona, a Spanish supermarket chain, tested an early-detection system using facial recognition in 48 stores to identify people with judicial restraining orders. The system, which operated with court authorisation, notified police when such a person was detected. The Spanish data protection authority (AEPD) imposed a €2.5 million fine, which Mercadona paid, and the company has since removed the system citing legal uncertainty.

Company involved

Mercadona

AI system involved

Sistema de Detección Anticipada (Early Detection System)

10 source articles · read the reporting →

California regulator accuses Maxpread Technologies of using AI-generated fake CEO to scam investors

Maxpread Technologies allegedly used an AI-generated avatar to pose as its CEO in a YouTube video to deceive investors. The California DFPI issued cease and desist letters to Maxpread and four other companies for offering unqualified securities and making false promises. The regulator claims the companies promised high daily returns using AI trading, but these claims were false. Maxpread did not respond to requests for comment.

Company involved

Maxpread Technologies

8 source articles · read the reporting →

Amazon expands palm-scanning payments to all Whole Foods stores amid privacy concerns

Amazon is expanding its Amazon One palm-scanning payment system to all Whole Foods stores by the end of 2023. The biometric technology, which creates a unique palm signature from vein patterns, is used for payments and identification. Privacy advocates and lawmakers have raised concerns about surveillance, data sharing, and security risks. Amazon faces a class action lawsuit under New York City's biometric privacy law for allegedly failing to provide adequate notice.

Company involved

Amazon

AI system involved

Amazon One

9 source articles · read the reporting →

FTC settles with IntelliVision over deceptive facial recognition claims

The Federal Trade Commission (FTC) took action against IntelliVision Technologies Corp. for making false, misleading, or unsubstantiated claims about its AI-powered facial recognition software. The company allegedly claimed its software had one of the highest accuracy rates on the market and was free of gender or racial bias, without supporting evidence. The FTC also alleged that IntelliVision did not train its software on millions of faces as claimed, but on images of approximately 100,000 individuals. Under a proposed consent order, IntelliVision is prohibited from making such misrepresentations without competent and reliable testing.

Company involved

IntelliVision Technologies Corp.

AI system involved

IntelliVision facial recognition software

9 source articles · read the reporting →

ScaleFactor reportedly failed to deliver promised automated bookkeeping software

ScaleFactor, an Austin-based startup, is reported to have failed to deliver the automated, real-time bookkeeping tools it promised customers, instead relying on human bookkeepers and a Filipino contract accounting firm. The company told Forbes in June that it was shutting down, initially blaming the pandemic, but Forbes later reported that its problems predated Covid-19. Investors reportedly came to see the company as more of a services business than a software platform, and pulled funding after a pivot to a marketplace model. No legal or regulatory action is reported.

Company involved

ScaleFactor

10 source articles · read the reporting →

UnitedHealth used ALERT algorithm to limit mental health coverage, regulators found

ProPublica reports that UnitedHealth Group's Optum subsidiary used the ALERT algorithm to flag mental health patients and therapists as outliers, leading to therapy coverage denials. Regulators in California, Massachusetts and New York alleged this breached the federal Mental Health Parity and Addiction Equity Act, and UnitedHealth agreed to restrict the system in those jurisdictions. The company denies wrongdoing and says its programmes are compliant. Affected patients, including Medicaid enrollees, were left to pay out-of-pocket or go without care.

Company involved

UnitedHealth Group

AI system involved

ALERT

5 source articles · read the reporting →

Amazon faces class action over allegedly rigged buying algorithms

Two consumers filed a proposed class action against Amazon in Washington federal court, alleging that the company rigged its algorithms to boost sales of its own retail business and third-party sellers using its fulfillment service. The plaintiffs claim that hundreds of millions of customers have overpaid for items as a result. The lawsuit is pending.

Company involved

Amazon.com Inc.

AI system involved

Amazon's buying features

6 source articles · read the reporting →

SEC charges American Bitcoin Academy over $1.2M AI scam

Brian Sewell, through his company American Bitcoin Academy, allegedly defrauded 15 students of $1.2 million by claiming his Rockwell Fund would use AI to generate high returns. The SEC charged that Sewell never launched the fund and lost the investors' money when his Bitcoin wallet was hacked. The case was settled with Sewell agreeing to pay $1.6 million in disgorgement and a $233,229 penalty.

Company involved

American Bitcoin Academy

6 source articles · read the reporting →

Spanish police bust $20M AI-powered investment scam

Spanish law enforcement, collaborating with international authorities, dismantled a $20 million investment scam that used AI-driven algorithms to deceive individuals and organizations. Six suspects were detained and assets, including luxury cars and cryptocurrency, were seized. The article does not report any compensation for victims.

5 source articles · read the reporting →

Worldcoin collected biometric data from poor villagers in Indonesia without informed consent

Worldcoin, a cryptocurrency startup, recruited users in developing countries by offering free cash in exchange for iris scans. The company used deceptive marketing, collected more personal data than acknowledged, and failed to obtain meaningful informed consent. Many users received worthless tokens instead of promised money. The company acknowledged some friction but continued its operations.

Company involved

Worldcoin

AI system involved

chrome orb

5 source articles · read the reporting →

AI-generated article misstates Roadzen's Q1 revenue expectations

An AI-generated article on The Motley Fool incorrectly stated that Roadzen's analyst expectations for Q1 revenue were over $21 million, implying a revenue miss of more than 50%. Roadzen clarified that these figures were never issued by its covering analysts and that its actual revenue of $10.9 million was in line with estimates. The Motley Fool later corrected the article and added an editor's note acknowledging the error.

Company involved

The Motley Fool

4 source articles · read the reporting →

Italy fines Amazon $1.3 billion for algorithm favouring its logistics

Italy's antitrust regulator fined Amazon €1.13 billion ($1.3 billion) for abusing its market dominance. The regulator alleged that Amazon used an algorithm to give an unfair advantage to its own logistics service over third-party sellers. Amazon said it disagreed with the decision and would appeal.

Company involved

Amazon

9 source articles · read the reporting →

Delta uses AI from Fetcherr for domestic ticket pricing

Delta Air Lines is using generative AI from Fetcherr to determine some domestic flight prices, currently covering 3% of its network with plans to reach 20% by end of 2025. Democratic senators expressed concern that the AI could be used for individualized pricing based on personal data, leading to higher fares. Delta denies using personal data in pricing and states it complies with regulations. No actual harm has been reported.

Company involved

Delta Air Lines

AI system involved

Fetcherr

8 source articles · read the reporting →

Dutch DPA fines Clearview AI €30.5 million for GDPR violations

The Dutch Data Protection Authority fined Clearview AI €30.5 million for creating an illegal database of biometric codes scraped from social media without consent. The regulator accused the company of failing to inform individuals about how their data is used and continuing violations after investigation. Clearview AI denies the allegations, stating it has no presence in the Netherlands or EU and that the fine is unenforceable.

Company involved

Clearview AI

AI system involved

Clearview AI facial recognition database

7 source articles · read the reporting →

Bots and short sellers spread misinformation to trigger First Republic Bank run

A report by Valent Technologies alleges that AI-powered bots and fake accounts spread misinformation about First Republic Bank in March 2023, coinciding with a $100 billion deposit withdrawal and the bank's collapse. The report suggests short sellers likely orchestrated the bots to drive down the share price. Federal regulators closed the bank and sold its assets to J.P. Morgan, and the bank's former CEO attributed the collapse to 'contamination' by anxiety.

2 source articles · read the reporting →