Where automated decisions went wrong

"This is so sad": AI nurse scam costs Australian pensioner R2m - briefly.co.za

The AI-generated deepfake video persona convinced the pensioner to send money.

1 source article · read the reporting →

Woman Scammed by Deepfake Ad of Her Doctor for Fake Lipedema Cream

Beth Holland, who has lipedema, saw a Facebook ad for a cream called Svelta Venastra that appeared to be endorsed by her doctor and celebrities. The ad was an AI-generated deepfake, and she paid $300 for six bottles. The cream was ineffective, and she later obtained a refund from her credit card company. The incident is part of a rise in medical scams using deepfakes.

1 source article · read the reporting →

Meta makes $$$ from deepfake AI scams… then fails to assist victims - Independent Australia

Meta makes $$$ from deepfake AI scams… then fails to assist victims Independent Australia

Company involved

Meta

1 source article · read the reporting →

Cigna uses algorithm to deny medical claims without doctor review

Cigna, one of the largest health insurers, used an algorithm called PXDX to automatically deny medical claims. The system flagged mismatches between diagnoses and procedures, and company doctors signed off on denials in batches without reviewing patient files. One patient, Nick van Terheyden, was denied coverage for a $350 vitamin D test that his doctor had ordered. Former employees said the system saved Cigna billions of dollars but left patients with unexpected bills.

Company involved

Cigna

AI system involved

PXDX

10 source articles · read the reporting →

Meta Agrees to Record KES 2.3 Trillion Settlement Over Addictive Teenage Social Media Design - streamlinefeed.co.ke

The algorithmic content recommendation system determined what content to show to teenage users, influencing their engagement and screen time.

Company involved

Meta Platforms

AI system involved

Facebook and Instagram

1 source article · read the reporting →

Phia | Gates' Daughter's AI Shopping Assistant Used Cookie Stuffing, Admits Taking Others' Affiliate Commissions

Phia︱蓋茨女兒「AI購物助理」偷塞Cookie 認收他人推廣佣金 - singtao.ca

Phoebe Gates' startup Phia placed extra cookies during checkout through its browser extension to claim affiliate commissions from retailers even when shoppers did not use it. Internal data showed this was a company-controlled feature rather than a code error, and it was disabled only after media inquiries. Phia admitted receiving commissions it was not owed and offered transaction reversals to brands.

Company involved

Phia

AI system involved

Phia

1 source article · read the reporting →

Deepfake video of Jack Tame used to scam people with diabetes

A deepfake video featuring TVNZ presenter Jack Tame and diabetes researcher Professor Rinki Murphy circulated online, falsely promoting a medical breakthrough. The video convinced people with diabetes to purchase a product and stop taking their medications. One victim, Steve Davis, paid $320 and has not received a refund. Health NZ warned the public about the scam, and police received an online report on April 30.

2 source articles · read the reporting →

Lieff Cabraser and Greer Injury Lawyers Announce Federal Class Action Lawsuit Against X.AI Corp and MZX Tech LLC for Nuisance,…

The data centers and gas turbines emitted constant noise, vibrations, and particulate matter, affecting residents' ability to use and enjoy their properties.

Company involved

X.AI Corp and MZX Tech LLC

AI system involved

Colossus I, Colossus II, MACROHARDRR, Minihard data centers and gas turbines

1 source article · read the reporting →

Deepfake scam targets Alzheimer's patients with false honey cure

A social media scam used AI-generated deepfake videos of celebrities like Bill Gates and Anderson Cooper to falsely endorse a honey-based supplement as a cure for Alzheimer's disease. Victims who purchased the product reported unauthorized recurring credit card charges and difficulty obtaining refunds. The Better Business Bureau warned that there is no scientific evidence for the claims and advised consumers to be wary of such scams. The Alzheimer's Association and Gates' office denied any endorsement.

2 source articles · read the reporting →

Deepfake videos of doctors used to peddle fake supplements, AMA warns

Deepfake videos impersonating trusted Australian doctors such as Dr Norman Swan and Professor Jonathan Shaw have been used to advertise unproven dietary supplements online. The Australian Medical Association warns that vulnerable people with serious health issues are being targeted by dodgy companies exploiting the technology. The AMA has written to the Minister for Communications urging mandatory identification of companies behind such videos and a takedown portal, and has criticised Meta for weakened content moderation.

2 source articles · read the reporting →

AI deepfakes of doctors used to scam diabetes patients in Australia

Scammers used AI-generated deepfake videos and fake interviews impersonating diabetes expert Professor Jonathan Shaw and other health professionals to promote a dietary supplement called Glyco Balance. The fraudulent ads falsely claimed the supplement, sold by Vellec Group, was developed at the Baker Institute and that commonly prescribed diabetes medication is dangerous. One victim, Michael, purchased the supplement for $340 before realising it was a scam. The Therapeutic Goods Administration is assessing the situation, and Meta removed the fake video after the Baker Institute reported it.

Company involved

Vellec Group

AI system involved

Generative AI for deepfake video creation

1 source article · read the reporting →

NSW Reconstruction Authority data breach exposes 2031 people's info via ChatGPT

A former temporary staff member of the NSW Reconstruction Authority uploaded personal information of 2031 Northern Rivers Resilient Homes Program applicants to ChatGPT without authorisation. The data included names, contact details, dates of birth, and sensitive health information. The breach occurred between 12 and 15 March 2025. The authority has notified affected individuals, reported the breach to the NSW Privacy Commissioner, and implemented measures to prevent future unauthorised use of AI tools.

Company involved

NSW Reconstruction Authority

AI system involved

ChatGPT

2 source articles · read the reporting →

Scammers use AI deepfakes on TikTok to sell wellness products

Media Matters found numerous TikTok accounts using AI-generated deepfake influencers and fabricated storytimes to deceive users into purchasing wellness products. The accounts, part of an affiliated network, used fake personas and testimonials to promote items like batana oil and supplements. The scam preys on users' health concerns, including fertility issues. All identified accounts have been removed from TikTok since publication.

1 source article · read the reporting →

Prozenith used AI deepfake of Oprah to sell turmeric supplements

A Utah woman says she paid more than $400 for Prozenith supplements after seeing videos that appeared to show Oprah Winfrey endorsing them, but the product was mostly turmeric. The endorsements are alleged to be AI deepfakes, and Winfrey has said she has nothing to do with weight-loss pills. KSL TV contacted Prozenith but received no response; the woman was told she could return the product.

Company involved

Prozenith

2 source articles · read the reporting →

Guy's and St Thomas' NHS Trust warns of AI-generated fake doctor weight loss videos

Guy's and St Thomas' NHS Foundation Trust issued an alert after AI-generated videos on social media falsely showed its doctors endorsing weight loss patches. The videos, found on Facebook and TikTok, are fraudulent and the trust is working to have them taken down. A financial crime expert said the scam aims to sell a herbal product by bypassing rational responses. The BBC contacted the company behind the products but received no response.

1 source article · read the reporting →

Mumbai Cyber Police Bust Deepfake Share Trading Scam by Valueleaf

Mumbai Cyber Police arrested four individuals from Bengaluru-based advertising agency Valueleaf for allegedly circulating deepfake videos of stock market experts to deceive investors. The videos, created for Hong Kong-based First Bridge, were promoted despite knowledge of their falsity and potential financial harm. Meta flagged the content, but the accused evaded detection by increasing ad accounts and changing domain locations. The case, registered under the Bhartiya Nyaya Sanhita and Information Technology Act, is under investigation to determine the number of defrauded investors.

Company involved

Valueleaf

1 source article · read the reporting →

Hassan v ABC International Bank (Employment Tribunals): AI-hallucinated content in court filing, Costs Order

The AI generated fabricated legal citations that the claimant submitted to the tribunal, resulting in a costs order against the claimant.

1 source article · read the reporting →

Otago professor's likeness used in AI deepfake diabetes scam

A deepfake video and false online story manipulated the likeness of University of Otago professor Sir Jim Mann to promote a diabetes product. The AI-generated advertisements urged people with type 2 diabetes to stop taking their medication Metformin and instead buy alternative natural products. Professor Mann was alerted to the scam and expressed deep concern that patients who trust him may become sick after being misled. The university and law enforcement are investigating.

Company involved

Smart Hemp Gummies

AI system involved

Glyco Balance

2 source articles · read the reporting →

Myeesha Parker v. Costco Wholesale Corp. (W.D. Washington): AI-hallucinated content in court filing, Fine, Adverse Costs Order, Bar Referral

The lawyer submitted AI-generated court documents containing false citations and quotes, harming his client's discrimination case.

AI system involved

Callidus AI

1 source article · read the reporting →

Hong Kong tycoon sues Tyndaris over AI hedge fund losses

Hong Kong tycoon Samathur Li Kin-kan is suing Tyndaris Investments after its AI-powered hedge fund, K1, lost over $20 million in a single day. Li alleges that the system was not as sophisticated as claimed. The trial is set to begin in London in April 2020.

Company involved

Tyndaris Investments

AI system involved

K1

10 source articles · read the reporting →

Benjamin v. Costco Wholesale Corp (E.D. New York): AI-hallucinated content in court filing, Monetary sanction; public reprimand; order to serve…

The AI generated fake case citations that were included in a court filing, misleading the court and opposing counsel.

Company involved

Hovsepyan Law Group, PC

AI system involved

ChatOn

1 source article · read the reporting →

Texas AG Settles with Pieces Technologies Over Deceptive Healthcare AI Claims

The Texas Attorney General investigated Pieces Technologies, a Dallas-based healthcare AI company, for making false and misleading statements about the accuracy of its generative AI product used in hospitals. The company claimed an error rate of less than 1 per 100,000, but the investigation found these metrics were likely inaccurate. As part of the settlement, Pieces agreed to accurately disclose its product's accuracy and ensure hospital staff understand the appropriate reliance on its AI. The case marks the first-of-its-kind healthcare generative AI investigation by the AG.

Company involved

Pieces Technologies

4 source articles · read the reporting →

Hangzhou plans permanent health codes with scoring, sparking privacy backlash

Hangzhou health authorities announced plans to launch a permanent health-tracking QR code system that would assign residents a color and numerical score based on medical records and lifestyle choices. The proposal, an expansion of the existing COVID-19 health code system, has sparked online backlash over privacy concerns and fears of discrimination. Critics argue that the system could lead to public exposure of personal health data and potential misuse by employers. The authorities aim to complete the project by May or June 2020.

Company involved

Hangzhou Health Authorities

AI system involved

Health Code

10 source articles · read the reporting →

Kohler, BMW, MaxMara secretly collected customers' facial recognition data

During the 2021 CCTV 3·15 Gala, it was revealed that Kohler, BMW, and MaxMara stores had installed facial recognition cameras from vendors such as Wandianzhang and Youluoke. These cameras captured customers' facial data without their knowledge or consent, and the data was used to track customer visits and inform sales strategies. The systems were deployed in thousands of stores across China, collecting over 100 million facial records. The companies did not inform customers or obtain consent, violating Chinese privacy laws.

Company involved

Kohler (China) Investment Co., Ltd., ZhengTong Auto (BMW dealerships), MaxMara

AI system involved

Facial recognition cameras from Wandianzhang, Youluoke, Yaliang, Ruiwei

10 source articles · read the reporting →