Where automated decisions went wrong

In-house counsel group accuses rival of training AI on its materials - Reuters

In-house counsel group accuses rival of training AI on its materials Reuters

1 source article · read the reporting →

AI detector scores banned as evidence at Yale and Johns Hopkins, vendor conflict exposed - Pasquale Pillitteri

The system flagged student assignments as AI-generated, potentially leading to academic misconduct accusations.

Company involved

Yale University and Johns Hopkins University

AI system involved

Turnitin AI detector

1 source article · read the reporting →

Deepfake Video Leads to ₹3.19 Crore Investment Fraud as Scammers Gain Computer Access - The420.in

The AI-generated deepfake video falsely endorsed an investment opportunity, leading a victim to invest and lose £250,000.

1 source article · read the reporting →

What a $150,000 scam reveals about AI and elder fraud - Rolling Out

AI-generated deepfakes and voice cloning convinced a 63-year-old woman to send $150,000 to a scammer posing as a romantic partner

1 source article · read the reporting →

AI Scammers Clone Exante Broker, Use JPMorgan Account to Defraud US Victim

Scammers used generative AI to create a fake clone of the brokerage firm Exante, including a replicated trading platform and AI-generated passports. They opened a real JPMorgan Chase bank account using a US address and tricked at least one US victim into transferring funds. Exante, which does not serve US clients, discovered the scam when the victim was registered on its real platform and reported the incident to the FBI, SEC, CFTC, and other authorities. The scammers remain unidentified, and the victim's funds have not been recovered.

2 source articles · read the reporting →

Retd official duped of ₹2.36cr in deepfake-backed ‘digital arrest’ racket; mastermind held - The Times of India

Deepfake system impersonated judicial and government officials to coerce a retired official into transferring money

1 source article · read the reporting →

East TN woman mistakenly jailed for months after A.I. flagged her as bank fraud suspect, lawsuit says - WVLT

The facial recognition system misidentified Angela Lipps as a North Dakota bank fraud suspect, leading to her arrest and nearly six months in jail.

Company involved

City of Fargo

1 source article · read the reporting →

Istanbul gang used AI to defraud European citizens from call center

A gang in Maslak, Istanbul, set up a call centre using AI-supported software to impersonate police, prosecutors, soldiers, and bank officials. They defrauded many Czech and European citizens, obtaining high amounts of money, which they laundered through bank accounts and crypto platforms. Turkish authorities, in cooperation with Czech judicial authorities, conducted simultaneous raids on three addresses, seizing digital materials and detaining 80 foreign suspects.

1 source article · read the reporting →

AI-Generated Fake Investment Websites Target Australian Fraud Analyst

A fraud analyst, Leon, was targeted by scammers who used AI to create convincing fake websites for non-existent investment firms. The scammers posed as employees of Assent Advisory and directed him to elaborate sites for APPC Capital Singapore and Thackeray Mines and Minerals Inc., complete with AI-generated images and fake details. Leon recognised the scam, but the sophistication of the AI-generated content made him second-guess his own judgment. The incident highlights how AI is enabling more convincing investment scams.

1 source article · read the reporting →

Deepfake Albanese Pitching $40,000 Returns Led $7.4M in Australian Scam Losses - Tech Times

AI-generated deepfake videos misrepresented public figures to deceive victims into investing in fake investment platforms.

1 source article · read the reporting →

TRT-RS's Galileu AI Detects Prompt Injection Attempt in Legal Petition

The Galileu AI system, developed by the Tribunal Regional do Trabalho da 4ª Região (TRT-RS) and nationalised by the Conselho Superior da Justiça do Trabalho (CSJT), detected a prompt injection attempt in a petition filed at the 3rd Labour Court of Parauapebas, Pará. The system alerted the magistrate, who reviewed the content and made a decision based on human verification, in line with judicial AI supervision requirements. The court reported that the system prevented the malicious content from being processed and highlighted the importance of institutional AI tools with security measures.

Company involved

Tribunal Regional do Trabalho da 4ª Região

AI system involved

Galileu

1 source article · read the reporting →

Florida Title Company Stops AI Deepfake Property Fraud Attempt

A title company owner in Hallandale Beach, Florida, detected an attempted property fraud using a deepfake video. The scammer posed as the owner of a vacant lot, providing a fake driver's licence and a looped video during a video call. The title company owner noticed the video was not interactive and halted the sale. The true owner's property tax bills had been sent to the Bahamas, and the scammer used a photo of a missing woman from California.

Company involved

Florida Title and Trust

4 source articles · read the reporting →

Meta criticised as AI fake profiles drive Swedish investment fraud

AI-generated fake profiles on Meta are being used to lure Swedish small savers into 'pump and dump' investment schemes. The Swedish Financial Supervisory Authority reports that more than 5,000 savers have lost a total of half a billion kronor. Journalist Gabriel Mellqvist, whose identity was used without consent, criticises Meta for not taking responsibility.

Company involved

Meta

4 source articles · read the reporting →

Scammers use AI deepfakes to impersonate BMO strategist and economist in investment fraud

Scammers used AI-generated deepfake videos to impersonate Bank of Montreal's chief investment strategist Brian Belski and economist David Rosenberg, luring investors into WhatsApp groups with fake stock tips. Victims reported collective losses exceeding $1-million. The real experts struggled to get the content removed from Meta platforms, with some deepfakes reaching nearly a million users. Meta has since enrolled one victim in a facial recognition program to detect future impersonations.

2 source articles · read the reporting →

Scammers use AI-generated identities to steal $5.6 million in FTX debt claims fraud

In June 2024, a scam group posing as FTX debt claimants allegedly used AI-generated identities and manipulated facial appearances to defraud two companies of more than $5.6 million. The perpetrators accessed FTX customer data through public bankruptcy filings or a 2023 data breach at Kroll. Blockchain analysis traced the stolen funds through Binance, CoinEx, and Gate.io. The incident remains unresolved.

6 source articles · read the reporting →

Rotterdam's Welfare Fraud Algorithm Discriminated by Gender and Ethnicity

The city of Rotterdam deployed a machine learning algorithm built by Accenture to flag welfare recipients for fraud investigation. The system used personal data including gender, language, and subjective caseworker notes to generate risk scores, leading to investigations that disproportionately targeted women and migrants. An external review found the algorithm discriminatory and inaccurate, prompting the city to suspend its use in 2021. The system's opacity made it nearly impossible for those flagged to challenge the decisions.

Company involved

City of Rotterdam

6 source articles · read the reporting →

SEC Charges Delphia and Global Predictions for False AI Claims

The SEC charged Delphia (USA) Inc. and Global Predictions Inc. for making false and misleading statements about their use of artificial intelligence. Delphia claimed from 2019 to 2023 that it used AI and machine learning to predict investments, while Global Predictions falsely claimed in 2023 to be the 'first regulated AI financial advisor'. Both firms settled the charges without admitting or denying the findings, agreeing to pay a total of $400,000 in civil penalties and to cease and desist from further violations.

Company involved

Delphia (USA) Inc. and Global Predictions Inc.

1 source article · read the reporting →

Michigan's MiDAS system falsely accused 34,000 of unemployment fraud

Michigan's Unemployment Insurance Agency (UIA) deployed the Michigan Integrated Data Automated System (MiDAS) to automate fraud detection. The system algorithmically adjudicated over 40,000 fraud cases between October 2013 and September 2015, with 85 percent of those resulting in incorrect fraud determinations. The false accusations led to severe financial penalties, ruined credit, and homelessness for many victims. A class-action lawsuit was filed, and while the state has apologised and returned fines, it has fought against compensation claims, though legislators have promised to seek at least $30 million for those affected.

Company involved

Michigan Unemployment Insurance Agency

AI system involved

Michigan Integrated Data Automated System (MiDAS)

10 source articles · read the reporting →

Sound Intelligence Aggression Detectors Prove Unreliable in School Tests

ProPublica tested Sound Intelligence's aggression detection software, used in hundreds of U.S. schools and hospitals. The algorithm, which analyzes audio for signs of stress and anger, frequently misidentified innocent sounds like coughing as aggressive and failed to detect actual screams. At a New Jersey hospital, the system ignored an agitated man screaming and pounding a desk, escalating the situation. Sound Intelligence's CEO acknowledged the imperfections but defended the technology as an early warning system.

AI system involved

Sound Intelligence aggression detector

2 source articles · read the reporting →

UAE Company Loses $35 Million in Deepfake Audio Scam

Fraudsters used forged emails and deepfake audio to impersonate a company director, convincing a branch manager at a UAE company to transfer $35 million as part of a fake acquisition. The manager believed he was following instructions from the director and a US-based lawyer. The incident, which occurred in January 2020, led to a US federal court request for assistance in tracing the stolen funds. The case highlights the growing threat of deepfake technology in corporate fraud.

6 source articles · read the reporting →

Chicago Tribune sues Perplexity AI for copyright infringement

Chicago Tribune Company, LLC filed a lawsuit against Perplexity AI, Inc. in the Southern District of New York on December 4, 2025. The complaint alleges copyright infringement under 17 U.S.C. § 501. The case is assigned to Judge Loretta A. Preska.

Company involved

Perplexity AI, Inc.

5 source articles · read the reporting →

Mumbai Cyber Police Bust Deepfake Share Trading Scam by Valueleaf

Mumbai Cyber Police arrested four individuals from Bengaluru-based advertising agency Valueleaf for allegedly circulating deepfake videos of stock market experts to deceive investors. The videos, created for Hong Kong-based First Bridge, were promoted despite knowledge of their falsity and potential financial harm. Meta flagged the content, but the accused evaded detection by increasing ad accounts and changing domain locations. The case, registered under the Bhartiya Nyaya Sanhita and Information Technology Act, is under investigation to determine the number of defrauded investors.

Company involved

Valueleaf

1 source article · read the reporting →

Google sues Chinese gang over AI-powered fraud targeting Americans

Google has filed a lawsuit against a Chinese cybercrime group called Outsider Enterprise, alleging it used Google's Gemini AI to create hundreds of fake websites impersonating companies and government services. The group allegedly sent millions of phishing messages to Android users, defrauding hundreds of thousands of Americans of millions of dollars. Google is coordinating with the FBI and wireless carriers to dismantle the network. The lawsuit, filed in the Southern District of New York, seeks an injunction to take down the operation.

Company involved

Outsider Enterprise

AI system involved

Gemini

3 source articles · read the reporting →

Montenegro police warn of rising AI identity abuse in online fraud

Montenegrin police say AI is fuelling a rise in identity abuse by online fraudsters, using deepfake videos of public figures to promote bogus remedies and investment schemes. Surgeon and MP Vladimir Dobricanin found a manipulated video of himself apparently endorsing eyedrops and other products, while Bosnian fact-checkers reported similar misuse of statements to lure people into cryptocurrency scams. Victims reported 42 online scams in 2025 with financial losses of around 300,000 euros, but authorities say investigations are complex and often lengthy due to the use of servers abroad and encryption. Experts called for stronger technical capacities, international cooperation, and faster platform responses to tackle the problem.

Company involved

Limited Charm

AI system involved

Deepfake social media advertisements

1 source article · read the reporting →