The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

68 incidents closest to “Pagaya Technologies” · matched on meaning · public reporting

Gamma AI Presentation Tool Exploited in Multi-Stage Phishing Campaign

Threat actors used Gamma, an AI-powered presentation builder, to host a page that redirected recipients to a fake Microsoft SharePoint login portal. Emails sent from compromised legitimate accounts passed authentication checks, while a Cloudflare Turnstile blocked automated security scanners. An adversary-in-the-middle framework validated credentials in real time and captured session cookies, enabling multi-factor authentication bypass on Microsoft accounts. Abnormal reported the campaign on 15 April 2025.

AI system involved
Gamma

7 source articles · read the reporting →

WF-HW23LM1 Dec 2023

Alibaba among firms fooled by AI-hallucinated software package

Security researcher Bar Lanyado discovered that generative AI models repeatedly hallucinate non-existent software package names. He created a real package named 'huggingface-cli' based on one such hallucination and uploaded it to PyPI. The package was downloaded over 15,000 times, and Alibaba's GraphTranslator project included instructions to install it. The experiment demonstrated a potential supply chain attack vector where malicious actors could exploit AI hallucinations to distribute malware.

Company involved
Alibaba
AI system involved
GraphTranslator

4 source articles · read the reporting →

WF-J8EHEE23 Sep 2021

Xpeng P7 Collides with Truck During NGP Assisted Driving, Driver Injured

On 23 September 2021, a Xpeng P7 owner in China was using the NGP automatic navigation assisted driving system when the vehicle failed to identify a flatbed truck ahead and collided with its rear. The driver, who trusted the system to brake automatically, sustained slight injuries. Xpeng Motors stated that the system's functions were operating normally before the crash and that further analysis would be conducted. Experts suggested the visual camera may have misjudged the distance due to the truck's unusual shape.

Company involved
Xpeng Motors
AI system involved
NGP automatic navigation assisted driving system

1 source article · read the reporting →

WF-R72X6B10 Jul 2025

APT28 uses LLM-powered malware LAMEHUG against Ukraine's security and defence sector

CERT-UA reports that the threat group UAC-0001 (APT28) distributed phishing emails to Ukrainian executive bodies, impersonating a ministry representative. The emails contained a malicious attachment that deployed LAMEHUG, a Python-based tool which uses the Qwen 2.5-Coder-32B-Instruct large language model via Hugging Face to generate commands for data collection and exfiltration. The malware gathered system information and searched for Microsoft Office, TXT and PDF documents in common user directories, exfiltrating them via SFTP or HTTP POST requests.

Company involved
UAC-0001 (APT28)
AI system involved
LAMEHUG

2 source articles · read the reporting →

WF-5BYPQK1 Oct 2022

Researchers identify Twitter botnet likely using ChatGPT for crypto promotion

Researchers at Indiana University identified a botnet of 1,140 Twitter accounts, dubbed 'fox8', that appears to use ChatGPT to generate human-like content promoting crypto, blockchain, and NFT websites. The accounts were discovered because they accidentally posted the phrase 'as an ai language model' in tweets. The researchers warn that this botnet is likely the tip of the iceberg and that LLM-powered bots could be used for disinformation and political manipulation.

AI system involved
ChatGPT

5 source articles · read the reporting →

WF-02DLSM24 Jun 2019

Sidewalk Labs releases Toronto waterfront plan amid privacy concerns

Sidewalk Labs, a subsidiary of Alphabet, released a 1,500-page plan for a new development on Toronto's eastern waterfront, including the Quayside site. The plan involves extensive data collection and management, raising privacy concerns from critics and the public. Waterfront Toronto, the overseeing body, is reviewing the plan and has expressed concerns about its scope and data governance. The company has proposed an independent trust to oversee data, but critics remain unsatisfied.

Company involved
Sidewalk Labs
AI system involved
Quayside development

10 source articles · read the reporting →

WF-RQ6Z5D10 Jul 2024

Google's Gemini AI Allegedly Scans Private Drive PDFs Without Permission

Kevin Bankston, a privacy activist, alleges that Google's Gemini AI automatically summarised his private tax return PDF in Google Drive without his explicit consent. Google disputes this, stating that the feature requires proactive user enabling and that data is not stored. Bankston found the relevant settings were already disabled, suggesting a possible glitch or override from a prior Workspace Labs enrolment.

Company involved
Google
AI system involved
Gemini AI

1 source article · read the reporting →

WF-U5X4EM1 May 2026

Google sues Chinese gang over AI-powered fraud targeting Americans

Google has filed a lawsuit against a Chinese cybercrime group called Outsider Enterprise, alleging it used Google's Gemini AI to create hundreds of fake websites impersonating companies and government services. The group allegedly sent millions of phishing messages to Android users, defrauding hundreds of thousands of Americans of millions of dollars. Google is coordinating with the FBI and wireless carriers to dismantle the network. The lawsuit, filed in the Southern District of New York, seeks an injunction to take down the operation.

Company involved
Outsider Enterprise
AI system involved
Gemini

3 source articles · read the reporting →

WF-3PTUVG1 Mar 2022

Speedcam Anywhere developers face abuse from UK drivers after app launch

Speedcam Anywhere, an app that uses AI to estimate the speed of passing vehicles, was launched in March 2022. Its developers have received abusive emails from drivers and are now hiding their identities. Google and Apple have not approved the app for distribution, and the Home Office has not vetted it for speeding prosecutions.

AI system involved
Speedcam Anywhere

10 source articles · read the reporting →

WF-VVTY7V19 Feb 2024

North Korean hackers use ChatGPT to scam LinkedIn users

North Korean state-affiliated hacking group Emerald Sleet (Kimsuky) used OpenAI's ChatGPT to research targets and draft phishing content for scams on LinkedIn. Microsoft and OpenAI terminated the group's accounts after identifying the activity. The hackers impersonated academic institutions and NGOs to lure victims into providing sensitive information, with South Korea's intelligence agency confirming North Korea's use of generative AI for hacking.

Company involved
OpenAI
AI system involved
ChatGPT

6 source articles · read the reporting →

Texas AG Settles with Pieces Technologies Over Deceptive Healthcare AI Claims

The Texas Attorney General investigated Pieces Technologies, a Dallas-based healthcare AI company, for making false and misleading statements about the accuracy of its generative AI product used in hospitals. The company claimed an error rate of less than 1 per 100,000, but the investigation found these metrics were likely inaccurate. As part of the settlement, Pieces agreed to accurately disclose its product's accuracy and ensure hospital staff understand the appropriate reliance on its AI. The case marks the first-of-its-kind healthcare generative AI investigation by the AG.

Company involved
Pieces Technologies

4 source articles · read the reporting →

WF-LH77B55 Sep 2024

Italy terminates contracts with Paragon spyware after surveillance scandal

Italy has terminated its contracts with Israeli spyware company Paragon after revelations that the spyware was used against government critics, including journalists and migrant rescue workers. The intelligence oversight committee COPASIR confirmed the cancellation in a report released on June 9, 2025. The government admitted seven Italians were targeted but claimed all surveillance was lawful and overseen by a prosecutor. Opposition parties are demanding a full investigation.

Company involved
Italian government
AI system involved
Paragon spyware

9 source articles · read the reporting →

WF-YBB25K1 Jan 2025

Amazon AI Crawler Overwhelms Open Source Developer's Git Service

Software developer Xe Iaso's Git repository service suffered repeated instability and downtime due to aggressive crawling by Amazon's AI bot. Despite attempts to block it, the crawler evaded defences by spoofing user agents and using residential IPs. Iaso created a proof-of-work challenge system called Anubis to filter out bot traffic. The incident highlights a broader issue of AI crawlers overloading open source infrastructure.

Company involved
Amazon

2 source articles · read the reporting →

Banjo's Live Time system overstated capabilities and posed privacy risks for Utah public safety data

The Utah State Auditor reviewed the contract for Banjo's Live Time public safety application. The system was found to lack advertised AI technology and posed security risks due to direct database queries to PSAP databases. Banjo also overstated its capabilities in its response to the RFP. The auditor concluded that personally identifiable information was unlikely accessed but recommended more rigorous vetting and better security practices.

Company involved
Utah Attorney General's Office
AI system involved
Live Time

5 source articles · read the reporting →

Chinese tech companies patented Uyghur detection analytics

IPVM reported that Huawei, Megvii, SenseTime, and other Chinese tech companies filed patents in China for AI systems that can detect Uyghur ethnicity. The patents included Uyghur as a detectable attribute for facial recognition and image retrieval. The companies responded by saying they would amend or withdraw the patents, claiming the references were misunderstood or regrettable, while the technology is used by police for surveillance and targeting of Uyghurs.

10 source articles · read the reporting →

WF-6D5AJ41 Jan 2023

ChatGPT falsely tells users OpenCage offers phone lookup service

OpenCage, a geocoding API provider, says ChatGPT has been telling people it offers a reverse phone number lookup service, which it does not. Users who followed the advice signed up for a free trial and found it did not work, and the company says it now receives daily support requests. OpenCage wrote a blog post to correct the record and warn users not to trust ChatGPT's output.

AI system involved
ChatGPT

7 source articles · read the reporting →

Uttar Pradesh Police uses AI cameras to track women's distress

Uttar Pradesh Police, in collaboration with Staqu Technologies, deployed AI-powered cameras in Lucknow for surveillance of women in distress. Activists have criticised the system for invading privacy. The system is currently in use.

Company involved
Uttar Pradesh Police
AI system involved
Trinetra

10 source articles · read the reporting →

WF-FST9Z61 Apr 2018

ViaQuatro's facial recognition system in São Paulo metro challenged in court

In April 2018, ViaQuatro installed the Digital Interactive Doors System, developed by AdMobilize, on the São Paulo metro's yellow line. The system used cameras to detect passengers' faces and claimed to infer their emotion, age, and gender in order to target advertisements. The Brazilian Institute of Consumer Protection (IDEC) filed a public civil action alleging that the system violated consumer and data protection laws by processing biometric data without consent and making pseudoscientific and discriminatory inferences. A judge ordered the cameras removed in August 2018, and the case is pending a final ruling.

Company involved
ViaQuatro
AI system involved
Digital Interactive Doors System (DID system)

10 source articles · read the reporting →

WF-VU5D658 Apr 2023

California regulator accuses Maxpread Technologies of using AI-generated fake CEO to scam investors

Maxpread Technologies allegedly used an AI-generated avatar to pose as its CEO in a YouTube video to deceive investors. The California DFPI issued cease and desist letters to Maxpread and four other companies for offering unqualified securities and making false promises. The regulator claims the companies promised high daily returns using AI trading, but these claims were false. Maxpread did not respond to requests for comment.

Company involved
Maxpread Technologies

8 source articles · read the reporting →

Tourists rescued after following ChatGPT route in Tatra mountains

Three Polish tourists used ChatGPT to plan a hiking route from Hala Gąsienicowa to Dolina Pięciu Stawów in the Tatra mountains. The AI recommended a difficult route through Przełęcz Krzyżne, which proved too dangerous in winter conditions with limited visibility and icy rocks. The tourists called the TOPR rescue service and were safely brought down. A TOPR rescuer advised against relying on ChatGPT or Google Maps for mountain route planning.

Company involved
OpenAI
AI system involved
ChatGPT

2 source articles · read the reporting →

WF-2ABL3N30 Nov 2023

Bavarian police test Palantir data mining with real personal data

The Bavarian State Criminal Police Office (LKA) has been testing Palantir's data mining software, called VeRa, with real personal data for months. The Bavarian data protection commissioner only learned of the test through a media inquiry and has announced a review. The Interior Ministry claims the test is lawful under current law, but critics argue a legal basis is missing.

Company involved
Bayerisches Landeskriminalamt
AI system involved
VeRa

7 source articles · read the reporting →

N-Tech.lab's FindFace used to identify St Petersburg metro passengers without consent

Egor Tsvetkov photographed passengers on the St Petersburg metro without their permission and used N-Tech.lab's facial recognition service FindFace to match their faces to public Vkontakte profiles. He published the results in an art project called 'Your Face is Big Data', saying he wanted to show how 'digital narcissism' can lead to stalking. Privacy advocates said the project was ethically problematic because the subjects had not consented and their identities were exposed. FindFace had been launched by N-Tech.lab in February 2016.

Company involved
N-Tech.lab
AI system involved
FindFace

8 source articles · read the reporting →

WildBrain disputes Kartoon Studios' Gadget A.I. trademark use

Kartoon Studios announced its Gadget A.I. toolkit, which aggregates AI tools for animation production. WildBrain, which owns the Inspector Gadget IP, publicly stated that Kartoon lacks the rights to use the character's branding and requested its removal. Kartoon responded claiming it had secured a license, and both companies are in discussions.

Company involved
Kartoon Studios
AI system involved
Gadget A.I.

2 source articles · read the reporting →

WF-03V5V41 Jan 2021

PimEyes faces fine proceedings over biometric facial recognition in Baden-Württemberg

PimEyes, a facial recognition search engine, is accused of scraping images from the internet and processing biometric data without a valid legal basis under the GDPR. The data protection authority of Baden-Württemberg (LfDI) opened fine proceedings after it found PimEyes's response to its questions inadequate. PimEyes argues that the images it processes are publicly available and not personal data. The LfDI says the processing endangers citizens' rights and freedoms and is not covered by the GDPR exceptions.

Company involved
PimEyes
AI system involved
PimEyes

9 source articles · read the reporting →

← Newerpage 2 of 3Older →