Hong Kong police arrest eight over deepfake bank account scam
Hong Kong police arrested eight people accused of running a scam ring that used deepfake images to bypass bank verification checks and open accounts. The deepfakes were created by merging scammers' faces with those on lost identity cards using artificial intelligence. The arrests were announced on 19 April 2025.
2 source articles · read the reporting →
Leonardo AI misused to create nonconsensual explicit images of celebrities
Leonardo AI, an AI image-generating platform backed by Samsung, has been misused to create nonconsensual explicit images of celebrities. The platform allows users to generate images using user-generated Stable Diffusion models. Users bypassed ethical guardrails by misspelling celebrity names and using sexually suggestive terms. The misuse was reported by 404 Media.
- Company involved
- Leonardo AI
- AI system involved
- Leonardo AI
5 source articles · read the reporting →
Deepfake Biden robocall likely made with ElevenLabs tools, researchers say
In January 2024, some voters in New Hampshire received an AI-generated robocall impersonating President Joe Biden, telling them not to vote in the state's primary election. Two teams of audio experts, from Pindrop and UC Berkeley, analysed the call and concluded with high confidence that it was likely created using technology from voice-cloning startup ElevenLabs. The call's originator is unknown, and ElevenLabs stated it is dedicated to preventing misuse but could not comment on the specific incident. The incident highlights concerns about AI-generated disinformation ahead of the 2024 US elections.
- Company involved
- ElevenLabs
- AI system involved
- ElevenLabs voice-cloning tools
9 source articles · read the reporting →
NYPD uses facial recognition to identify crime suspects since 2011
The New York City Police Department has used facial recognition technology since 2011 to compare images from criminal investigations with lawfully possessed arrest photos. The system generates possible matches that are then reviewed by human investigators before being used as leads. The NYPD states that no one has been arrested solely based on a facial recognition search and that it knows of no false arrests from such matches, though studies have found accuracy variations for certain demographic groups.
- Company involved
- New York City Police Department
- AI system involved
- Facial recognition
8 source articles · read the reporting →
NYC Mayor Eric Adams uses AI deepfake robocalls without disclosure
New York City Mayor Eric Adams announced that his office has been using AI-generated deepfake robocalls in multiple languages since March 2022. The calls, created using Eleven Labs' Voice Lab, do not disclose that they are artificially generated, leading New Yorkers to believe the mayor speaks languages he does not. Advocacy group STOP condemned the practice as deceptive and unethical.
- Company involved
- New York City
- AI system involved
- Voice Lab by Eleven Labs
10 source articles · read the reporting →
PimEyes faces fine proceedings over biometric facial recognition in Baden-Württemberg
PimEyes, a facial recognition search engine, is accused of scraping images from the internet and processing biometric data without a valid legal basis under the GDPR. The data protection authority of Baden-Württemberg (LfDI) opened fine proceedings after it found PimEyes's response to its questions inadequate. PimEyes argues that the images it processes are publicly available and not personal data. The LfDI says the processing endangers citizens' rights and freedoms and is not covered by the GDPR exceptions.
- Company involved
- PimEyes
- AI system involved
- PimEyes
9 source articles · read the reporting →
OpenDream AI art site allowed users to generate child sexual abuse material
OpenDream, an AI image generation platform, allowed users to generate and publicly display child sexual abuse material (CSAM) and non-consensual deepfakes from at least December 2023 until July 2024. The platform, operated by CBM Media Pte Ltd in Singapore, offered paid plans with NSFW prompts and models. Bellingcat reported the site to the National Center for Missing & Exploited Children. After Bellingcat's inquiry, the CSAM was removed from the site and search engines, and Google terminated OpenDream's AdSense account.
- Company involved
- CBM Media Pte Ltd
- AI system involved
- OpenDream
3 source articles · read the reporting →
Council Rock student investigated for creating AI deepfakes of classmates
A middle school student in the Council Rock School District allegedly used AI to create pornographic deepfake images of up to 12 underage girls. The incident occurred in March and was reported to Newtown Township police, who are investigating. The student could be prosecuted under a new Pennsylvania law criminalising the distribution of such deepfakes. The school district has declined to comment further.
1 source article · read the reporting →
French police secretly used Briefcam facial recognition since 2015
The French National Police has been using Briefcam's Video Synopsis software since 2015, according to internal documents obtained by Disclose. The software, which includes facial recognition capabilities, was deployed without the required data protection impact assessment or notification to the CNIL. The Ministry of the Interior concealed the use of this tool for eight years. The police hierarchy allegedly used the software for facial recognition without judicial requisition, and the DGPN did not respond to requests for comment.
- Company involved
- French National Police
- AI system involved
- Briefcam Video Synopsis
10 source articles · read the reporting →
Google Lens AI overviews share misleading information about images
Google Lens's AI overviews provided false and misleading information about images, including miscaptioned videos and AI-generated footage. Full Fact found that the overviews repeated debunked claims and failed to identify inauthentic content. Google acknowledged the errors and said they were caused by problems with visual search results.
- Company involved
- Google
- AI system involved
- Google Lens AI overviews
2 source articles · read the reporting →
AI-generated fact check falsely accuses MSNBC of using outdated protest footage
An AI-generated fact check on X falsely claimed that MSNBC aired outdated video footage to inflate the size of anti-Trump protests. Conservatives, including Senator Ted Cruz, relied on the AI hallucination to accuse MSNBC. NewsGuard debunked the claim, stating the footage was legitimate.
3 source articles · read the reporting →
Delhi Police use facial recognition to screen PM Modi rally attendees
Delhi Police used an Automated Facial Recognition System (AFRS) to screen crowds at Prime Minister Narendra Modi's rally on December 22, 2019. The system, originally installed to find missing children, was used to identify possible disruptions. Privacy advocates called the move illegal and unconstitutional, saying it amounts to mass surveillance. Police defended the use, citing credible intelligence about possible disruptions.
- Company involved
- Delhi Police
- AI system involved
- Automated Facial Recognition System (AFRS)
6 source articles · read the reporting →
Developer iperov releases DeepFaceLive real-time face-swap AI on GitHub
The developer iperov has published DeepFaceLive, a neural network for real-time face swapping, on GitHub. The tool automatically replaces a user's face in live streams and video calls with a nonexistent model or a celebrity, and the installation instructions are simple. The developer claims 95% of deepfakes on YouTube were made with the related DeepFaceLab. No specific harm is reported, but the article highlights the tool's potential for misuse.
- Company involved
- iperov
- AI system involved
- DeepFaceLive
7 source articles · read the reporting →
Outabox hack exposes biometric data of patrons at bars, clubs and casinos
Hackers claiming to be former employees published a website allowing searches of Outabox's facial recognition database, exposing biometric and other sensitive data of patrons used for age verification at bars, clubs and casinos. The Surveillance Technology Oversight Project warns that the breach demonstrates the danger of facial recognition for age verification. S.T.O.P. has launched a campaign to ban facial recognition in public accommodations.
- Company involved
- Outabox
8 source articles · read the reporting →
Vumacam's AI CCTV system flagged 28 black people as suspicious in Johannesburg suburbs
In Johannesburg suburbs, Vumacam's AI-powered CCTV network using iSentry software flagged 28 black individuals as 'suspicious' in a shift report, according to a 2019 article. The system, deployed by private security firms, uses video analytics to detect abnormal behavior and alerts security guards. The article alleges that the system disproportionately targets people of color, reflecting racial bias in a racially divided country.
- Company involved
- Vumacam
- AI system involved
- iSentry
6 source articles · read the reporting →
Dutch DPA fines Clearview AI €30.5 million for GDPR violations
The Dutch Data Protection Authority fined Clearview AI €30.5 million for creating an illegal database of biometric codes scraped from social media without consent. The regulator accused the company of failing to inform individuals about how their data is used and continuing violations after investigation. Clearview AI denies the allegations, stating it has no presence in the Netherlands or EU and that the fine is unenforceable.
- Company involved
- Clearview AI
- AI system involved
- Clearview AI facial recognition database
7 source articles · read the reporting →
PimEyes facial recognition search engine identifies individuals from public photos
PimEyes, a Polish facial recognition search engine, allows users to upload a photo and find matching images from across the internet. The system scrapes billions of photos and creates biometric profiles without consent. A banker named Dylan (pseudonym) discovered that an old party photo of him was found by the system, potentially outing his private life. Privacy advocates and platforms like Instagram and YouTube are taking action against PimEyes, alleging violations of the GDPR.
- Company involved
- PimEyes
- AI system involved
- PimEyes
6 source articles · read the reporting →
Buenos Aires city government uses live facial recognition to track minors in criminal database
The Buenos Aires city government deployed a live facial recognition system in April 2019 that scans subway passengers and matches them against CONARC, a national database of alleged offenders. Human Rights Watch found that the database includes at least 166 children, some as young as one to three years old, and that the system has led to numerous false arrests. The system was implemented without public consultation, and there is no mechanism to correct mistakes. A civil rights organization filed a lawsuit, and the government is pushing a bill to legalize the technology.
- Company involved
- Buenos Aires city government
1 source article · read the reporting →
Ukrainian Hacker Pleads Guilty to Operating OnlyFake AI ID Scam Site
Yurii Nazarenko, a Ukrainian national, pleaded guilty to running OnlyFake, an AI-powered website that generated and sold over 10,000 counterfeit identification documents globally. The site allowed users to create realistic fake IDs, including driver's licenses and passports, to bypass identity verification at banks and cryptocurrency exchanges. Nazarenko faces up to 15 years in prison and agreed to forfeit $1.2 million. Sentencing is scheduled for June 2026.
- Company involved
- OnlyFake
- AI system involved
- OnlyFake
3 source articles · read the reporting →
Bahia Facial Recognition System Misidentifies Black Man as Wanted Criminal
Davi, a black administrative assistant, was tracked across 15 metro stations and approached by police after Bahia's facial recognition system falsely matched him to a wanted person. He was released after officers verified his identity. The system, supplied by Iecisa and Huawei, has a low accuracy rate and disproportionately affects black people. The government plans to expand it despite concerns over racial bias and errors.
- Company involved
- Secretaria de Segurança Pública da Bahia
2 source articles · read the reporting →
Google contractors targeted homeless people for Pixel 4 facial recognition data
Google hired Randstad to collect facial data to train the Pixel 4's face recognition system. Contractors allegedly targeted homeless people and unaware students, using deceptive tactics such as calling it a "selfie game" and offering $5 without informing subjects they were being recorded. Google suspended the research program and opened an investigation following the report.
- Company involved
- Google
- AI system involved
- Pixel 4 facial recognition
1 source article · read the reporting →
White House investigates AI voice-cloning hack impersonating Chief of Staff
The White House is investigating a cyber breach in which contacts of Chief of Staff Susie Wiles were accessed and used to impersonate her with AI voice cloning. Multiple high-level figures reportedly received calls and texts appearing to be from her. The White House spokesperson said the matter is under investigation, but no specific harm from the calls has been confirmed.
5 source articles · read the reporting →
Clearview AI facial recognition app scrapes billions of images and is used by police
Clearview AI, a secretive start-up, built a facial recognition app that matches photos to a database of more than three billion images scraped from social media and websites. More than 600 law enforcement agencies, including the FBI and Department of Homeland Security, have used the tool to identify suspects in crimes such as shoplifting, identity theft and murder. The company monitored officers who ran a reporter's photo through the app, and its founder acknowledged designing an augmented-reality prototype but said there were no plans to release it. Critics warned the tool could end anonymity and enable misuse.
- Company involved
- Clearview AI
- AI system involved
- Clearview AI facial recognition app
2 source articles · read the reporting →
X's Grok AI falsely 'unmasks' ICE agent, leading to harassment of two men
After the fatal shooting of Renee Good by an ICE agent in Minneapolis, users on X asked the AI chatbot Grok to 'unmask' the agent. Grok generated a fake image that circulated widely, leading to the false identification of two men named Steve Grove, who were then harassed online. Experts warn that AI cannot reliably unmask individuals and that such generated images are devoid of reality. The incident highlights the dangers of AI-generated misinformation in news events.
- Company involved
- X
- AI system involved
- Grok
1 source article · read the reporting →