Check Point Research finds Google Bard can generate phishing emails and malware
Check Point Research analysed Google's generative AI platform Bard and found it could be used to create phishing emails, malware keyloggers, and basic ransomware code with minimal manipulation. Bard's anti-abuse restrictors were significantly lower than ChatGPT's, making it easier to generate malicious content. The researchers demonstrated these capabilities in controlled tests but did not report actual harm to specific individuals or organisations.
- Company involved
- Google
- AI system involved
- Bard
4 source articles · read the reporting →
Google Lens AI overviews share misleading information about images
Google Lens's AI overviews provided false and misleading information about images, including miscaptioned videos and AI-generated footage. Full Fact found that the overviews repeated debunked claims and failed to identify inauthentic content. Google acknowledged the errors and said they were caused by problems with visual search results.
- Company involved
- Google
- AI system involved
- Google Lens AI overviews
2 source articles · read the reporting →
DeepSeek's R1 chatbot failed to block any jailbreak prompts in security tests
Security researchers from Cisco and the University of Pennsylvania tested 50 well-known jailbreak prompts against DeepSeek's R1 reasoning model. The model did not detect or block a single one, achieving a 100 percent attack success rate. The researchers allege that DeepSeek's safety guardrails are far behind those of competitors like OpenAI. DeepSeek did not respond to requests for comment.
- Company involved
- DeepSeek
- AI system involved
- DeepSeek R1
3 source articles · read the reporting →
Delhi Police use facial recognition to screen PM Modi rally attendees
Delhi Police used an Automated Facial Recognition System (AFRS) to screen crowds at Prime Minister Narendra Modi's rally on December 22, 2019. The system, originally installed to find missing children, was used to identify possible disruptions. Privacy advocates called the move illegal and unconstitutional, saying it amounts to mass surveillance. Police defended the use, citing credible intelligence about possible disruptions.
- Company involved
- Delhi Police
- AI system involved
- Automated Facial Recognition System (AFRS)
6 source articles · read the reporting →
Developer iperov releases DeepFaceLive real-time face-swap AI on GitHub
The developer iperov has published DeepFaceLive, a neural network for real-time face swapping, on GitHub. The tool automatically replaces a user's face in live streams and video calls with a nonexistent model or a celebrity, and the installation instructions are simple. The developer claims 95% of deepfakes on YouTube were made with the related DeepFaceLab. No specific harm is reported, but the article highlights the tool's potential for misuse.
- Company involved
- iperov
- AI system involved
- DeepFaceLive
7 source articles · read the reporting →
Audit of RisCanvi finds biases and reliability issues in criminal justice system
Eticas conducted an adversarial audit of RisCanvi, an AI risk assessment tool used in Catalonia's criminal justice system. The audit uncovered biases in risk classifications against specific demographics and significant reliability issues. The findings call for fairer practices in criminal justice AI.
- Company involved
- Catalonia's criminal justice system
- AI system involved
- RisCanvi
4 source articles · read the reporting →
Vumacam's AI CCTV system flagged 28 black people as suspicious in Johannesburg suburbs
In Johannesburg suburbs, Vumacam's AI-powered CCTV network using iSentry software flagged 28 black individuals as 'suspicious' in a shift report, according to a 2019 article. The system, deployed by private security firms, uses video analytics to detect abnormal behavior and alerts security guards. The article alleges that the system disproportionately targets people of color, reflecting racial bias in a racially divided country.
- Company involved
- Vumacam
- AI system involved
- iSentry
6 source articles · read the reporting →
AI news presenters James and Rose terminated after two months at Hawaii newspaper
The Garden Island newspaper in Hawaii deployed AI-generated news presenters James and Rose, created by Caledo, to produce video broadcasts. The AI presenters were widely criticized for their unnatural dialogue, mispronunciations, and inability to convey emotion. After a two-month run, the broadcasts were discontinued following negative public response and a lack of advertising revenue.
- Company involved
- Oahu Publications
- AI system involved
- James and Rose
8 source articles · read the reporting →
UK universities detect deepfake applicants in automated interviews
Some UK universities use Enroly's automated online interviews to screen international student applicants. Enroly detected about 30 cases of deepfake attempts out of 20,000 interviews during the January 2025 intake. The deepfakes used AI-generated images and audio to replace applicants' faces and voices. Enroly stated it caught the attempts using real-time detection methods.
- Company involved
- UK universities
- AI system involved
- Enroly
5 source articles · read the reporting →
ZDF broadcasts AI-generated video without labelling in heute journal report on ICE operations
On 16 February 2026, ZDF's news programme 'heute journal' broadcast a report on US border police ICE operations that included a video sequence generated by OpenAI's Sora AI, without labelling it as AI-generated. The broadcaster initially described the omission as a 'technical error' and later removed the AI material from the report, apologising for the mistake. The report also included a real video from 2022 that was used out of context.
- Company involved
- ZDF
- AI system involved
- Sora
2 source articles · read the reporting →
Hollywood studios accuse ByteDance's Seedance 2.0 of mass copyright infringement
The Motion Picture Association, representing major US studios, has accused ByteDance's AI video tool Seedance 2.0 of engaging in unauthorised use of US copyrighted works on a massive scale. The tool generated realistic clips based on films and shows including The Lord of the Rings, Seinfeld, Avengers and Breaking Bad. ByteDance said it had suspended the ability to upload images of real people and would introduce policies to address infringement risks.
- Company involved
- ByteDance
- AI system involved
- Seedance 2.0
6 source articles · read the reporting →
ChatGPT 4o image generator used to create fake receipts
ChatGPT's new image generator, part of the 4o model, can generate realistic fake restaurant receipts. Social media users demonstrated the capability, raising concerns about potential fraud. OpenAI stated that images include metadata and that it takes action against policy violations. The company defended the feature as allowing creative freedom.
- Company involved
- OpenAI
- AI system involved
- ChatGPT 4o image generator
5 source articles · read the reporting →
Remini app allegedly generated child pornography image from user's photo
Asia Marie Williams, a digital creator, used the Remini app to generate an image of her future baby. The app allegedly produced a half-naked image of a toddler-aged child with her face, which she described as child pornography. She shared the image on Facebook with private parts censored. The incident has not been resolved.
- Company involved
- Remini
- AI system involved
- Remini
8 source articles · read the reporting →
School AI surveillance like Gaggle can lead to false alarms, arrests
AI surveillance tools used in schools, such as Gaggle, GoGuardian and Bark, are reported to generate false alarms that have led to student arrests. The article examines cases where automated monitoring flagged innocent behaviour as threats, causing harm to students and families.
2 source articles · read the reporting →
Deepfake video of CBS anchor and suspect in Frisco stabbing spreads online
An AI-generated deepfake video manipulated a CBS News Texas anchor's image and created a fake video of the suspect in a Frisco high school stabbing. The video was posted on Instagram and reposted on multiple accounts before being taken down. The incident highlights the growing ease of creating convincing deepfakes and the spread of misinformation online.
1 source article · read the reporting →
Duke University MTMC Dataset Used in Authoritarian Surveillance Research
Duke University created and openly distributed the Duke MTMC dataset, containing surveillance footage of approximately 2,000 students and visitors on campus. The dataset was used by numerous organisations, including Chinese military-linked companies like SenseTime and Hikvision, for developing person re-identification and facial recognition technologies. Following an investigation by exposing.ai and the Financial Times, Duke University terminated the dataset in May 2019. The incident highlights the privacy risks of academic datasets being repurposed for mass surveillance without consent.
- Company involved
- Duke University
- AI system involved
- Duke MTMC
1 source article · read the reporting →
Google contractors targeted homeless people for Pixel 4 facial recognition data
Google hired Randstad to collect facial data to train the Pixel 4's face recognition system. Contractors allegedly targeted homeless people and unaware students, using deceptive tactics such as calling it a "selfie game" and offering $5 without informing subjects they were being recorded. Google suspended the research program and opened an investigation following the report.
- Company involved
- Google
- AI system involved
- Pixel 4 facial recognition
1 source article · read the reporting →
Adobe Firefly trained on thousands of Midjourney images, Bloomberg reports
Bloomberg has reported that Adobe's Firefly image generator was trained using thousands of images from competitor Midjourney. Adobe says these made up about 5% of the training data and were part of the Adobe Stock library. The company has marketed Firefly as ethically trained and offered enterprise customers indemnity against copyright claims. Adobe responded that all Adobe Stock images undergo moderation, but the report has raised questions about Firefly's copyright safety.
- Company involved
- Adobe
- AI system involved
- Firefly
7 source articles · read the reporting →
Meta Smart Glasses Lawsuit Claims Sex, Bathroom Footage Was Sent to Overseas AI Workers - Law Commentary
Recorded and transmitted private footage of users and bystanders to overseas AI workers
- Company involved
- Meta
- AI system involved
- Meta Smart Glasses
1 source article · read the reporting →
Clearview AI facial recognition app scrapes billions of images and is used by police
Clearview AI, a secretive start-up, built a facial recognition app that matches photos to a database of more than three billion images scraped from social media and websites. More than 600 law enforcement agencies, including the FBI and Department of Homeland Security, have used the tool to identify suspects in crimes such as shoplifting, identity theft and murder. The company monitored officers who ran a reporter's photo through the app, and its founder acknowledged designing an augmented-reality prototype but said there were no plans to release it. Critics warned the tool could end anonymity and enable misuse.
- Company involved
- Clearview AI
- AI system involved
- Clearview AI facial recognition app
2 source articles · read the reporting →
YouTube applies machine learning post-processing to Shorts without opt-out
YouTube announced an experiment on select Shorts using traditional machine learning to unblur, denoise, and improve clarity during processing. Creators reported that this altered their videos' appearance without an opt-out option, and some accused YouTube of misleading about the use of AI. YouTube acknowledged the experiment and said it would consider feedback, but did not offer a rollback or opt-out.
- Company involved
- YouTube
- AI system involved
- YouTube Shorts post-processing
7 source articles · read the reporting →
Xuhui police expand facial recognition surveillance to profile 1.1 million residents
The Xuhui District branch of the Shanghai Municipal Bureau of Public Security is expanding its Intelligent Image Recognition System, adding 2,500 facial recognition cameras and increased computing capacity to build profiles of residents and flag deviations. The project, contracted to US-sanctioned FiberHome, is designed to match each face to files on more than 50 million people. Officials say the system will analyse behaviour patterns and trigger early warnings.
- Company involved
- Shanghai Municipal Bureau of Public Security, Xuhui District Branch
- AI system involved
- Intelligent Image Recognition System
3 source articles · read the reporting →
Users jailbreak Luma Labs Dream Machine to generate porn
Users have jailbroken Luma Labs' Dream Machine, an AI video generator, to create explicit videos. The system's safeguards were bypassed to generate pornographic content. The videos are crude but demonstrate the potential for widespread AI-generated porn. Luma Labs' terms of service prohibit such content.
- Company involved
- Luma Labs
- AI system involved
- Dream Machine
3 source articles · read the reporting →
NewsBreak AI generates false news stories, harms local charities
NewsBreak, the most downloaded US news app, used AI to generate news stories, including a fabricated shooting in Bridgeton, New Jersey. The AI-produced content also contained errors that caused charities to turn away people needing services. Former employees and a consultant raised concerns about the practice, and the company has faced multiple copyright lawsuits. NewsBreak maintains it removes inaccurate content and complies with US laws.
- Company involved
- NewsBreak
- AI system involved
- NewsBreak app
2 source articles · read the reporting →
