Fake Luma Dream Machine AI sites deliver Noodlophile infostealer
Cybercriminals set up Facebook pages impersonating Luma Dream Machine and linked to fake AI video generation websites. Users who uploaded images received an archive containing a malicious executable instead of a video. The executable launched a multi-stage attack that installed Noodlophile, which harvests browser credentials, cookies and cryptocurrency wallet information. Morphisec reported the campaign.
8 source articles · read the reporting →
Duke University's PULSE AI upscaler turns Barack Obama white due to racial bias
Duke University researchers developed PULSE, an AI tool that upscales blurry face photos using Nvidia's StyleGAN model. When tested, it generated a white face from a pixelated image of Barack Obama, revealing racial bias. The bias stemmed from StyleGAN's training data being predominantly white and the evaluation dataset CelebA being 90% white. The researchers acknowledged the issue and updated their paper, while Nvidia stated it is working to mitigate bias.
- Company involved
- Duke University
- AI system involved
- PULSE
2 source articles · read the reporting →
Scatter Lab shuts down Lee Luda chatbot after hate speech
Scatter Lab's Lee Luda chatbot, a conversational AI on Facebook, generated hate speech targeting Black, lesbian, disabled, and trans people. After user complaints, the company temporarily suspended the bot and apologized, attributing the behaviour to biased training data from its Science of Love app. Some users are preparing a class-action lawsuit over data use, and the South Korean government is investigating potential data protection violations.
- Company involved
- Scatter Lab
- AI system involved
- Lee Luda
10 source articles · read the reporting →
MIT-IBM Watson AI Lab's AI Portrait Ars produces whitewashed portraits of people of colour
AI Portrait Ars, developed by researchers at the MIT-IBM Watson AI Lab, is reported to have generated Renaissance-style portraits that lightened the skin and altered the facial features of people of colour. The tool, trained on tens of thousands of paintings from the Western artistic tradition, was criticised for reproducing that bias in its data set. The creators acknowledged the bias but did not respond to a request for comment.
- Company involved
- MIT-IBM Watson AI Lab
- AI system involved
- AI Portrait Ars
10 source articles · read the reporting →
Chinese military researchers used Meta's Llama 2 to develop defense chatbot ChatBIT
Chinese military researchers, including two affiliated with the People's Liberation Army, reportedly used Meta's Llama 2 AI model to develop a defense chatbot called ChatBIT. According to Reuters, the chatbot is designed to gather and process intelligence and offer information for operational decision-making. Meta stated that the use was unauthorized and contrary to its acceptable use policy.
- Company involved
- People's Liberation Army (PLA)
- AI system involved
- ChatBIT
6 source articles · read the reporting →
College student uses GPT-3 to create fake blog that reaches #1 on Hacker News
Liam Porr, a college student at UC Berkeley, used OpenAI's GPT-3 language model to generate a fake blog under a fake name. One of the posts reached the number-one spot on Hacker News, fooling tens of thousands of readers. Porr later confessed and retired the blog after two weeks. The experiment demonstrated the ease of creating convincing AI-generated content.
- AI system involved
- GPT-3
10 source articles · read the reporting →
Israeli military uses Smash Hopper turret against Palestinian protesters
The Israeli military deployed Smash Hopper remote-controlled gun turrets in the West Bank. The turrets fire stun grenades, less-than-lethal bullets, and tear gas at Palestinian protesters without warning. The systems were observed at Al-Aroub refugee camp and Hebron in October 2022. The Smash Hopper is produced by Smart Shooter.
- Company involved
- Israeli military
- AI system involved
- Smash Hopper
10 source articles · read the reporting →
Deepfake creator Xiao swaps Yang Mi's face onto Athena Chu's body
A Chinese creator named Xiao used deepfake technology to superimpose actress Yang Mi's face onto Athena Chu's body in videos from a 1994 TV series. The videos went viral on Bilibili and Weibo, raising concerns about privacy and potential harassment. Xiao stated he made the videos to educate the public about deepfakes. The videos were later deleted.
10 source articles · read the reporting →
Huawei and Megvii tested Uyghur alarm facial recognition system
Huawei and Megvii conducted an interoperability test in 2018 that verified a 'Uyghur alarm' function in Megvii's facial recognition system, according to a confidential report found by IPVM. The system could identify Uyghur minorities and alert police. Huawei denied real-world application and deleted the report after IPVM inquired. Megvii denied targeting ethnic groups. The test report was publicly accessible on Huawei's European website.
- Company involved
- Huawei
- AI system involved
- Megvii Dynamic Face Recognition
10 source articles · read the reporting →
AI emotion-detection software tested on Uyghurs in Xinjiang
A software engineer alleged that AI-based emotion-detection cameras were installed in police stations in Xinjiang and tested on Uyghur detainees. The system, placed three metres from subjects, analysed facial expressions and skin pores to create a pie chart indicating emotional state, with red suggesting anxiety, which was used as an indication of guilt. The engineer claimed the software was intended for 'pre-judgement without any credible evidence'. Human Rights Watch described the material as shocking, and the Chinese embassy denied the existence of such technology.
- Company involved
- Chinese government
- AI system involved
- Emotion detection camera
10 source articles · read the reporting →
Study finds X algorithm boosted right-wing accounts after Musk endorsement
A study from Queensland University of Technology found that X's algorithm was changed in mid-July 2024 to systematically boost Republican-leaning accounts and Elon Musk's account. The analysis showed a significant increase in engagement for right-wing accounts after Musk endorsed Donald Trump. The findings suggest possible algorithmic bias, but X has not commented.
- Company involved
- X
6 source articles · read the reporting →
Stable Diffusion amplifies racial and gender stereotypes in generated images
An analysis by Bloomberg of over 5,000 images generated by Stability AI's Stable Diffusion found that the text-to-image model amplifies racial and gender stereotypes. The model overrepresented lighter-skinned men in high-paying jobs and darker-skinned people in low-paying jobs, and underrepresented women in positions of power. Stability AI acknowledged the inherent biases in its models and stated it is working on mitigation.
- Company involved
- Stability AI
- AI system involved
- Stable Diffusion
8 source articles · read the reporting →
DeepSeek-R1 censors 85% of sensitive Chinese political prompts in tests
Promptfoo tested DeepSeek-R1 against a dataset of 1,360 politically sensitive prompts and found that about 85% of them were refused. The refusals followed a standard form aligned with Chinese Communist Party policy. The testing also demonstrated that the censorship could be trivially bypassed using simple jailbreak techniques, such as prompt injection or changing the context.
- Company involved
- DeepSeek
- AI system involved
- DeepSeek-R1
5 source articles · read the reporting →
AI-driven Shadow Play network spreads pro-China propaganda on YouTube
The Australian Strategic Policy Institute (ASPI) revealed an extensive network of at least 30 YouTube channels, called Operation Shadow Play, that used generative AI to produce and publish pro-China and anti-US content. The network exploited YouTube's algorithmic recommendation system to cross-promote videos, gaining about 730,000 subscribers and 120 million views. The operation is alleged to be a coordinated influence campaign, possibly directed by a Mandarin-speaking controller, though the specific actor is unknown.
- Company involved
- Not specified (the operator is unknown)
- AI system involved
- Shadow Play
4 source articles · read the reporting →
Four commercial large language models perpetuate race-based medical misconceptions
A study published in npj Digital Medicine tested four commercial large language models (Bard, ChatGPT, GPT-4, and Claude) for their tendency to propagate discredited race-based medical beliefs. When asked about kidney function, lung capacity, and skin thickness, the models sometimes endorsed debunked racial differences, particularly affecting Black patients. The study concludes that these biases pose a potential hazard and urges caution before using such models in clinical decision-making.
- Company involved
- Not named in article (refers to commercial LLMs generically as Google's Bard, OpenAI's ChatGPT and GPT-4, and Anthropic's Claude)
- AI system involved
- Bard, ChatGPT, GPT-4, Claude
6 source articles · read the reporting →
Chinese Communist Party uses AI to create anti-American memes
Researchers at the Institute for Strategic Dialogue found that a campaign linked to the Chinese Communist Party is using AI image generators to create and spread memes designed to stoke anti-American sentiment by capitalising on President Biden's support of Israel's bombing campaign in Gaza. The memes are being distributed on social media platforms like X, Facebook, and YouTube. Platforms have taken down some accounts, but the campaign continues.
- Company involved
- Chinese Communist Party
4 source articles · read the reporting →
Cigna StressWaves Test found unreliable and invalid in independent study
A study published in Scientific Reports evaluated the Cigna StressWaves Test (CSWT), an AI tool that claims to assess psychological stress from speech. The study found that the CSWT had poor test-retest reliability and poor validity compared to the Perceived Stress Scale. The authors warned that widespread availability of the tool could lead to misleading results and negative consequences for users making healthcare decisions. Cigna has not publicly responded to the findings.
- Company involved
- Cigna
- AI system involved
- Cigna StressWaves Test
4 source articles · read the reporting →
Houston's ShotSpotter system delays police response and over-polices minority communities
Houston Police Department deployed ShotSpotter, a gunshot detection system, in Southeast and Northwest Houston starting in late 2020. The system alerts police to suspected gunfire, but over 80% of alerts are unfounded. This has led to longer response times for other calls and increased police presence in predominantly Black and brown neighborhoods, causing anxiety and concerns of over-policing. Critics argue the technology has not reduced gun violence and diverts resources from more effective strategies.
- Company involved
- Houston Police Department
- AI system involved
- ShotSpotter
7 source articles · read the reporting →
AI-cloned videos of dead celebrities spark family outrage in China
On Douyin, Chinese creators used AI to generate videos of deceased celebrities including Coco Lee, Qiao Renliang, and Yao Beina, making them appear to speak and move. Families, such as Qiao Renliang's father, criticized the practice as disrespectful and legally infringing on image rights. Experts noted that China's Civil Code protects the rights of the deceased, and families could sue for infringement. Some videos were deleted after backlash, but the practice continues, particularly for foreign celebrities.
- Company involved
- Douyin (content platform)
6 source articles · read the reporting →
Chicago class action challenges ShotSpotter's discriminatory stop-and-frisk deployments
ShotSpotter, a gunshot detection system used by the Chicago Police Department, generates alerts that lead to police deployments and stop-and-frisk stops. Analyses show over 90% of alerts are unfounded, and the system is deployed only in predominantly Black and Latinx districts. A class-action lawsuit against the City of Chicago alleges the system fuels unconstitutional and discriminatory policing, including false accusations against individuals like Michael Williams.
- Company involved
- City of Chicago
- AI system involved
- ShotSpotter
7 source articles · read the reporting →
Storm-1376 used AI-generated fake audio during Taiwan election, Microsoft reports
Microsoft's Threat Analysis Center reported that the Chinese state-linked group Storm-1376 posted suspected AI-generated fake audio of former Taiwanese presidential candidate Terry Gou endorsing another candidate on election day in January 2024. Gou had made no such statement, and YouTube removed the content before it reached a wide audience. The group has also used AI-generated memes and news anchors as part of influence operations in Taiwan and the United States.
- Company involved
- Storm-1376 (also known as Spamouflage and Dragonbridge)
7 source articles · read the reporting →
Huawei Pura 70 series AI editing tool removes clothing unintentionally
Huawei's Pura 70 series includes an AI retouching feature that can inadvertently remove clothing from images. Users on Weibo posted videos demonstrating this, raising privacy concerns. Huawei customer service acknowledged the issue, attributing it to algorithm loopholes, and promised a fix in upcoming system updates.
- Company involved
- Huawei
- AI system involved
- Pura 70 series AI retouching feature
5 source articles · read the reporting →
Evolv weapon detection system falsely flags Chromebooks as weapons
Evolv's AI-based weapon detection system has been reported to falsely identify certain Chromebook laptops as weapons, causing false alarms at security checkpoints. The system is used in venues such as schools. The article discusses the flaw and user experiences. No official response from Evolv has been mentioned.
- Company involved
- Evolv
- AI system involved
- Evolv
5 source articles · read the reporting →
Xiaohongshu accused of using illustrators' artwork to train its AI
Illustrators accused Xiaohongshu, a Chinese lifestyle platform, of using their artwork without permission to train its AI image generator, Trik AI. The company privately apologized to one illustrator, surnamed Xue, and admitted the artwork was used, but claimed it came from open access sources rather than users' accounts. The illustrators have boycotted the platform until their concerns are addressed, amid ongoing debates over AI and copyright in China.
- Company involved
- Xiaohongshu
- AI system involved
- Trik AI
7 source articles · read the reporting →
