South Korean AI recreates deceased singer's voice for TV show
South Korean broadcaster SBS used AI technology from Supertone to recreate the voice of deceased folk singer Kim Kwang-seok for a TV show. The AI system, Singing Voice Synthesis, learned from 20 of Kim's songs to perform a cover of 'I Miss You'. The family gave permission after hearing the AI version. The show aired on January 29, 2021.
- Company involved
- SBS
- AI system involved
- Singing Voice Synthesis (SVS)
10 source articles · read the reporting →
iFlytek suspends AI study device after it generates essay critical of Mao
iFlytek, a Chinese AI firm, suspended sales of its study assistance device after a parent complained that the device generated an essay critical of Mao Zedong, calling him 'narrow-minded' and 'intolerant' for starting the Cultural Revolution. The company blamed a supplier for the content and said both the supplier and staff had been punished. Shares in iFlytek fell 10% following the news.
- Company involved
- iFlytek
- AI system involved
- study assistance device
3 source articles · read the reporting →
AI companies trained on Hollywood writers' dialogue without consent
The Atlantic investigation found that dialogue from over 53,000 movies and 85,000 TV episodes was included in the OpenSubtitles data set used by Apple, Anthropic, Meta, Nvidia, Salesforce, and Bloomberg to train AI systems. The article alleges that this use of copyrighted material was done without permission from the writers.
4 source articles · read the reporting →
Deep-Live-Cam enables real-time face swapping, raising fraud concerns
A software package called Deep-Live-Cam has gone viral for enabling real-time face swapping on webcams using a single photo. The tool, which is free and open source, allows anyone to impersonate another person's appearance in a video chat, raising concerns about remote deception and fraud. The article notes that similar technology has been used in a $25 million heist in Hong Kong and in audio deepfake scams.
- AI system involved
- Deep-Live-Cam
7 source articles · read the reporting →
ElevenLabs voice cloning tool used to create deepfake celebrity audio clips
Speech AI startup ElevenLabs launched a beta voice cloning tool. Within days, users on 4chan posted deepfake audio clips featuring voices resembling celebrities like Emma Watson reading offensive material. ElevenLabs acknowledged the misuse and said it is considering additional safeguards.
- Company involved
- ElevenLabs
- AI system involved
- ElevenLabs platform
10 source articles · read the reporting →
Microsoft Copilot gives false election information in Swiss and Bavarian elections
AI Forensics and Algorithm Watch tested Microsoft's Copilot chatbot during the 2023 Swiss federal elections and German state elections in Hesse and Bavaria. They found that one-third of its answers contained factual errors, including incorrect election dates, outdated candidates, or fabricated controversies. The chatbot also evaded questions 40% of the time and sometimes attributed false information to credible sources. The organisations alerted Microsoft multiple times.
- Company involved
- Microsoft
- AI system involved
- Copilot (Bing Chat)
10 source articles · read the reporting →
Presto Automation uses off-site human agents to double-check AI drive-thru orders
Presto Automation Inc, which markets an AI voice assistant for drive-thru ordering, used off-site human agents in countries including the Philippines to double-check orders in more than 70% of customer interactions, according to SEC filings reported by Bloomberg. The company told Bloomberg that the process helps train its system and should reduce human intervention over time. Presto's drive-thru AI is used in more than 400 restaurants, including Del Taco, Carl's Jr and Checkers, and its stock fell more than 10% after the reports.
- Company involved
- Presto Automation Inc.
8 source articles · read the reporting →
Meta tracks employee keystrokes on Google, LinkedIn, Wikipedia for AI training
Meta is using an internal tool, Model Capability Initiative (MCI), to capture employees' keystrokes, mouse movements and screen contents on work computers, including on sites such as Google, LinkedIn, Wikipedia and Slack, to train AI agents. Meta confirmed the project and said safeguards protect sensitive content and that the data is not used for other purposes. Employees raised concerns in internal messages that the tool could expose passwords, product details and personal information. A Meta memo said staff can avoid capture by not doing personal work on work computers.
- Company involved
- Meta
- AI system involved
- Model Capability Initiative (MCI)
5 source articles · read the reporting →
Voice actor Greg Marston alleges IBM cloned his voice without permission
Voice actor Greg Marston alleges that IBM used voice recordings he made for them 20 years ago to create an AI clone of his voice without his consent. The clone is being sold by a third party under the name 'Conor', and Marston says he is now competing with it for work. IBM has acknowledged the complaint and claims it obtained the rights from the original agreement. Marston is sending a take-down notice to stop the use.
- Company involved
- IBM
7 source articles · read the reporting →
Google used 11,000 novels without permission to train AI
Google researchers fed 11,000 novels, including works by authors Rebecca Forster and Erin McCarthy, into a neural network to improve conversational AI without seeking permission. The Authors Guild condemned the use as a 'blatantly commercial use of expressive authorship'. Google defended the practice as fair use, stating it does not harm authors. The authors expressed frustration at not being informed or compensated.
- Company involved
- Google
10 source articles · read the reporting →
AI image generators produce misleading election images, study finds
A study by the Center for Countering Digital Hate found that leading AI image generators, including Midjourney, DreamStudio, ChatGPT Plus, and Microsoft Image Creator, could be manipulated to create misleading election-related images. The researchers used jailbreaking techniques to bypass safety measures, producing photorealistic images of candidates in compromising situations or of voting fraud. The companies responded by stating they are updating policies and implementing safeguards, but the study suggests existing protections are inadequate.
- Company involved
- Midjourney, Stability AI, OpenAI, Microsoft
- AI system involved
- Midjourney, DreamStudio, ChatGPT Plus, Microsoft Image Creator
8 source articles · read the reporting →
Deepfake Biden robocall likely made with ElevenLabs tools, researchers say
In January 2024, some voters in New Hampshire received an AI-generated robocall impersonating President Joe Biden, telling them not to vote in the state's primary election. Two teams of audio experts, from Pindrop and UC Berkeley, analysed the call and concluded with high confidence that it was likely created using technology from voice-cloning startup ElevenLabs. The call's originator is unknown, and ElevenLabs stated it is dedicated to preventing misuse but could not comment on the specific incident. The incident highlights concerns about AI-generated disinformation ahead of the 2024 US elections.
- Company involved
- ElevenLabs
- AI system involved
- ElevenLabs voice-cloning tools
9 source articles · read the reporting →
Guangzhou court finds AI company infringed Ultraman copyright
The Guangzhou Internet Court ruled on 8 February 2024 that an unnamed AI company infringed the copyright and adaptation rights of the plaintiff, the exclusive licensee of the Ultraman series images in China. The defendant operated a website offering AI conversation and AI-generated painting services, accessible only to paying members. The plaintiff alleged that the defendant used its Ultraman images without authorisation to train its model and generate substantially similar images. The court ordered the defendant to pay 10,000 yuan ($1,389) in compensation.
- Company involved
- unnamed AI company
- AI system involved
- AI conversation and AI-generated painting website
9 source articles · read the reporting →
Storm-1376 used AI-generated fake audio during Taiwan election, Microsoft reports
Microsoft's Threat Analysis Center reported that the Chinese state-linked group Storm-1376 posted suspected AI-generated fake audio of former Taiwanese presidential candidate Terry Gou endorsing another candidate on election day in January 2024. Gou had made no such statement, and YouTube removed the content before it reached a wide audience. The group has also used AI-generated memes and news anchors as part of influence operations in Taiwan and the United States.
- Company involved
- Storm-1376 (also known as Spamouflage and Dragonbridge)
7 source articles · read the reporting →
Beijing Internet Court rules AI voice cloning infringes personality rights
A Chinese dubbing artist, Yin, discovered that his voice was being used without permission in a text-to-speech AI product. The product was developed by a software company using recordings provided by a cultural media company, and was made available on a platform operated by a smart technology company. The Beijing Internet Court ruled that the AI-generated voice was highly similar to the plaintiff's voice and infringed his personality rights. The court ordered the defendants to stop infringement and pay damages of 250,000 RMB.
5 source articles · read the reporting →
Voice actors sue LOVO over alleged theft of voices for AI training
Voice actors Paul Skye Lehrman and Linnea Sage filed a proposed class action against AI startup LOVO, alleging the company misappropriated their voices to train its text-to-speech system Genny. The actors claim they were hired on Fiverr under the pretence of academic research but later discovered their voices were used commercially without consent. Lehrman reports a 50% decline in work and loss of control over how his voice is used. The lawsuit, filed in New York federal court, seeks to represent other affected voiceover artists and obtain a court order blocking the practice.
- Company involved
- LOVO
- AI system involved
- Genny
6 source articles · read the reporting →
Microsoft sued after MSN circulates AI-generated fake article accusing Irish broadcaster of sexual misconduct
Dave Fanning, an Irish DJ and talk show host, is suing Microsoft and BNN Breaking after MSN featured an AI-generated article falsely claiming he faced trial for alleged sexual misconduct. The article was produced by BNN Breaking's AI system without human fact-checking and appeared on MSN's homepage for several hours before removal. Fanning alleges defamation and reputational damage. The lawsuit is pending.
- Company involved
- Microsoft
- AI system involved
- BNN Breaking
6 source articles · read the reporting →
Mistral releases unmoderated chatbot that gives instructions on murder and ethnic cleansing
Mistral, a French AI startup valued at $260 million, released an open-source large language model named Mistral-7B-v0.1 without safety evaluations or moderation mechanisms. The model readily provides detailed instructions on murder, ethnic cleansing, suicide, and other harmful content. Mistral added a statement after the release acknowledging the lack of moderation but did not remove the model, which is distributed via torrent and cannot be deleted.
- Company involved
- Mistral
- AI system involved
- Mistral-7B-v0.1
7 source articles · read the reporting →
ChatGPT and Copilot repeated false claim about CNN debate delay
On June 27, 2024, OpenAI's ChatGPT and Microsoft's Copilot generated false information about a broadcast delay during the CNN presidential debate. The chatbots repeated a debunked claim that CNN would implement a 1-2 minute delay, citing conservative misinformation. OpenAI later corrected ChatGPT's response, while Microsoft did not respond to requests for comment.
- Company involved
- OpenAI and Microsoft
- AI system involved
- ChatGPT and Microsoft Copilot
6 source articles · read the reporting →
Delta uses AI from Fetcherr for domestic ticket pricing
Delta Air Lines is using generative AI from Fetcherr to determine some domestic flight prices, currently covering 3% of its network with plans to reach 20% by end of 2025. Democratic senators expressed concern that the AI could be used for individualized pricing based on personal data, leading to higher fares. Delta denies using personal data in pricing and states it complies with regulations. No actual harm has been reported.
- Company involved
- Delta Air Lines
- AI system involved
- Fetcherr
8 source articles · read the reporting →
Luma's Dream Machine generates video with Disney character
Luma's AI video tool Dream Machine generated a trailer that included a recognizable character from Disney's Monsters, Inc. The company's CEO said a user uploaded an image containing the character, which the system then animated. The incident raises concerns about lack of transparency in training data and potential copyright infringement. Disney has not publicly commented.
- Company involved
- Luma
- AI system involved
- Dream Machine
7 source articles · read the reporting →
OpenAI's Sora 2 allowed unauthorized use of Bryan Cranston's likeness
OpenAI's Sora 2 video generation platform allowed users to create images of actor Bryan Cranston without his permission, leading to a video of his 'Breaking Bad' character interacting with Michael Jackson. After outcry from Cranston, SAG-AFTRA, and talent agencies, OpenAI added guardrails and an opt-in protocol to protect performers' voices and likenesses. Cranston thanked OpenAI for the changes, and the union called it a positive resolution.
- Company involved
- OpenAI
- AI system involved
- Sora 2
6 source articles · read the reporting →
Microsoft Copilot vulnerable to automated phishing and data theft
Security researcher Michael Bargury demonstrated at Black Hat that Microsoft's Copilot AI can be manipulated by attackers to send phishing emails, extract private data, and bypass security protections. The attacks exploit the AI's access to corporate data and its ability to perform actions on behalf of users. Microsoft acknowledged the findings and said it is working with the researcher to assess the vulnerabilities.
- Company involved
- Microsoft
- AI system involved
- Copilot
3 source articles · read the reporting →
BBC study finds AI chatbots produce inaccurate news summaries
A BBC study found that four major AI chatbots – ChatGPT, Copilot, Gemini and Perplexity – produced inaccurate summaries of BBC news articles. The study, conducted in December 2024, found that 51% of AI answers had significant issues and 19% introduced factual errors. The BBC's CEO called on tech companies to pull back their AI news summaries, warning of potential real-world harm. OpenAI responded by stating it supports publishers and helps users discover quality content.
- Company involved
- OpenAI, Microsoft, Google, Perplexity
- AI system involved
- ChatGPT, Copilot, Gemini, Perplexity
5 source articles · read the reporting →
