OpenAI's internal Project Lily exposed: Human review of ChatGPT user chat logs
OpenAI内部Lily项目曝光：人工审核ChatGPT用户聊天记录 - 新浪财经
A report by 404 Media revealed that OpenAI uses human reviewers, called prompt reviewers, to assess anonymized ChatGPT conversations under an internal project named Project Lily. Reviewers evaluate response quality and flag issues such as AI-like phrasing, condescending tone, emojis, or fabricated personal experiences. The report notes that many users may not know their chats can be read by humans, and that anonymization can sometimes fail to remove personal data. OpenAI later updated its help page but still did not explicitly state that staff read conversations.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
1 source article · read the reporting →
Gmail users warned of AI voice phishing scam impersonating Google support
Cybercriminals are using AI-generated voices to impersonate Google support in phone calls to Gmail users, attempting to trick them into revealing account credentials. The scam involves a caller ID that appears legitimate and a follow-up email from a spoofed Google address. Victims are told their account has been compromised and are asked to provide a recovery code. Google has advised users to enable Advanced Protection to secure their accounts.
8 source articles · read the reporting →
xAI Data Center Noise Controversy, Explained - basenor.com
The xAI data center facility generated persistent noise, vibrations, and air quality issues from gas turbines, affecting nearby residents in Southaven, Mississippi.
- Company involved
- xAI
1 source article · read the reporting →
OpenAI’s Whisper AI hallucinates violent language in transcriptions
Researchers at Cornell discovered that OpenAI's Whisper speech-to-text system occasionally hallucinates violent language, invented personal information, and fake websites, particularly when transcribing speech from people with longer pauses, such as those with speech impairments. The findings were presented at the FAccT conference. OpenAI has since improved the model, reducing the hallucination rate. The researchers caution that even a small percentage of such errors could cause significant harm in contexts like hiring, courtrooms, or medical settings.
- Company involved
- OpenAI
- AI system involved
- Whisper
1 source article · read the reporting →
Services Australia voiceprint system fooled by AI voice clone
A Guardian Australia investigation found that the voiceprint system used by Services Australia's Centrelink and the Australian Taxation Office can be bypassed using an AI-generated voice clone. A journalist created a clone of their own voice from four minutes of audio and used it with a customer reference number to access their Centrelink self-service account. The system is used by millions of Australians for identity verification over the phone. Services Australia stated that it continually assesses risks and applies additional tests if unusual circumstances are detected, but did not commit to changing the technology.
- Company involved
- Services Australia
- AI system involved
- Voiceprint
1 source article · read the reporting →
Voice Actors Doxed and Harassed with AI-Generated Voices on Twitter
In February 2023, several video game voice actors were targeted by online trolls who used AI voice generation technology to create synthetic replicas of their voices. The fake voices were used to read out the actors' home addresses and offensive statements, which were then posted on Twitter. The actors reported the posts to Twitter, but the platform's support system initially deemed some of the content acceptable. ElevenLabs, the AI voice company allegedly used, denied that the doxing clips were created with its software and stated it had introduced additional safeguards to prevent misuse.
- Company involved
- ElevenLabs
1 source article · read the reporting →
Journalist Bypasses Lloyds Bank Voice ID with AI-Generated Voice Clone
A journalist used an AI-generated clone of his own voice to bypass the voice authentication system of Lloyds Bank, gaining access to his account. The experiment, conducted using ElevenLabs' free voice synthesis service, demonstrated that voice biometrics can be fooled by synthetic voices. Lloyds Bank stated it is aware of the threat and is deploying countermeasures, but has not seen real-world fraud using this method. The incident raises concerns about the security of voice verification used by many banks.
- Company involved
- Lloyds Bank
- AI system involved
- Voice ID
1 source article · read the reporting →
Yandex Voice Assistant Alice Gives Pro-Stalin and Pro-Violence Responses
Yandex's Russian-language voice assistant Alice was found to provide responses supportive of Joseph Stalin, violence, domestic violence, and opposing gay marriage. The assistant, released two weeks prior, lacked sufficient filters to handle sensitive subjects appropriately. Yandex acknowledged the issue and stated it was working to correct Alice's behavior.
- Company involved
- Yandex
- AI system involved
- Alice
5 source articles · read the reporting →
OpenAI Pauses ChatGPT Voice After Scarlett Johansson Likeness Complaint
Scarlett Johansson alleges that OpenAI's ChatGPT 4.0 voice named Sky sounded eerily similar to her own voice, despite her declining an offer to be the voice. OpenAI paused the use of Sky after she hired legal counsel and sent letters demanding an explanation. The company stated the voice belongs to a different actress and was not intended to mimic Johansson. Johansson seeks transparency and resolution.
- Company involved
- OpenAI
- AI system involved
- ChatGPT 4.0 Voice Mode (Sky)
10 source articles · read the reporting →
ScotRail replaces AI announcer after voice cloning dispute
ScotRail launched a new AI onboard announcer using the voice of an employee, Vannessa Sloan, after voiceover artist Gayanne Potter alleged her voice was used without permission to train the previous AI announcer, Iona. Potter claimed she was told her recordings for ReadSpeaker would be used for accessibility software, not commercial purposes. The new system uses Sloan's voice and is described by ScotRail as 'ethically produced'.
- Company involved
- ScotRail
- AI system involved
- Iona
6 source articles · read the reporting →
Voiceverse NFT firm admits it stole AI voice work
NFT company Voiceverse admitted to using text-to-speech audio from the free service 15.ai without credit in marketing for its AI-generated voice NFTs. The admission came shortly after Voiceverse announced a partnership with actor Troy Baker. 15.ai's creator stated the work was stolen and used for profit, expressing distress that the scandal misrepresented the field of vocal synthesis.
- Company involved
- Voiceverse
1 source article · read the reporting →
MBN introduces AI news anchor AI Kim in South Korea
MBN, a South Korean cable channel, introduced an AI-powered news anchor based on anchor Kim Ju-ha. The AI, named AI Kim, was created through deep learning using 10 hours of video of Kim. It presents news in her voice and manner, intended to allow 24/7 broadcasting.
- Company involved
- MBN
- AI system involved
- AI Kim
10 source articles · read the reporting →
Attempted deepfake CEO voicemail fraud against an unnamed technology company
An attacker used synthetic audio deepfake to leave a voicemail impersonating the CEO of a technology company, asking an employee to call back to finalize an urgent business deal. The employee found it suspicious and referred the matter to the legal department, avoiding any loss. Nisos analyzed the audio and found inconsistencies in pitch and tone.
10 source articles · read the reporting →
South Korean AI recreates deceased singer's voice for TV show
South Korean broadcaster SBS used AI technology from Supertone to recreate the voice of deceased folk singer Kim Kwang-seok for a TV show. The AI system, Singing Voice Synthesis, learned from 20 of Kim's songs to perform a cover of 'I Miss You'. The family gave permission after hearing the AI version. The show aired on January 29, 2021.
- Company involved
- SBS
- AI system involved
- Singing Voice Synthesis (SVS)
10 source articles · read the reporting →
David Attenborough objects to AI replica of his voice
An AI-generated replica of Sir David Attenborough's voice was found on a website, speaking about topics he never addressed, including politics. Attenborough said he was 'profoundly disturbed' and 'greatly object' to the use of his identity. BBC News played the cloned voice alongside a real recording, showing little difference. The incident highlights concerns about AI voice cloning of trusted public figures.
10 source articles · read the reporting →
ElevenLabs AI voice generation used in Russian influence operation targeting Europe
The article reports that a Russian influence campaign, dubbed "Operation Undercut," very likely used ElevenLabs' AI voice generation technology to create realistic voiceovers for fake news videos. The videos targeted European audiences to undermine support for Ukraine. Recorded Future's researchers used ElevenLabs' own AI Speech Classifier to detect the AI-generated audio. The campaign was attributed to the Russia-based Social Design Agency, which the U.S. government sanctioned. The overall impact on public opinion was minimal.
- Company involved
- Social Design Agency
- AI system involved
- ElevenLabs AI voice generation
7 source articles · read the reporting →
ElevenLabs voice cloning tool used to create deepfake celebrity audio clips
Speech AI startup ElevenLabs launched a beta voice cloning tool. Within days, users on 4chan posted deepfake audio clips featuring voices resembling celebrities like Emma Watson reading offensive material. ElevenLabs acknowledged the misuse and said it is considering additional safeguards.
- Company involved
- ElevenLabs
- AI system involved
- ElevenLabs platform
10 source articles · read the reporting →
Amazon Q chatbot leaks confidential data and hallucinates in public preview
Amazon's AI chatbot Q, launched in public preview, is experiencing severe hallucinations and leaking confidential data including AWS data center locations and internal discount programs, according to internal documents obtained by Platformer. Employees marked the incident as severity 2, requiring urgent fixes. Amazon denied the leak and said it will continue to tune the system.
- Company involved
- Amazon
- AI system involved
- Amazon Q
10 source articles · read the reporting →
Presto Automation uses off-site human agents to double-check AI drive-thru orders
Presto Automation Inc, which markets an AI voice assistant for drive-thru ordering, used off-site human agents in countries including the Philippines to double-check orders in more than 70% of customer interactions, according to SEC filings reported by Bloomberg. The company told Bloomberg that the process helps train its system and should reduce human intervention over time. Presto's drive-thru AI is used in more than 400 restaurants, including Del Taco, Carl's Jr and Checkers, and its stock fell more than 10% after the reports.
- Company involved
- Presto Automation Inc.
8 source articles · read the reporting →
Witcher 3 mod uses AI to replicate Geralt's voice without actor's consent
A mod for The Witcher 3: Wild Hunt called 'A Night to Remember' uses AI technology (CyberVoice) to generate new voice lines for Geralt, simulating the voice of actor Doug Cockle without his permission. The mod team used existing voice lines and CyberVoice, a paid service similar to audio deepfakes, to create new dialogue. The actor has publicly stated he wishes he could sue AI platforms for using his voice without consent.
- Company involved
- CD Projekt Red
- AI system involved
- CyberVoice
9 source articles · read the reporting →
Teleperformance deploys AI to neutralise Indian call centre agents' accents
Teleperformance, the world's largest call centre operator, has announced it is using AI from Sanas to modify the accents of its Indian employees in real time. The technology, called accent translation, aims to make agents sound more neutral to native English speakers. The company invested $13 million in Sanas and gained exclusive rights. No specific incident of harm has been reported.
- Company involved
- Teleperformance
- AI system involved
- Sanas AI
6 source articles · read the reporting →
Fake AI audio of Sadiq Khan nearly caused disorder
A fake audio clip using AI to clone London Mayor Sadiq Khan's voice was shared online in November 2023, allegedly making inflammatory remarks about Remembrance Day. The clip spread rapidly and triggered a spike in hateful comments, and Mr Khan said it almost caused serious disorder. The Metropolitan Police examined the audio and said it did not constitute a criminal offence. The creator of the audio has not been identified.
8 source articles · read the reporting →
Beijing Internet Court rules AI voice cloning infringes personality rights
A Chinese dubbing artist, Yin, discovered that his voice was being used without permission in a text-to-speech AI product. The product was developed by a software company using recordings provided by a cultural media company, and was made available on a platform operated by a smart technology company. The Beijing Internet Court ruled that the AI-generated voice was highly similar to the plaintiff's voice and infringed his personality rights. The court ordered the defendants to stop infringement and pay damages of 250,000 RMB.
5 source articles · read the reporting →
Voice actors sue LOVO over alleged theft of voices for AI training
Voice actors Paul Skye Lehrman and Linnea Sage filed a proposed class action against AI startup LOVO, alleging the company misappropriated their voices to train its text-to-speech system Genny. The actors claim they were hired on Fiverr under the pretence of academic research but later discovered their voices were used commercially without consent. Lehrman reports a 50% decline in work and loss of control over how his voice is used. The lawsuit, filed in New York federal court, seeks to represent other affected voiceover artists and obtain a court order blocking the practice.
- Company involved
- LOVO
- AI system involved
- Genny
6 source articles · read the reporting →
