Apple AI voicemail transcription turns garage invite into X-rated message
Louise Littlejohn, 66, from Dunfermline, received a voicemail from a Lookers Land Rover garage inviting her to an event. Apple's voicemail transcription service converted the message to text, but mistakenly inserted a question about sex and an insult. The actual voicemail was a conventional business call, and the error is believed to have been caused by background noise and scripted speech. Apple and the garage declined to comment.
- Company involved
- Apple
- AI system involved
- Apple voicemail transcription
1 source article · read the reporting →
The Voice of Sudan campaign uses AI to impersonate Omar al-Bashir on TikTok
An anonymous campaign called 'The Voice of Sudan' used AI voice conversion to create fake recordings of former Sudanese leader Omar al-Bashir, posting them on TikTok since late August 2023. The recordings, taken from a US-based commentator's broadcasts, caused confusion amid Sudan's civil war. TikTok removed the account after being contacted by the BBC, citing violations of its synthetic media policy. The incident highlights the ease of spreading audio disinformation.
- Company involved
- The Voice of Sudan
3 source articles · read the reporting →
Genoa prosecutor's automatic transcription error changes 'lawful' to 'illicit'
During an investigation into alleged corruption, the Genoa prosecutor's office used automatic speech-to-text software to transcribe an interrogation of suspect Roberto Spinelli. The software incorrectly transcribed 'finanziamenti leciti' (lawful financing) as 'finanziamenti illeciti' (illicit financing), an error that could have significantly influenced the case. The mistake was discovered on 26 May when the judge re-listened to the audio at the request of Spinelli's lawyers. The incident highlights concerns about the reliability of AI transcription tools in the Italian judicial system.
- Company involved
- Procura della Repubblica di Genova
1 source article · read the reporting →
AI-Written Complaint and Press Release in Pro Se Lawsuit Surprises Reporter
訴状もプレスリリースも「AIだのみ」 弁護士つけずに〝本人訴訟〟 驚いて取材すると… - ｄメニューニュース
A man relied entirely on AI to draft a lawsuit and a press release, then filed a pro se complaint against a yakiniku restaurant. He alleged the restaurant served him cold noodles that had fallen into a drain and sought 30,000 yen in damages. Even the decision to contact reporters was made with AI guidance.
- AI system involved
- Gemini
1 source article · read the reporting →
Federal Court: Barron v BT Funds AI Pleading Denied as 'AI Slop' - Wansom AI
The Federal Court denied leave to file an AI-generated pleading, delaying the applicant's case.
1 source article · read the reporting →
Apple fixing dictation bug that substitutes 'Trump' for 'racist'
Apple's iPhone dictation feature briefly displayed 'Trump' when users said 'racist' due to a speech recognition bug. The company acknowledged the issue and is rolling out a fix. A former Siri team member suggested the substitution could have been a deliberate internal prank, while Apple attributed it to phonetic overlap.
- Company involved
- Apple
- AI system involved
- iPhone dictation feature
1 source article · read the reporting →
Publicity Rights Extended to Voice in AI Imitation Suit; Tokyo Court Dismisses Deletion Request
声にも「パブリシティー権」初判断＝生成ＡＩで模倣、人気声優訴え―削除請求は退ける・東京地裁 - リスク対策.com
Voice actor Kenjiro Tsuda sued the operator of TikTok, alleging that AI-generated videos mimicked his voice without permission and violated his publicity rights. The Tokyo District Court ruled for the first time that a person's voice can be protected under publicity rights, akin to name and portrait. The court dismissed the request to delete the videos because they had already been removed from the platform.
- Company involved
- TikTok operator
1 source article · read the reporting →
NHK AI Translation Error Uses Chinese Name for Senkaku Islands
NHK's English language news report on the Japan-U.S. summit talks used Chinese subtitles that referred to the Senkaku Islands as the Diaoyu Islands. The error was caused by an instant automatic translation system using Google's AI. NHK terminated its multilingual subtitling service in nine languages following the incident, with its president stating the service was not appropriate due to unstable translations.
- Company involved
- NHK
- AI system involved
- automatic translation system
3 source articles · read the reporting →
Microsoft Bing Chat prompt injection reveals internal instructions
A Stanford student used a prompt injection attack to trick Microsoft's Bing Chat into revealing its hidden initial prompt instructions. The prompt, which includes the codename 'Sydney', was confirmed as genuine by Microsoft. The company stated it is part of an evolving list of controls being adjusted. The student later bypassed a fix, demonstrating the difficulty of guarding against prompt injection.
- Company involved
- Microsoft
- AI system involved
- Bing Chat
1 source article · read the reporting →
Google Duplex AI assistant to identify itself as robot after criticism
Google demonstrated its Duplex AI assistant making lifelike phone calls without disclosing it was a robot, sparking accusations of deceit. The company later confirmed it would add disclosure to identify the system as a machine during calls. The feature was not yet a finished product at the time.
- Company involved
- Google
- AI system involved
- Google Duplex
10 source articles · read the reporting →
Jason Isbell, Musicians Sue Suno Over Alleged AI Use of Artists’ Names and Voices - Law Commentary
Suno AI generated music mimicking the identities of Jason Isbell and other artists without authorization.
- Company involved
- Suno
- AI system involved
- Suno
1 source article · read the reporting →
Sugar and Dice loses money in AI voice scam
Sugar and Dice, a game and coffee store in Havre, Montana, was targeted by a phone scam using AI voice replication. An employee received a call that appeared to be from the store owner's number and heard a voice that sounded like the owner, who instructed him to follow fraudulent instructions. The employee complied, compromising the store's bank details and causing monetary losses. Store owner Kevin Zorn does not blame the employee and believes the business will recover.
- Company involved
- Sugar and Dice
1 source article · read the reporting →
Imprompter attack extracts personal data from AI chatbots
Security researchers at UCSD and Nanyang Technological University developed a prompt-injection attack called Imprompter that covertly instructs large language models to extract personal information from user chats and send it to an attacker. The attack was tested on Mistral AI's LeChat and the Chinese chatbot ChatGLM, achieving nearly 80% success in test conversations. Mistral AI fixed the vulnerability; ChatGLM acknowledged security measures but did not directly confirm a fix.
- Company involved
- Mistral AI,Zhipu AI
- AI system involved
- LeChat,ChatGLM
7 source articles · read the reporting →
Attempted deepfake CEO voicemail fraud against an unnamed technology company
An attacker used synthetic audio deepfake to leave a voicemail impersonating the CEO of a technology company, asking an employee to call back to finalize an urgent business deal. The employee found it suspicious and referred the matter to the legal department, avoiding any loss. Nisos analyzed the audio and found inconsistencies in pitch and tone.
10 source articles · read the reporting →
The People v. Raziel Ruiz Alvarez (CA California): AI-hallucinated content in court filing, Published order; monetary sanction payable by Counsel…
The AI generated fabricated legal citations that were included in a court filing, affecting the defendant's legal representation and the court's resources.
- Company involved
- Attorney Siddell
1 source article · read the reporting →
South Korean AI recreates deceased singer's voice for TV show
South Korean broadcaster SBS used AI technology from Supertone to recreate the voice of deceased folk singer Kim Kwang-seok for a TV show. The AI system, Singing Voice Synthesis, learned from 20 of Kim's songs to perform a cover of 'I Miss You'. The family gave permission after hearing the AI version. The show aired on January 29, 2021.
- Company involved
- SBS
- AI system involved
- Singing Voice Synthesis (SVS)
10 source articles · read the reporting →
Snapchat's My AI chatbot denied knowing user location while keeping it
Snapchat's My AI chatbot, a GPT-based assistant, denied knowing the user's location when asked directly. Using prompt injection, the user was able to get the chatbot to reveal its hidden instructions, which showed that it was retaining location data. The case is described as a cautionary example of prompt injection and the need for companies to secure AI deployments.
- Company involved
- Snapchat
- AI system involved
- My AI
10 source articles · read the reporting →
Michael L. Ruiz v. Magellan Financial & Insurance Services (2) (D. Arizona): AI-hallucinated content in court filing, Formal public reprimand
AI-hallucinated fake quotations were included in court filings submitted by counsel Elizabeth Tate, misleading the court.
1 source article · read the reporting →
Bev Standing sues TikTok over unauthorized use of her voice for AI text-to-speech.
Voice actress Bev Standing is suing TikTok, alleging the company used her voice for its text-to-speech feature without compensation or consent. The lawsuit claims that TikTok's AI-generated voice spoke 'foul and offensive language,' causing irreparable harm to her reputation. The case highlights concerns among voice actors about AI voice cloning and the lack of legal protections.
- Company involved
- TikTok
- AI system involved
- TikTok text-to-speech feature
10 source articles · read the reporting →
Joann LeDoux v. Outliers, Inc. (2) (W.D. Washington): AI-hallucinated content in court filing, Expert Brief excluded/struck
The AI-generated hallucinated citations in an expert report led to the exclusion of the expert and dismissal of the plaintiff's case.
1 source article · read the reporting →
AI-generated songs using Stefanie Sun's voice go viral on Bilibili
AI-generated songs featuring the cloned voice of Mandopop singer Stefanie Sun have gone viral on Chinese video platform Bilibili. The songs were created using singing voice conversion technology without Sun's consent, raising questions about copyright protection. Sun has not publicly responded to the phenomenon.
10 source articles · read the reporting →
Answer.AI tests Devin and reports 14 failures in 20 tasks
Answer.AI's team tested Devin, an autonomous AI coding assistant, on 20 real-world tasks over a month. Devin succeeded in only 3 tasks, failed 14, and was inconclusive in 3. The team found Devin often produced overly complex or hallucinated solutions and could not recognize fundamental blockers. They ultimately decided to stick with tools that allow more human control.
- AI system involved
- Devin
5 source articles · read the reporting →
DeepSeek-R1 censors 85% of sensitive Chinese political prompts in tests
Promptfoo tested DeepSeek-R1 against a dataset of 1,360 politically sensitive prompts and found that about 85% of them were refused. The refusals followed a standard form aligned with Chinese Communist Party policy. The testing also demonstrated that the censorship could be trivially bypassed using simple jailbreak techniques, such as prompt injection or changing the context.
- Company involved
- DeepSeek
- AI system involved
- DeepSeek-R1
5 source articles · read the reporting →
Fake AI audio of Sadiq Khan nearly caused disorder
A fake audio clip using AI to clone London Mayor Sadiq Khan's voice was shared online in November 2023, allegedly making inflammatory remarks about Remembrance Day. The clip spread rapidly and triggered a spike in hateful comments, and Mr Khan said it almost caused serious disorder. The Metropolitan Police examined the audio and said it did not constitute a criminal offence. The creator of the audio has not been identified.
8 source articles · read the reporting →
