Indore resident defrauded of Rs 1.83 lakh via AI voice cloning scam
A resident of Indore, India, was defrauded of Rs 1.83 lakh after scammers used AI-generated voice cloning to impersonate his brother-in-law in Australia. The fraudsters claimed the relative's visa had expired and threatened jail time unless fees were paid, prompting the victim to transfer the money. The victim discovered the fraud the next day upon contacting his relative directly and filed a complaint with the Indore crime branch's cyber cell. Police are investigating and have frozen suspicious accounts in an attempt to recover the funds.
1 source article · read the reporting →
Gmail users warned of AI voice phishing scam impersonating Google support
Cybercriminals are using AI-generated voices to impersonate Google support in phone calls to Gmail users, attempting to trick them into revealing account credentials. The scam involves a caller ID that appears legitimate and a follow-up email from a spoofed Google address. Victims are told their account has been compromised and are asked to provide a recovery code. Google has advised users to enable Advanced Protection to secure their accounts.
8 source articles · read the reporting →
Frustrated by Airline Hotline's Soulless AI Chatbot Responses
Điên tiết vì gọi hotline hãng hàng không chỉ thấy chatbot AI trả lời vô hồn - VnExpress
A customer called an airline hotline for urgent help but only reached an automated AI voice system. The chatbot on the website also failed to understand the issue and gave irrelevant answers. The customer felt blocked from speaking to a real person and questioned the overuse of AI in customer service.
1 source article · read the reporting →
Data Breach Exposes Over 10 Million Conversations from Middle Eastern AI Call Center Platform
Resecurity discovered a dark web posting on 8 October 2024 offering data stolen from a major AI-powered cloud call center platform in the Middle East. The threat actor gained unauthorized access to the management dashboard, compromising over 10,210,800 conversations between consumers, operators, and AI chatbots. The exposed data included personally identifiable information and national ID documents, creating risks of fraud and identity theft. Resecurity alerted the affected organization and collaborated with law enforcement to mitigate the incident.
3 source articles · read the reporting →
Resume prompt injection tricks AI hiring - moneywise.com
AI screening system determined which job applicants to advance to the next stage of recruitment.
1 source article · read the reporting →
Retool Breached After Hacker Uses AI Deepfake Voice in Phishing Call
A hacker used AI to deepfake an employee's voice and trick a Retool staff member into providing a multi-factor authentication code. The attacker sent phishing SMS messages and then called the employee, impersonating an IT team member with a synthetic voice. This allowed the hacker to add their own device to the employee's account and access internal systems, compromising 27 cloud customers. Retool revoked the access and disclosed the incident, blaming a weakness in Google Authenticator's cloud sync feature.
- Company involved
- Retool
1 source article · read the reporting →
Washington licensing department's Spanish line gave English recording with accent
Maya Edwards and her husband called the Washington State Department of Licensing to speak to someone in Spanish. When they selected the Spanish-language self-service option, they heard an automated voice speaking English with a Hispanic accent instead of Spanish. After the issue persisted for months, Edwards posted a video on TikTok that went viral. The department acknowledged a technical issue with the automated voiceover system and said it was working on a fix.
- Company involved
- Washington Department of Licensing
5 source articles · read the reporting →
Services Australia voiceprint system fooled by AI voice clone
A Guardian Australia investigation found that the voiceprint system used by Services Australia's Centrelink and the Australian Taxation Office can be bypassed using an AI-generated voice clone. A journalist created a clone of their own voice from four minutes of audio and used it with a customer reference number to access their Centrelink self-service account. The system is used by millions of Australians for identity verification over the phone. Services Australia stated that it continually assesses risks and applies additional tests if unusual circumstances are detected, but did not commit to changing the technology.
- Company involved
- Services Australia
- AI system involved
- Voiceprint
1 source article · read the reporting →
Voice Actors Doxed and Harassed with AI-Generated Voices on Twitter
In February 2023, several video game voice actors were targeted by online trolls who used AI voice generation technology to create synthetic replicas of their voices. The fake voices were used to read out the actors' home addresses and offensive statements, which were then posted on Twitter. The actors reported the posts to Twitter, but the platform's support system initially deemed some of the content acceptable. ElevenLabs, the AI voice company allegedly used, denied that the doxing clips were created with its software and stated it had introduced additional safeguards to prevent misuse.
- Company involved
- ElevenLabs
1 source article · read the reporting →
Journalist Bypasses Lloyds Bank Voice ID with AI-Generated Voice Clone
A journalist used an AI-generated clone of his own voice to bypass the voice authentication system of Lloyds Bank, gaining access to his account. The experiment, conducted using ElevenLabs' free voice synthesis service, demonstrated that voice biometrics can be fooled by synthetic voices. Lloyds Bank stated it is aware of the threat and is deploying countermeasures, but has not seen real-world fraud using this method. The incident raises concerns about the security of voice verification used by many banks.
- Company involved
- Lloyds Bank
- AI system involved
- Voice ID
1 source article · read the reporting →
AI voice bots used for telemarketing in China raise privacy concerns
In China, companies use AI-powered voice bots to cold-call consumers, often without disclosing that the caller is a bot. The bots can respond flexibly and sometimes know the recipient's name, leading to privacy concerns. The practice has generated numerous complaints to the Ministry of Industry and Information Technology, and lawyers say it violates China's privacy laws.
8 source articles · read the reporting →
Criminal gang used AI voice cloning of Bolivian Education Minister to defraud job seekers
A criminal gang in Bolivia used artificial intelligence to clone the voice of Education Minister Omar Veliz, then called victims offering jobs in the ministry in exchange for an initial payment of 3,500 bolivianos. At least 19 people were defrauded, with the gang collecting around 5 million bolivianos. Police arrested four people, including one who operated the voice cloning from prison. The accused face charges of fraud, concusión, and other crimes.
4 source articles · read the reporting →
Air Canada Chatbot Fabricates Discount, Leading to Court Case
Air Canada's customer service chatbot allegedly fabricated a discount during a conversation with a customer last year. The incident resulted in a court case against the airline. Insurer Armilla stated that its new AI mishap policy would have covered the loss from selling tickets at the discounted price if the chatbot was found to have underperformed.
- Company involved
- Air Canada
- AI system involved
- chatbot
6 source articles · read the reporting →
Companies face AI deepfake job candidates for remote roles
US companies report a surge in fake job seekers using generative AI tools to fabricate identities, employment histories, and conduct deepfake video interviews for remote positions. Cybersecurity firms Pindrop and CAT Labs, along with BrightHire, describe incidents where scammers, including North Korean operatives, attempted to gain employment to install malware, demand ransoms, steal data, or collect salaries fraudulently. One candidate, 'Ivan X', was detected by Pindrop's video authentication tool after a recruiter noticed his facial expressions were out of sync with his words.
- Company involved
- Pindrop Security
- AI system involved
- video authentication program
1 source article · read the reporting →
Facebook hired contractors to transcribe Messenger voice calls
Facebook hired contractors to listen to and transcribe select voice conversations made through its Messenger platform, without users' knowledge, in order to improve artificial intelligence. The practice was exposed by Bloomberg, prompting Facebook to pause the program. The Irish Data Protection Commission stated it was seeking details from Facebook on compliance with GDPR.
- Company involved
- Facebook
- AI system involved
- Messenger
10 source articles · read the reporting →
Fraudsters use AI voice deepfake to trick UK energy firm CEO into transferring $243,000
In March 2019, criminals used commercially available AI voice-generation software to impersonate the CEO of a German parent company. They tricked the CEO of a UK-based energy firm into urgently wiring $243,000 to a Hungarian supplier. The fraud was discovered when the fraudsters attempted a second transfer, which the CEO refused. The company was insured and the loss was covered.
- Company involved
- Unnamed UK-based energy firm
- AI system involved
- Commercially available voice-generating AI software
10 source articles · read the reporting →
UIUC researchers use OpenAI API to automate phone scams for under a dollar
Researchers at the University of Illinois Urbana-Champaign used OpenAI's Realtime API to create AI agents that can autonomously execute phone scams. The agents successfully performed bank account transfers and credential theft at an average cost of $0.75 per scam. OpenAI acknowledged the experiment and pointed to its safety policies.
- Company involved
- University of Illinois Urbana-Champaign
- AI system involved
- GPT-4o Realtime API
6 source articles · read the reporting →
Attempted deepfake CEO voicemail fraud against an unnamed technology company
An attacker used synthetic audio deepfake to leave a voicemail impersonating the CEO of a technology company, asking an employee to call back to finalize an urgent business deal. The employee found it suspicious and referred the matter to the legal department, avoiding any loss. Nisos analyzed the audio and found inconsistencies in pitch and tone.
10 source articles · read the reporting →
Teleperformance plans AI webcam surveillance for home-working staff
Teleperformance, a global call centre company, told some staff it would install AI-powered webcams to monitor home-working infractions such as eating, phone use, or leaving desks. The system would randomly scan for breaches and send alerts to managers. After the Guardian inquired, the company said the remote scans would not be used in the UK, but the plan raised concerns from unions and MPs about invasive surveillance.
- Company involved
- Teleperformance
10 source articles · read the reporting →
ElevenLabs AI voice generation used in Russian influence operation targeting Europe
The article reports that a Russian influence campaign, dubbed "Operation Undercut," very likely used ElevenLabs' AI voice generation technology to create realistic voiceovers for fake news videos. The videos targeted European audiences to undermine support for Ukraine. Recorded Future's researchers used ElevenLabs' own AI Speech Classifier to detect the AI-generated audio. The campaign was attributed to the Russia-based Social Design Agency, which the U.S. government sanctioned. The overall impact on public opinion was minimal.
- Company involved
- Social Design Agency
- AI system involved
- ElevenLabs AI voice generation
7 source articles · read the reporting →
Presto Automation uses off-site human agents to double-check AI drive-thru orders
Presto Automation Inc, which markets an AI voice assistant for drive-thru ordering, used off-site human agents in countries including the Philippines to double-check orders in more than 70% of customer interactions, according to SEC filings reported by Bloomberg. The company told Bloomberg that the process helps train its system and should reduce human intervention over time. Presto's drive-thru AI is used in more than 400 restaurants, including Del Taco, Carl's Jr and Checkers, and its stock fell more than 10% after the reports.
- Company involved
- Presto Automation Inc.
8 source articles · read the reporting →
Teleperformance deploys AI to neutralise Indian call centre agents' accents
Teleperformance, the world's largest call centre operator, has announced it is using AI from Sanas to modify the accents of its Indian employees in real time. The technology, called accent translation, aims to make agents sound more neutral to native English speakers. The company invested $13 million in Sanas and gained exclusive rights. No specific incident of harm has been reported.
- Company involved
- Teleperformance
- AI system involved
- Sanas AI
6 source articles · read the reporting →
Voice actors sue LOVO over alleged theft of voices for AI training
Voice actors Paul Skye Lehrman and Linnea Sage filed a proposed class action against AI startup LOVO, alleging the company misappropriated their voices to train its text-to-speech system Genny. The actors claim they were hired on Fiverr under the pretence of academic research but later discovered their voices were used commercially without consent. Lehrman reports a 50% decline in work and loss of control over how his voice is used. The lawsuit, filed in New York federal court, seeks to represent other affected voiceover artists and obtain a court order blocking the practice.
- Company involved
- LOVO
- AI system involved
- Genny
6 source articles · read the reporting →
Mumbai businessman loses Rs 80,000 in AI voice cloning scam
A 68-year-old businessman from Powai, Mumbai, was defrauded of Rs 80,000 after receiving a call from scammers posing as the Indian Embassy in Dubai. The fraudsters used AI voice cloning to mimic his son's voice, claiming he was arrested and needed bail money. The victim transferred the money via GPay before realising it was a scam when the call was disconnected. The Kanjurmarg police have registered a case and are investigating.
2 source articles · read the reporting →
