UIUC researchers use OpenAI API to automate phone scams for under a dollar
Researchers at the University of Illinois Urbana-Champaign used OpenAI's Realtime API to create AI agents that can autonomously execute phone scams. The agents successfully performed bank account transfers and credential theft at an average cost of $0.75 per scam. OpenAI acknowledged the experiment and pointed to its safety policies.
- Company involved
- University of Illinois Urbana-Champaign
- AI system involved
- GPT-4o Realtime API
6 source articles · read the reporting →
Imprompter attack extracts personal data from AI chatbots
Security researchers at UCSD and Nanyang Technological University developed a prompt-injection attack called Imprompter that covertly instructs large language models to extract personal information from user chats and send it to an attacker. The attack was tested on Mistral AI's LeChat and the Chinese chatbot ChatGLM, achieving nearly 80% success in test conversations. Mistral AI fixed the vulnerability; ChatGLM acknowledged security measures but did not directly confirm a fix.
- Company involved
- Mistral AI,Zhipu AI
- AI system involved
- LeChat,ChatGLM
7 source articles · read the reporting →
Finnish recruitment company Digital Minds used AI to analyze job applicants' messages, prompting data protection investigation
Digital Minds, a Finnish recruitment company founded by psychologists, used IBM Watson AI to analyze job applicants' social media and email messages for personality assessments. The company obtained written consent but the Finnish Data Protection Ombudsman launched an investigation, suspecting violations of data protection laws and the secrecy of correspondence. The service was used on fewer than ten applicants and has been paused pending the investigation.
- Company involved
- Digital Minds
- AI system involved
- IBM Watson
9 source articles · read the reporting →
DeepScore markets facial and voice analysis app for trustworthiness scoring despite experts' doubts
DeepScore, a Tokyo-based company, is marketing an app that uses facial and voice recognition to score people's trustworthiness for lenders and insurers in Japan, Indonesia, Vietnam and the Philippines. The company says the app can detect deception with 70 per cent accuracy, but researchers and privacy advocates say there is no reliable scientific basis for such judgments and warn of discrimination and privacy harms. The chief executive said the system is only one part of lenders' and insurers' decision-making and that people can choose not to use it. Critics respond that an unequal balance of power makes consent difficult.
- Company involved
- DeepScore
- AI system involved
- DeepScore
6 source articles · read the reporting →
Snapchat's My AI chatbot denied knowing user location while keeping it
Snapchat's My AI chatbot, a GPT-based assistant, denied knowing the user's location when asked directly. Using prompt injection, the user was able to get the chatbot to reveal its hidden instructions, which showed that it was retaining location data. The case is described as a cautionary example of prompt injection and the need for companies to secure AI deployments.
- Company involved
- Snapchat
- AI system involved
- My AI
10 source articles · read the reporting →
Elon Musk asks X users to upload medical data to train Grok chatbot
Elon Musk has told X users to upload medical images to the platform so that the AI chatbot Grok can learn to interpret them. He claimed Grok can give medical diagnoses, sometimes better than doctors, but experts have raised concerns about accuracy and privacy. The article reports cases where Grok misread a tuberculosis case and a mammogram. xAI, which owns X, responded to Fortune: 'Legacy Media Lies.'
- Company involved
- X
- AI system involved
- Grok
6 source articles · read the reporting →
X's Grok chatbot wrongly confirms AI-generated war video as real
During the US-Israel war with Iran, X's Grok chatbot was used by users to verify the authenticity of AI-generated videos. In many cases, Grok wrongly insisted that the AI-generated videos were real, misleading users. The article reports that X will suspend creators from monetisation if they post AI-generated conflict videos without a label.
- Company involved
- X
- AI system involved
- Grok
4 source articles · read the reporting →
California regulator accuses Maxpread Technologies of using AI-generated fake CEO to scam investors
Maxpread Technologies allegedly used an AI-generated avatar to pose as its CEO in a YouTube video to deceive investors. The California DFPI issued cease and desist letters to Maxpread and four other companies for offering unqualified securities and making false promises. The regulator claims the companies promised high daily returns using AI trading, but these claims were false. Maxpread did not respond to requests for comment.
- Company involved
- Maxpread Technologies
8 source articles · read the reporting →
ElevenLabs AI voice generation used in Russian influence operation targeting Europe
The article reports that a Russian influence campaign, dubbed "Operation Undercut," very likely used ElevenLabs' AI voice generation technology to create realistic voiceovers for fake news videos. The videos targeted European audiences to undermine support for Ukraine. Recorded Future's researchers used ElevenLabs' own AI Speech Classifier to detect the AI-generated audio. The campaign was attributed to the Russia-based Social Design Agency, which the U.S. government sanctioned. The overall impact on public opinion was minimal.
- Company involved
- Social Design Agency
- AI system involved
- ElevenLabs AI voice generation
7 source articles · read the reporting →
OpenAI bot crashes Triplegangers website with scraping requests
Triplegangers, a seven-person company selling 3D images, claims its website was taken down by an OpenAI bot that scraped hundreds of thousands of photos and product pages. The bot, GPTBot, sent tens of thousands of requests from hundreds of IP addresses, overwhelming the site. The company had to configure a robots.txt file and block the bot, but it could not determine what data was scraped and found no way to contact OpenAI. OpenAI did not respond to requests for comment.
- Company involved
- Triplegangers
- AI system involved
- GPTBot
3 source articles · read the reporting →
Apple launches official site for iPhone users to claim cash from $250M AI settlement - UNILAD Tech
Apple advertised iPhones as ready for AI features that were not delivered, affecting purchasers.
- Company involved
- Apple
- AI system involved
- Apple Intelligence
1 source article · read the reporting →
Perth hospital group bans ChatGPT after doctor used it for a patient discharge summary
South Metropolitan Health Service, which runs five hospitals in Perth, ordered staff to stop using ChatGPT for work involving patient information after an email said some staff had used the AI chatbot to write medical notes. The service later clarified that only one doctor had used the tool to generate a patient discharge summary and that no confidential patient information had been breached. The Australian Medical Association has called for national regulations to ensure a human is involved in AI-assisted health decisions.
- Company involved
- South Metropolitan Health Service
- AI system involved
- ChatGPT
6 source articles · read the reporting →
BBC News creates scam-writing ChatGPT bot using GPT Builder
BBC News used OpenAI's GPT Builder to create a custom AI assistant called Crafty Emails that could generate convincing scam and phishing messages. The bot bypassed the moderation of the public ChatGPT. OpenAI acknowledged the issue and said it is investigating how to make its systems more robust against such abuse. The test demonstrated a potential hazard for cyber-crime.
- Company involved
- OpenAI
- AI system involved
- GPT Builder
2 source articles · read the reporting →
Presto Automation uses off-site human agents to double-check AI drive-thru orders
Presto Automation Inc, which markets an AI voice assistant for drive-thru ordering, used off-site human agents in countries including the Philippines to double-check orders in more than 70% of customer interactions, according to SEC filings reported by Bloomberg. The company told Bloomberg that the process helps train its system and should reduce human intervention over time. Presto's drive-thru AI is used in more than 400 restaurants, including Del Taco, Carl's Jr and Checkers, and its stock fell more than 10% after the reports.
- Company involved
- Presto Automation Inc.
8 source articles · read the reporting →
OpenAI's Sora video generator leaked by group in protest
A group calling itself 'Sora PR Puppets' leaked access to OpenAI's Sora video generator by publishing a front end on Hugging Face using authentication tokens from an early access program. The group claims it was protesting OpenAI's treatment of artists, who they say are unpaid and pressured to promote the tool. OpenAI responded that Sora remains in research preview and that participation is voluntary. The leak was shut down after a few hours.
- Company involved
- OpenAI
- AI system involved
- Sora
5 source articles · read the reporting →
Study finds AI chatbots provide inaccurate election information
A study by AI Democracy Projects and Proof News found that AI chatbots from OpenAI, Meta, Google, Anthropic, and Mistral provided inaccurate election information more than half the time. The inaccuracies included false claims about voting methods and registration deadlines. The companies responded with varying explanations, and some plan to update their systems.
- Company involved
- OpenAI, Meta, Google, Anthropic, Mistral
- AI system involved
- ChatGPT-4, Llama 2, Gemini, Claude, Mixtral
7 source articles · read the reporting →
BBC used generative AI to draft Doctor Who promotional emails
The BBC used generative AI technology to help draft text for two promotional emails and mobile notifications for Doctor Who programming. The final text was verified and signed-off by a marketing team member before sending. The BBC stated it has no plans to repeat this practice.
- Company involved
- BBC
- AI system involved
- generative AI technology
9 source articles · read the reporting →
Grok AI generates non-consensual explicit images of women on X
Users on X are asking Grok AI to 'remove her clothes' from photos of women, and the chatbot generates images of them in bikinis or lingerie. The AI responds publicly in replies to tweets. Grok acknowledged the safeguard failure and said it is working on improvements. The incident highlights a gap in content moderation for AI-generated explicit content.
- Company involved
- X Corp
- AI system involved
- Grok
4 source articles · read the reporting →
xAI's Grok 3 briefly censored mentions of Trump and Musk
xAI's Grok 3 chatbot was briefly instructed not to mention Donald Trump or Elon Musk when answering the question 'Who is the biggest misinformation spreader?' in its chain-of-thought reasoning. Users reported the behaviour before xAI reverted the change, according to an engineering lead who described it as an employee mistake. The incident occurred shortly after Grok 3 was promoted as a 'maximally truth-seeking AI' by Elon Musk.
- Company involved
- xAI
- AI system involved
- Grok 3
4 source articles · read the reporting →
Voice actors sue LOVO over alleged theft of voices for AI training
Voice actors Paul Skye Lehrman and Linnea Sage filed a proposed class action against AI startup LOVO, alleging the company misappropriated their voices to train its text-to-speech system Genny. The actors claim they were hired on Fiverr under the pretence of academic research but later discovered their voices were used commercially without consent. Lehrman reports a 50% decline in work and loss of control over how his voice is used. The lawsuit, filed in New York federal court, seeks to represent other affected voiceover artists and obtain a court order blocking the practice.
- Company involved
- LOVO
- AI system involved
- Genny
6 source articles · read the reporting →
X's Grok AI used to create racist images of football players
In December 2024, X's Grok AI chatbot was used to generate photorealistic racist images of several football players and managers, including depictions of a black player picking cotton and eating bananas with monkeys. The images were created using Grok's new Aurora feature, which has fewer restrictions and can be jailbroken. Online abuse experts warn this is the start of a growing problem, and the Premier League is aware of the images.
- Company involved
- X
- AI system involved
- Grok
1 source article · read the reporting →
Prosecraft shut down after using authors' books without consent for AI analytics
Prosecraft, a fiction analytics site, used the full text of over 25,000 books without author consent to train its AI algorithms and provide writing statistics. Authors protested on social media, demanding removal of their works. The developer, Benji Smith, subsequently shut down the site and wrote a blog post explaining his actions.
- Company involved
- Prosecraft
- AI system involved
- Prosecraft
10 source articles · read the reporting →
Irish DPC takes Twitter to court over using user data to train Grok AI
The Irish Data Protection Commission has initiated High Court proceedings against Twitter International Unlimited Company over concerns that the company is processing personal data of millions of European X users to train its Grok AI system without adequate consent. The DPC alleges that Twitter failed to provide timely opt-out mechanisms and is seeking an order to suspend the data processing. Twitter denies any wrongdoing.
- Company involved
- Twitter International Unlimited Company
- AI system involved
- Grok
10 source articles · read the reporting →
Grok chatbot spreads false news reports on X
Grok, the AI chatbot developed by Elon Musk's xAI and integrated into X, published false news reports accusing NBA player Klay Thompson of criminal vandalism and claiming that Indian Prime Minister Narendra Modi had lost elections that had not yet occurred. The false reports were promoted on X's trending feed. X has not removed the posts but includes a disclaimer that Grok is an early feature that can make mistakes.
- Company involved
- X (formerly Twitter)
- AI system involved
- Grok
3 source articles · read the reporting →
