Austrian court lifts ban on AMS job-chance prediction algorithm
The Austrian Federal Administrative Court overturned a ban by the Data Protection Authority on the AMS algorithm, which predicts job chances of unemployed people. The system, trained on historical data including age, gender, and nationality, categorises individuals as high, medium or low chance of re-employment. Critics argue it discriminates against women and mothers, and that the data is now outdated due to the COVID-19 pandemic. The court ruled that the algorithm is lawful as long as a human advisor makes the final decision on training support.
- Company involved
- Arbeitsmarktservice (AMS)
- AI system involved
- AMS-Algorithmus
5 source articles · read the reporting →
UK councils use Covid OneView AI to harvest personal data for risk scoring
UK local authorities are using a system called Covid OneView, developed by data analytics firm Xantura, to harvest millions of personal details from council records. The system uses predictive analytics and AI to assign risk scores to households and individuals, aiming to identify those vulnerable to Covid or likely to break lockdown rules. Privacy campaigners and MPs have criticised the lack of transparency and the extent of data collection, which includes sensitive information such as debt levels, living arrangements, and even notes on unfaithful sex. Xantura and Barking and Dagenham Council have defended the system as compliant with data protection rules and focused on providing support.
- Company involved
- UK local authorities
- AI system involved
- Covid OneView
6 source articles · read the reporting →
German Federal Constitutional Court restricts police automated data evaluation
The German Federal Constitutional Court ruled in a case brought by the Gesellschaft für Freiheitsrechte (GFF) that automated data evaluation by police violates fundamental rights. The court restricted the practice, setting limits on police use of automated data analysis. The decision is a success for privacy and civil liberties.
- Company involved
- German police
10 source articles · read the reporting →
Finnish recruitment company Digital Minds used AI to analyze job applicants' messages, prompting data protection investigation
Digital Minds, a Finnish recruitment company founded by psychologists, used IBM Watson AI to analyze job applicants' social media and email messages for personality assessments. The company obtained written consent but the Finnish Data Protection Ombudsman launched an investigation, suspecting violations of data protection laws and the secrecy of correspondence. The service was used on fewer than ten applicants and has been paused pending the investigation.
- Company involved
- Digital Minds
- AI system involved
- IBM Watson
9 source articles · read the reporting →
Attempted deepfake CEO voicemail fraud against an unnamed technology company
An attacker used synthetic audio deepfake to leave a voicemail impersonating the CEO of a technology company, asking an employee to call back to finalize an urgent business deal. The employee found it suspicious and referred the matter to the legal department, avoiding any loss. Nisos analyzed the audio and found inconsistencies in pitch and tone.
10 source articles · read the reporting →
Judge rules police search using Flock was mass surveillance
A judge ruled that a police search using Flock's automated license plate readers constituted a form of mass surveillance. The ruling concerns the deployment of the AI-based camera system by law enforcement, which the court found to be an invasive surveillance practice. No further details of the case were provided in the article.
- AI system involved
- Flock
4 source articles · read the reporting →
DeepScore markets facial and voice analysis app for trustworthiness scoring despite experts' doubts
DeepScore, a Tokyo-based company, is marketing an app that uses facial and voice recognition to score people's trustworthiness for lenders and insurers in Japan, Indonesia, Vietnam and the Philippines. The company says the app can detect deception with 70 per cent accuracy, but researchers and privacy advocates say there is no reliable scientific basis for such judgments and warn of discrimination and privacy harms. The chief executive said the system is only one part of lenders' and insurers' decision-making and that people can choose not to use it. Critics respond that an unequal balance of power makes consent difficult.
- Company involved
- DeepScore
- AI system involved
- DeepScore
6 source articles · read the reporting →
Lemonade backtracks on tweet claiming AI scans customer faces for fraud
Lemonade, an insurance firm, posted a tweet claiming it uses AI to detect non-verbal cues in customer videos to deny claims. After backlash comparing the practice to phrenology, the company deleted the tweet and denied using AI to deny claims based on facial characteristics. Lemonade stated it uses facial recognition only to flag claims submitted under different identities, which are then reviewed by human investigators.
- Company involved
- Lemonade
10 source articles · read the reporting →
Italian privacy regulator fines Foodinho €2.6 million for algorithmic discrimination against riders
The Italian Data Protection Authority (Garante) fined Foodinho S.r.l., a subsidiary of GlovoApp23, €2.6 million for privacy violations related to its algorithmic management of food delivery riders. The Garante found that the company's digital platform used algorithms to assign orders and rate riders without adequate transparency, human oversight, or the right to contest decisions. The system allegedly penalized riders who did not accept orders quickly, leading to reduced work opportunities and exclusion from the platform. The Garante ordered Foodinho to implement corrective measures within 60 days and to overhaul the algorithms within 90 days.
- Company involved
- Foodinho S.r.l.
10 source articles · read the reporting →
FTC settles with IntelliVision over deceptive facial recognition claims
The Federal Trade Commission (FTC) took action against IntelliVision Technologies Corp. for making false, misleading, or unsubstantiated claims about its AI-powered facial recognition software. The company allegedly claimed its software had one of the highest accuracy rates on the market and was free of gender or racial bias, without supporting evidence. The FTC also alleged that IntelliVision did not train its software on millions of faces as claimed, but on images of approximately 100,000 individuals. Under a proposed consent order, IntelliVision is prohibited from making such misrepresentations without competent and reliable testing.
- Company involved
- IntelliVision Technologies Corp.
- AI system involved
- IntelliVision facial recognition software
9 source articles · read the reporting →
ScaleFactor reportedly failed to deliver promised automated bookkeeping software
ScaleFactor, an Austin-based startup, is reported to have failed to deliver the automated, real-time bookkeeping tools it promised customers, instead relying on human bookkeepers and a Filipino contract accounting firm. The company told Forbes in June that it was shutting down, initially blaming the pandemic, but Forbes later reported that its problems predated Covid-19. Investors reportedly came to see the company as more of a services business than a software platform, and pulled funding after a pivot to a marketplace model. No legal or regulatory action is reported.
- Company involved
- ScaleFactor
10 source articles · read the reporting →
ElevenLabs AI voice generation used in Russian influence operation targeting Europe
The article reports that a Russian influence campaign, dubbed "Operation Undercut," very likely used ElevenLabs' AI voice generation technology to create realistic voiceovers for fake news videos. The videos targeted European audiences to undermine support for Ukraine. Recorded Future's researchers used ElevenLabs' own AI Speech Classifier to detect the AI-generated audio. The campaign was attributed to the Russia-based Social Design Agency, which the U.S. government sanctioned. The overall impact on public opinion was minimal.
- Company involved
- Social Design Agency
- AI system involved
- ElevenLabs AI voice generation
7 source articles · read the reporting →
DeepSeek exposed user data via open ClickHouse database
Cloud security firm Wiz discovered a ClickHouse database belonging to DeepSeek that was open to the internet without authentication, containing over a million lines of logs with chat histories, secret keys and backend details. Wiz disclosed the breach to DeepSeek, which promptly locked down the database. The incident highlights security risks in rapidly deploying AI services.
- Company involved
- DeepSeek
- AI system involved
- DeepSeek-R1
5 source articles · read the reporting →
Bavarian police test Palantir data mining with real personal data
The Bavarian State Criminal Police Office (LKA) has been testing Palantir's data mining software, called VeRa, with real personal data for months. The Bavarian data protection commissioner only learned of the test through a media inquiry and has announced a review. The Interior Ministry claims the test is lawful under current law, but critics argue a legal basis is missing.
- Company involved
- Bayerisches Landeskriminalamt
- AI system involved
- VeRa
7 source articles · read the reporting →
BBC News creates scam-writing ChatGPT bot using GPT Builder
BBC News used OpenAI's GPT Builder to create a custom AI assistant called Crafty Emails that could generate convincing scam and phishing messages. The bot bypassed the moderation of the public ChatGPT. OpenAI acknowledged the issue and said it is investigating how to make its systems more robust against such abuse. The test demonstrated a potential hazard for cyber-crime.
- Company involved
- OpenAI
- AI system involved
- GPT Builder
2 source articles · read the reporting →
Presto Automation uses off-site human agents to double-check AI drive-thru orders
Presto Automation Inc, which markets an AI voice assistant for drive-thru ordering, used off-site human agents in countries including the Philippines to double-check orders in more than 70% of customer interactions, according to SEC filings reported by Bloomberg. The company told Bloomberg that the process helps train its system and should reduce human intervention over time. Presto's drive-thru AI is used in more than 400 restaurants, including Del Taco, Carl's Jr and Checkers, and its stock fell more than 10% after the reports.
- Company involved
- Presto Automation Inc.
8 source articles · read the reporting →
N-Tech.lab's FindFace used to identify St Petersburg metro passengers without consent
Egor Tsvetkov photographed passengers on the St Petersburg metro without their permission and used N-Tech.lab's facial recognition service FindFace to match their faces to public Vkontakte profiles. He published the results in an art project called 'Your Face is Big Data', saying he wanted to show how 'digital narcissism' can lead to stalking. Privacy advocates said the project was ethically problematic because the subjects had not consented and their identities were exposed. FindFace had been launched by N-Tech.lab in February 2016.
- Company involved
- N-Tech.lab
- AI system involved
- FindFace
8 source articles · read the reporting →
OnlyFake site uses neural networks to generate fake IDs, bypasses OKX verification
An underground website called OnlyFake uses neural networks to generate realistic photos of fake IDs for $15. The journalist tested the service and obtained a convincing California driver's license. They then used another fake ID to successfully bypass the identity verification process on OKX, a cryptocurrency exchange. The article alleges that this technology could streamline bank fraud and money laundering, but reports no actual financial loss.
- Company involved
- OKX
- AI system involved
- OnlyFake
10 source articles · read the reporting →
OpenAI's Operator AI spent $31 on a dozen eggs for a journalist
Geoffrey A. Fowler, a Washington Post columnist, asked OpenAI's Operator AI agent to find cheap eggs in his neighborhood. Instead, the AI autonomously ordered a dozen eggs for $31 and had them delivered. The incident highlights the AI's inability to follow cost-saving instructions, resulting in a financial loss for the user.
- Company involved
- OpenAI
- AI system involved
- Operator
3 source articles · read the reporting →
Swedish welfare agency's AI system flags marginalized groups for fraud investigations
Försäkringskassan, Sweden's Social Insurance Agency, uses an AI risk-scoring system to flag welfare applicants for fraud investigations. The system disproportionately targets women, individuals with foreign backgrounds, low-income earners, and those without university degrees, according to an investigation by Lighthouse Reports and Svenska Dagbladet. Amnesty International has called for the system to be discontinued, citing violations of the right to equality and non-discrimination.
- Company involved
- Försäkringskassan (Swedish Social Insurance Agency)
6 source articles · read the reporting →
Italian privacy regulator investigates OpenAI's Sora video generation model
The Italian Data Protection Authority (Garante Privacy) has opened an investigation into OpenAI's new AI model 'Sora', which creates short videos from text instructions. The regulator has asked OpenAI to provide information on the algorithm's training, data sources, and compliance with European data protection regulations. OpenAI must respond within 20 days.
- Company involved
- OpenAI
- AI system involved
- Sora
7 source articles · read the reporting →
Queensland police trial AI to predict domestic violence risk
The Queensland Police Service is trialling an AI risk-assessment tool to identify high-risk domestic violence offenders from police records. Police then pre-emptively door-knock these individuals to deter violence. The author raises concerns about potential negative impacts, but police report a 56% reduction in incidents. The AI was developed in-house to increase transparency.
- Company involved
- Queensland Police Service
9 source articles · read the reporting →
CJEU rules Dun & Bradstreet must explain automated credit decisions under GDPR
A customer was refused a mobile phone contract because of an automated credit assessment by Dun & Bradstreet Austria. The customer took the case to court, which found that Dun & Bradstreet had infringed the GDPR by failing to provide meaningful information about the logic involved. The CJEU ruled that data controllers must explain automated decisions and that trade secrets cannot automatically override the right of access.
- Company involved
- Dun & Bradstreet Austria GmbH
7 source articles · read the reporting →
Klarna CEO faces backlash for using AI to halve marketing team
Klarna CEO Sebastian Siemiatkowski tweeted that the company saved $10 million by using generative AI to reduce its in-house marketing team to half its previous size and cut external agency spending by 25%. The post sparked criticism on social media, with commenters calling the remarks 'ghoulish' and highlighting concerns about AI's impact on jobs. Klarna previously laid off 700 workers in 2022 and reduced headcount by 23% by end of 2023.
- Company involved
- Klarna
- AI system involved
- Midjourney, DALL-E, Adobe Firefly
6 source articles · read the reporting →
