Where automated decisions went wrong

Presto Automation uses off-site human agents to double-check AI drive-thru orders

Presto Automation Inc, which markets an AI voice assistant for drive-thru ordering, used off-site human agents in countries including the Philippines to double-check orders in more than 70% of customer interactions, according to SEC filings reported by Bloomberg. The company told Bloomberg that the process helps train its system and should reduce human intervention over time. Presto's drive-thru AI is used in more than 400 restaurants, including Del Taco, Carl's Jr and Checkers, and its stock fell more than 10% after the reports.

Company involved

Presto Automation Inc.

8 source articles · read the reporting →

BC Tribunal Confirms Companies Remain Liable for AI Chatbot-Created Information - Lexology

The AI chatbot provided inaccurate information to a user.

1 source article · read the reporting →

Teleperformance deploys AI to neutralise Indian call centre agents' accents

Teleperformance, the world's largest call centre operator, has announced it is using AI from Sanas to modify the accents of its Indian employees in real time. The technology, called accent translation, aims to make agents sound more neutral to native English speakers. The company invested $13 million in Sanas and gained exclusive rights. No specific incident of harm has been reported.

Company involved

Teleperformance

AI system involved

Sanas AI

6 source articles · read the reporting →

Amazon France fined €32m for illegal worker surveillance

Amazon France Logistique was fined €32m (£27m) by France's data protection authority, the CNIL, for illegally monitoring warehouse workers through handheld scanners. The system tracked scanning speed, break durations, and stored data for 31 days, leading to workers potentially having to justify every brief interruption. The CNIL found the surveillance to be excessive and in breach of GDPR. Amazon disputes the findings and stated it may appeal.

Company involved

Amazon France Logistique

AI system involved

handheld scanner monitoring system

9 source articles · read the reporting →

Spanish police bust $20M AI-powered investment scam

Spanish law enforcement, collaborating with international authorities, dismantled a $20 million investment scam that used AI-driven algorithms to deceive individuals and organizations. Six suspects were detained and assets, including luxury cars and cryptocurrency, were seized. The article does not report any compensation for victims.

5 source articles · read the reporting →

Serco Leisure issued enforcement notices for unlawful biometric monitoring of employees

Serco Leisure and associated trusts used facial recognition and fingerprint scanning to monitor employee attendance at 38 leisure facilities. The ICO found they unlawfully processed biometric data of over 2,000 employees. Enforcement notices were issued ordering them to stop.

Company involved

Serco Leisure Operating Limited

8 source articles · read the reporting →

L'Observatoire de l'Europe uses AI to steal Euronews articles

Euronews reports that a site called L'Observatoire de l'Europe uses AI to generate a fake journalist named Jean Delaunay and automatically translate Euronews articles word-for-word, republishing them without permission. The site attributes all articles to the fake journalist. Euronews states that this happens daily and that the site will likely steal this article as well.

Company involved

L'Observatoire de l'Europe

6 source articles · read the reporting →

Klarna CEO faces backlash for using AI to halve marketing team

Klarna CEO Sebastian Siemiatkowski tweeted that the company saved $10 million by using generative AI to reduce its in-house marketing team to half its previous size and cut external agency spending by 25%. The post sparked criticism on social media, with commenters calling the remarks 'ghoulish' and highlighting concerns about AI's impact on jobs. Klarna previously laid off 700 workers in 2022 and reduced headcount by 23% by end of 2023.

Company involved

Klarna

AI system involved

Midjourney, DALL-E, Adobe Firefly

6 source articles · read the reporting →

French law legalizes algorithmic surveillance for 2024 Olympics

The French Parliament passed a law on March 23, 2023, that legalizes algorithmic video surveillance for the 2024 Olympic and Paralympic Games. Civil society organizations, including ECNL, La Quadrature du Net, and Amnesty International France, argue that the measures violate international human rights law and risk conflicting with the future EU AI Act. The law allows invasive surveillance under the pretext of securing big events, marking France as the first EU member state to explicitly legalize such practices.

Company involved

French government

AI system involved

Algorithmic video surveillance

10 source articles · read the reporting →

Lattice cancels plan to give AI digital workers employee records after backlash

Lattice, an HR software company, announced on July 9th that it would give AI digital workers official employee records. After strong backlash from HR professionals and others on LinkedIn, the company canceled the feature on July 12th, stating it 'will not further pursue digital workers in the product.' The feature was intended to manage AI bots such as Devin and Piper, but the company reversed course.

Company involved

Lattice

AI system involved

Lattice

6 source articles · read the reporting →

French police secretly used Briefcam facial recognition since 2015

The French National Police has been using Briefcam's Video Synopsis software since 2015, according to internal documents obtained by Disclose. The software, which includes facial recognition capabilities, was deployed without the required data protection impact assessment or notification to the CNIL. The Ministry of the Interior concealed the use of this tool for eight years. The police hierarchy allegedly used the software for facial recognition without judicial requisition, and the DGPN did not respond to requests for comment.

Company involved

French National Police

AI system involved

Briefcam Video Synopsis

10 source articles · read the reporting →

Irish DPC takes Twitter to court over using user data to train Grok AI

The Irish Data Protection Commission has initiated High Court proceedings against Twitter International Unlimited Company over concerns that the company is processing personal data of millions of European X users to train its Grok AI system without adequate consent. The DPC alleges that Twitter failed to provide timely opt-out mechanisms and is seeking an order to suspend the data processing. Twitter denies any wrongdoing.

Company involved

Twitter International Unlimited Company

AI system involved

Grok

10 source articles · read the reporting →

S Group food delivery robot crashes into car in Helsinki hit-and-run

On Saturday afternoon in Helsinki's Pasila neighbourhood, a passer-by observed an S Group food delivery robot collide with a parked car after it was forced off an unplowed sidewalk. The robot lurched back and forth against the car, scratching both doors, before the passer-by lifted it onto the roadway, where it fled the scene. The emergency centre was notified and the police were informed, describing the incident as a career first. S Group is currently investigating the event.

Company involved

S Group

AI system involved

Food delivery robot

3 source articles · read the reporting →

Starship delivery robots blocked wheelchair users at University of Pittsburgh

In October 2019, Starship Technologies' autonomous delivery robots blocked wheelchair users on the University of Pittsburgh campus. Doctoral student Emily Ackerman reported that a robot blocked the only accessible sidewalk entrance, trapping her near traffic. Another wheelchair user, Alisa Grishman, had a similar incident earlier that month. The university and Starship paused the robot program for review.

Company involved

Starship Technologies

AI system involved

Starship delivery robot

4 source articles · read the reporting →

DWP algorithm approved Kickstart gateways with no trading history or based abroad

An FE Week investigation found that the Department for Work and Pensions (DWP) approved dozens of companies as Kickstart gateways through automated due diligence checks using the Cabinet Office Spotlight Tool, although some had little or no trading history or were based abroad. The DWP said gateways were subject to stringent checks and later said human checks were also used. After the findings were shared with the Treasury and the DWP, the department stopped taking gateway applications and scrapped the requirement for small employers to use gateways from 3 February.

Company involved

Department for Work and Pensions

AI system involved

Cabinet Office Spotlight Tool

3 source articles · read the reporting →

Amazon cuts 14,000 corporate jobs in bet on AI

Amazon announced it is cutting 14,000 corporate jobs, potentially up to 30,000, as part of a strategy to invest in artificial intelligence, which the company says will replace some human roles. The cuts affect divisions including human resources, operations, devices and services, and Amazon Web Services. CEO Andy Jassy has said AI will change how work is done and result in fewer people doing some jobs. The company has not commented on the impact on its UK workforce.

Company involved

Amazon

5 source articles · read the reporting →

Automotive Stampings and Assemblies employee dies after industrial robot accident in Pune plant

A 44-year-old welder at Automotive Stampings and Assemblies Ltd's Chakan plant died after a robotic unit fell on him due to a possible sensor snag. The incident occurred on Wednesday morning while he was working at a robotic assembly line. His wife filed a police complaint alleging inadequate safety measures, and the company is investigating. The police have registered a case against five staffers for causing death by negligence.

Company involved

Automotive Stampings and Assemblies Ltd

1 source article · read the reporting →

Microsoft Dynamics 365 Field Service AI singles out workers in performance predictions

A report by Cracked Labs found that Microsoft's Dynamics 365 Field Service software uses AI to generate performance metrics and predict task durations, singling out individual workers. The AI predictions can be influenced by the worker's identity, such as increasing or decreasing estimated duration. Microsoft stated the system is not intended for employment decisions and is not a surveillance tool, but the report raises concerns about potential misuse for worker monitoring.

Company involved

Microsoft

AI system involved

Dynamics 365 Field Service

6 source articles · read the reporting →

Glovo driver died in crash, then received automated firing email

Sebastian Galassi, a 26-year-old Glovo delivery driver, died in a traffic collision in Florence, Italy, on 1 October 2022. The following day, his family received an automated email from Glovo stating his account had been deactivated for non-compliance. Glovo claimed the message was sent in error when suspending the account to protect his identity, and the company apologised and offered to pay some funeral costs.

Company involved

Glovo

AI system involved

Automated account deactivation system

2 source articles · read the reporting →

Zoox recalls 332 robotaxis after self-driving cars veer into oncoming traffic

Zoox, Amazon's self-driving car company, recalled 332 robotaxis after they were found veering into oncoming traffic near intersections. The issue was first noticed in August 2025 when a car took a wide right turn crossing the center line. Zoox said no crashes or injuries occurred and fixed the problem with a software update. The National Highway Traffic Safety Administration confirmed the recall and is in ongoing discussions with Zoox.

Company involved

Zoox

AI system involved

Zoox robotaxis

6 source articles · read the reporting →

Hollywood studios accuse ByteDance's Seedance 2.0 of mass copyright infringement

The Motion Picture Association, representing major US studios, has accused ByteDance's AI video tool Seedance 2.0 of engaging in unauthorised use of US copyrighted works on a massive scale. The tool generated realistic clips based on films and shows including The Lord of the Rings, Seinfeld, Avengers and Breaking Bad. ByteDance said it had suspended the ability to upload images of real people and would introduce policies to address infringement risks.

Company involved

ByteDance

AI system involved

Seedance 2.0

6 source articles · read the reporting →

PredictiveHire builds AI to predict job hopping from interviews

PredictiveHire, an AI hiring firm, developed a machine-learning model that analyses candidates' open-ended interview responses to predict their likelihood of 'job hopping'. The company used data from 45,899 applicants to build the 'flight risk' assessment, which it advertises as coming soon. Scholars warn that such tools can suppress wages by screening out workers who might seek better pay or conditions, continuing a historical trend of using personality tests to identify potential labour organisers.

Company involved

PredictiveHire

AI system involved

Phai

1 source article · read the reporting →

SEC charges YouPlus and CEO with defrauding investors

The SEC charged machine-learning startup YouPlus and its CEO Shaukat Shamim with defrauding investors. Shamim allegedly made false statements about the company's revenue and customer numbers, including providing falsified bank statements. The scheme unravelled when Shamim confessed to investors that the company had earned less than $500,000 and had only four paying customers since 2013. The SEC is seeking permanent injunctions, civil penalties, and an officer-and-director bar.

Company involved

YouPlus

AI system involved

YouPlus machine-learning tool

1 source article · read the reporting →

Facebook's automated ad system rejects adaptive fashion ads

Small adaptive fashion companies have had their advertisements repeatedly rejected by Facebook's automated advertising system, which misidentifies their products as medical devices. The system's inability to understand context leads to discrimination against people with disabilities. Facebook has acknowledged the issue and restored some ads, but the problem persists, highlighting biases in machine learning.

Company involved

Facebook

AI system involved

Facebook automated advertising review system

2 source articles · read the reporting →