Air Canada Chatbot Fabricates Discount, Leading to Court Case
Air Canada's customer service chatbot allegedly fabricated a discount during a conversation with a customer last year. The incident resulted in a court case against the airline. Insurer Armilla stated that its new AI mishap policy would have covered the loss from selling tickets at the discounted price if the chatbot was found to have underperformed.
- Company involved
- Air Canada
- AI system involved
- chatbot
6 source articles · read the reporting →
Beijing Subway to use facial recognition for passenger screening
The Beijing Subway plans to introduce security screenings that use an AI-powered camera system to divide passengers into groups based on facial recognition. The system is intended to single out people for different security measures, aiming to speed up subway traffic. The article does not report any specific incident of harm or complaint.
- Company involved
- Beijing Subway
- AI system involved
- AI-powered camera system
10 source articles · read the reporting →
AMS algorithm lacks transparency and may discriminate against job seekers
The Austrian Public Employment Service (AMS) uses an algorithm to classify job seekers into categories A, B, and C, determining their access to benefits and training. Scientists from TU Wien, WU Wien, and University of Vienna have criticised the algorithm for lacking transparency, as only two of 96 model variants have been published. They allege that the system may discriminate against women and people with migration background, and that job seekers are not informed about how the algorithm works or given a chance to appeal.
- Company involved
- AMS (Arbeitsmarktservice Österreich)
- AI system involved
- AMS-Algorithmus
10 source articles · read the reporting →
Southwest Airlines holiday meltdown due to crew scheduling software failure
In December 2022, Southwest Airlines experienced a catastrophic operational meltdown after its crew scheduling software failed to assign pilots and flight attendants to flights during a winter storm. The system, SkySolver and Crew Web Access, could not handle the volume of schedule changes, leading to over 15,800 flight cancellations and stranding thousands of passengers and crew. Southwest's CEO acknowledged the technology failure and promised upgrades, but the incident remained unresolved at the time of reporting.
- Company involved
- Southwest Airlines
- AI system involved
- SkySolver and Crew Web Access
10 source articles · read the reporting →
Austrian court lifts ban on AMS job-chance prediction algorithm
The Austrian Federal Administrative Court overturned a ban by the Data Protection Authority on the AMS algorithm, which predicts job chances of unemployed people. The system, trained on historical data including age, gender, and nationality, categorises individuals as high, medium or low chance of re-employment. Critics argue it discriminates against women and mothers, and that the data is now outdated due to the COVID-19 pandemic. The court ruled that the algorithm is lawful as long as a human advisor makes the final decision on training support.
- Company involved
- Arbeitsmarktservice (AMS)
- AI system involved
- AMS-Algorithmus
5 source articles · read the reporting →
Barclays pilot of Sapience monitoring software causes employee stress
Barclays introduced a pilot of employee monitoring software from Sapience in its product control department at Canary Wharf. The system monitors computer activity and admonishes staff if they are not deemed active enough, recording breaks as "unaccounted activity". Employees reported significant stress and worry about taking breaks. Barclays acknowledged the pilot and said it would listen to feedback.
- Company involved
- Barclays
- AI system involved
- Sapience employee monitoring software
10 source articles · read the reporting →
EU to trial AI lie detector at airports in Hungary, Latvia, Greece
The European Union is set to trial an AI-powered lie detector system called iBorderCtrl at airports in Hungary, Latvia and Greece. The system uses a virtual avatar to question passengers and monitors their facial expressions to detect deception. Privacy groups have raised concerns about bias and error rates, noting that the technology has only been tested on 32 people. The trial will require passenger consent and will be overseen by human guards.
- Company involved
- European Union (iBorderCtrl project)
- AI system involved
- iBorderCtrl
6 source articles · read the reporting →
Teleperformance plans AI webcam surveillance for home-working staff
Teleperformance, a global call centre company, told some staff it would install AI-powered webcams to monitor home-working infractions such as eating, phone use, or leaving desks. The system would randomly scan for breaches and send alerts to managers. After the Guardian inquired, the company said the remote scans would not be used in the UK, but the plan raised concerns from unions and MPs about invasive surveillance.
- Company involved
- Teleperformance
10 source articles · read the reporting →
ViaQuatro's facial recognition system in São Paulo metro challenged in court
In April 2018, ViaQuatro installed the Digital Interactive Doors System, developed by AdMobilize, on the São Paulo metro's yellow line. The system used cameras to detect passengers' faces and claimed to infer their emotion, age, and gender in order to target advertisements. The Brazilian Institute of Consumer Protection (IDEC) filed a public civil action alleging that the system violated consumer and data protection laws by processing biometric data without consent and making pseudoscientific and discriminatory inferences. A judge ordered the cameras removed in August 2018, and the case is pending a final ruling.
- Company involved
- ViaQuatro
- AI system involved
- Digital Interactive Doors System (DID system)
10 source articles · read the reporting →
USPS algorithm RRECS causes pay cuts for two-thirds of rural mail carriers
The United States Postal Service (USPS) implemented a new algorithm, RRECS, to evaluate rural carrier routes and determine pay. Due to flaws in how carriers scanned packages, the algorithm underestimated route times, resulting in pay cuts for 66 per cent of rural carriers, some losing thousands of dollars annually. Carriers report that they were not adequately trained on the system and that the cuts are scheduled to take effect, though they have been postponed multiple times. The USPS stated that the system is the result of a nationally negotiated agreement.
- Company involved
- United States Postal Service
- AI system involved
- RRECS
9 source articles · read the reporting →
Senators question privacy of facial recognition tablets in Uber and Lyft vehicles
U.S. Senators Amy Klobuchar and Richard Blumenthal sent a letter to Uber and Lyft demanding information about interactive tablets installed in vehicles that use facial recognition to target ads. The tablets, provided by company Alfi, use AI to identify passengers' race, gender, and approximate age to serve personalized content. The senators expressed concerns about data collection and privacy, while Alfi claims its automation respects privacy without tracking identifiable information. No specific harm has been reported, but the senators are seeking answers about how the technology is used.
- Company involved
- Uber and Lyft
- AI system involved
- Alfi interactive tablets with facial recognition AI
10 source articles · read the reporting →
Mercadona fined €2.5 million for facial recognition pilot
Mercadona, a Spanish supermarket chain, tested an early-detection system using facial recognition in 48 stores to identify people with judicial restraining orders. The system, which operated with court authorisation, notified police when such a person was detected. The Spanish data protection authority (AEPD) imposed a €2.5 million fine, which Mercadona paid, and the company has since removed the system citing legal uncertainty.
- Company involved
- Mercadona
- AI system involved
- Sistema de Detección Anticipada (Early Detection System)
10 source articles · read the reporting →
CBSA tested facial recognition on millions at Toronto Pearson Airport in 2016
In 2016, the Canada Border Services Agency (CBSA) secretly tested facial recognition technology on millions of travellers at Toronto Pearson Airport without their knowledge. The system, supplied by Face4 Systems, matched faces against a database of previously deported individuals. The CBSA stated that no one was deported as a result of the pilot, and the technology was removed after six months. Privacy advocates raised concerns about lack of consent, transparency, and potential racial bias.
- Company involved
- Canada Border Services Agency
- AI system involved
- Faces on the Move
8 source articles · read the reporting →
Company fires HR team after ATS auto-rejects manager's CV due to filtering error
A company's applicant tracking system (ATS) auto-rejected qualified candidates' resumes for three months because it was filtering for the outdated framework AngularJS instead of the required Angular framework. The manager discovered the flaw by submitting his own CV under a pseudonym and found it was rejected within seconds. After the manager reported the issue to upper management, the company investigated and dismissed half of its HR team. No legal action or regulatory involvement is reported.
4 source articles · read the reporting →
Met Police buys £3m retrospective facial recognition system
The Metropolitan Police Service (MPS) has awarded a £3 million, four-year contract to Northgate Public Services for a new retrospective facial-recognition (RFR) system to be deployed within three months. RFR processes biometric information from historic CCTV, social media, and other images to identify suspects and missing persons, operating retroactively unlike live facial recognition. The procurement was approved by the Mayor's Office for Policing and Crime in August 2021. The MPS states that human-in-the-loop decision-making will be used, but digital rights groups and a former biometrics commissioner have raised concerns about potential discrimination, overrepresentation of marginalised groups in watch lists, and lack of a legislative framework.
- Company involved
- Metropolitan Police Service
- AI system involved
- Retrospective facial-recognition software
9 source articles · read the reporting →
Tourists rescued after following ChatGPT route in Tatra mountains
Three Polish tourists used ChatGPT to plan a hiking route from Hala Gąsienicowa to Dolina Pięciu Stawów in the Tatra mountains. The AI recommended a difficult route through Przełęcz Krzyżne, which proved too dangerous in winter conditions with limited visibility and icy rocks. The tourists called the TOPR rescue service and were safely brought down. A TOPR rescuer advised against relying on ChatGPT or Google Maps for mountain route planning.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
2 source articles · read the reporting →
Presto Automation uses off-site human agents to double-check AI drive-thru orders
Presto Automation Inc, which markets an AI voice assistant for drive-thru ordering, used off-site human agents in countries including the Philippines to double-check orders in more than 70% of customer interactions, according to SEC filings reported by Bloomberg. The company told Bloomberg that the process helps train its system and should reduce human intervention over time. Presto's drive-thru AI is used in more than 400 restaurants, including Del Taco, Carl's Jr and Checkers, and its stock fell more than 10% after the reports.
- Company involved
- Presto Automation Inc.
8 source articles · read the reporting →
Teleperformance deploys AI to neutralise Indian call centre agents' accents
Teleperformance, the world's largest call centre operator, has announced it is using AI from Sanas to modify the accents of its Indian employees in real time. The technology, called accent translation, aims to make agents sound more neutral to native English speakers. The company invested $13 million in Sanas and gained exclusive rights. No specific incident of harm has been reported.
- Company involved
- Teleperformance
- AI system involved
- Sanas AI
6 source articles · read the reporting →
Air Canada liable for chatbot's misleading bereavement advice
Air Canada was found liable by the B.C. Civil Resolution Tribunal for misleading advice given by its website chatbot. The chatbot told a passenger they could retroactively claim a bereavement rate, but the airline later denied the claim. The tribunal ordered Air Canada to pay $812 in compensation, noting the airline's argument that the chatbot was a separate legal entity was 'remarkable'.
- Company involved
- Air Canada
- AI system involved
- Air Canada chatbot
10 source articles · read the reporting →
Italian DPA says Interior Ministry's Sari Real Time facial recognition lacks legal basis
The Garante per la protezione dei dati personali issued an opinion on Sari Real Time, a facial recognition system developed for the Italian Ministry of Interior. The system, yet to become operational, would compare live video footage of people in public areas with a watch-list and alert police operators. The authority found the planned biometric processing lacked a specific legal basis under Italian law and Directive (EU) 2016/680, and warned it could turn targeted surveillance into mass surveillance.
- Company involved
- Ministero dell'Interno – Dipartimento della pubblica sicurezza
- AI system involved
- Sari Real Time
10 source articles · read the reporting →
AEMPS withdraws AI medicines tool MeQA after detecting errors
AEMPS launched MeQA, an artificial intelligence tool for answering public questions about medicines, on 13 May 2025. Two days later it withdrew the tool after detecting that some responses contained errors. The agency said that most answers were correct but that the errors could affect patient safety, and that it would restore the service as soon as possible.
- Company involved
- Agencia Española de Medicamentos y Productos Sanitarios (AEMPS)
- AI system involved
- MeQA
4 source articles · read the reporting →
Manchester Arena's Evolv weapon scanners fail to detect some knives, report finds
ASM Global's use of Evolv Express AI weapon scanners at Manchester Arena has been questioned after a private report found the system failed to detect large knives in 42% of walkthroughs. The report, produced by NCS4 and obtained by IPVM, also suggested the scanners may miss some bombs and components. Evolv did not dispute the findings but said it communicates capabilities and limitations to customers. ASM Global declined to comment on security matters.
- Company involved
- ASM Global
- AI system involved
- Evolv Express
5 source articles · read the reporting →
Network Rail denies using AI cameras to detect passengers' emotions
Network Rail conducted a trial of AI cameras at major stations in 2022 that analysed demographic details and reportedly assessed emotions. Documents obtained by Big Brother Watch indicate the system was capable of determining whether a passenger was happy, sad or angry, with potential use for measuring satisfaction and advertising. Network Rail denied that any emotion analysis took place and stated the image analysis for demographic details has ended. Big Brother Watch has submitted a complaint to the Information Commissioner about the trial.
- Company involved
- Network Rail
- AI system involved
- Amazon Rekognition
7 source articles · read the reporting →
Parking Enforcement Services wrongly fines parents due to faulty licence plate cameras
Dozens of parents at a Christchurch childcare centre were wrongly issued $85 parking fines by Parking Enforcement Services after its licence plate recognition cameras failed to accurately capture multiple short visits. The company acknowledged some misreads and waived fines on appeal, but parents described the process as stressful and time-consuming. An additional camera was installed to improve accuracy.
- Company involved
- Parking Enforcement Services
1 source article · read the reporting →
