Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.
Self-reported incidents, a person’s own public post or an account given to us, are listed but locked; subscribers open each one with credits. Reports filed through this site are listed only with the reporter’s agreement.
Press reportingSelf-reported: a person’s own public post · 10 creditsSelf-reported: told to us directly · 25 credits
Five Australian schools are trialling a facial recognition system from LoopLearn to automate student attendance. The system scans students' faces and sends attendance data to an app. Privacy experts have raised concerns about data security and surveillance. The Victorian government previously tried to stop the trial due to privacy risks.
The Post Office's Horizon IT system, supplied by Fujitsu, falsely reported financial shortfalls at many branches across England and Wales. Subpostmasters like Pete Murray were wrongly accused of theft and prosecuted, leading to loss of livelihood, reputational damage, and in some cases suicide. A group litigation, Bates v Post Office, settled, but many victims are still seeking compensation and justice. The scandal remains under investigation by the Post Office Horizon IT Inquiry.
Niulanshan First Secondary School in Beijing has installed Hanwang Technology's Classroom Care System, which photographs students every second and analyses their facial expressions and behaviour. The system sends weekly reports to teachers and parents, and one student was described as low participation in English class despite being recorded as focused 94% of the time. The school has been highlighted in a report by Article 19, which says such emotion recognition tools rely on junk science and risk eroding privacy and human rights.
Researchers at Keele University trained an AI to classify behaviors of children with autism by scraping YouTube videos uploaded by parents and advocacy groups. The children and parents did not consent to their videos being used for research. The researchers stated that the software does not identify individuals but only reads body language. The study raises privacy concerns.
The Austrian Public Employment Service (AMS) uses an algorithm to classify job seekers into categories A, B, and C, determining their access to benefits and training. Scientists from TU Wien, WU Wien, and University of Vienna have criticised the algorithm for lacking transparency, as only two of 96 model variants have been published. They allege that the system may discriminate against women and people with migration background, and that job seekers are not informed about how the algorithm works or given a chance to appeal.
A university in eastern China has installed a facial recognition system at its entrance and in two classrooms to monitor student attendance and behaviour. The system automatically captures students' faces without their cooperation. The pilot project is intended to improve attendance rates. No specific incidents of harm have been reported.
In May 2015, Poland's Ministry of Labor and Social Policy introduced an algorithmic system to profile unemployed people into three categories based on their readiness to work, location, and disabilities. The system is accused of being discriminatory and lacking transparency, as it uses default criteria that may lead to social exclusion. A report by the Panoptykon Foundation in October 2015 highlighted these issues and recommended reforms, including granting the right to appeal.
The Evangelical Church in Hesse and Nassau launched a robot priest named BlessU-2 in Wittenberg, Germany, to give automated blessings to visitors. The robot, a metal box with a touch screen and arms, asks users if they want a blessing and then recites a biblical verse. Church spokesman Sebastian von Gehren said it is an experiment to inspire discussion and that the machine should not replace a pastor's blessing.
Robert Nelson, president of Nelson Management Group, planned to install a facial recognition system at Atlantic Plaza Towers, a rent-stabilized building in Brownsville, Brooklyn. Tenants protested, citing privacy concerns and studies showing the technology disproportionately impacts people of color. After a town hall meeting, Nelson announced he was withdrawing the application with the state housing agency. The reversal was seen as a victory by tenants, who continue to push for legislation banning facial recognition in residential buildings.
Joshua Bada, a 28-year-old black man from west London, attempted to renew his passport online using the UK government's automated photo checker. The system falsely flagged his photo, stating that his mouth appeared open, because it mistook his lips for an open mouth. Bada was forced to explain in a comment box that his mouth was closed and he had big lips. The Home Office acknowledged the issue and said it would work to improve the system.
A University of Washington study found that Google Search image results for 'CEO' predominantly show white men, with the first woman appearing being a Barbie doll from a satirical article. The study claims that such search results can influence people's perceptions about gender representation in jobs. The article notes that the bias reflects societal stereotypes fed into the system by users.
The Chinese government's social credit system blocked 17.5 million people from buying plane tickets and 5.5 million from train tickets in 2018 as punishment for unpaid fines and other offenses. 128 people were also barred from leaving China due to unpaid taxes. The system, which the ruling Communist Party says improves public behavior, operates automatically and has been criticized by human rights activists and U.S. Vice President Mike Pence as being too rigid and Orwellian.
YouTube acknowledged that an error in its automated comment moderation system caused comments containing phrases critical of the Chinese government to be automatically deleted. The company stated that the auto-deletions were an unintended side effect of systems designed to filter hate speech and harassment. Users reported the issue, and YouTube said it was investigating.
Chinese technology companies CloudWalk and Huawei have supplied facial recognition systems to the governments of Zimbabwe and Uganda for law enforcement and administration. The systems are alleged to be inaccurate for dark-skinned individuals, potentially leading to false positives and unwarranted surveillance. Digital rights advocates warn that these systems could exacerbate civil liberties abuses in countries with weak legal protections. The use of these technologies is part of China's growing geopolitical influence and corporate expansion in Africa.
Chinese police departments in multiple provinces have deployed facial recognition systems from AI companies such as Yitu, Megvii, SenseTime, and CloudWalk to identify and track Uyghur people, a Muslim minority. The systems scan crowds and trigger alerts when a Uyghur individual is detected, enabling targeted surveillance across cities including Hangzhou, Wenzhou, and Sanmenxia. Documents and interviews show that one system in Sanmenxia conducted over 500,000 screenings in a month, flagging thousands as Uyghur. Police use this technology to monitor and control the Uyghur population, who have been subjected to mass detention and other repressive measures. The article reports these practices from interviews with anonymous sources and procurement documents.
Motka Manjhi, a low-income villager in Jharkhand, India, was repeatedly denied subsidised food rations because his thumbprint failed to register on the Aadhaar biometric system. Unable to travel to a centre to update his data, he went without food and died of starvation on 22 May. His family believes the denial of rations caused his death. Activists allege that similar Aadhaar glitches have led to other starvation deaths.
Ernest Quintana, 79, was admitted to Kaiser Permanente Medical Center in Fremont, California, with failing lungs. A doctor appeared on a telepresence robot's video screen and told him he might not make it home. The family was devastated that the news was delivered through a machine rather than in person. Kaiser Permanente issued a statement expressing regret and said it would review its practices.
The Beijing Subway plans to introduce security screenings that use an AI-powered camera system to divide passengers into groups based on facial recognition. The system is intended to single out people for different security measures, aiming to speed up subway traffic. The article does not report any specific incident of harm or complaint.
Jacksons Food Stores is accused of violating a Portland, Oregon, city ordinance by using facial recognition technology in its stores after the ban took effect. The lawsuit, filed in December 2022, alleges the system, supplied by Blue Line Technology, wrongly identifies people as criminals and disproportionately affects women and people of colour. Customers were required to look at a camera and be scanned before entering. The retailer has not responded to the allegations.
An anonymous Google whistleblower states that the company acquired medical data of 50 million patients from Ascension without their knowledge. The transfer, known as Project Nightingale, raised privacy and security concerns. The article reports that a federal inquiry was launched into whether HIPAA protections were followed.
Amazon is being sued in two class-action lawsuits in Los Angeles and Seattle over its Alexa smart assistant's recordings of children. The lawsuits allege that Amazon lacks consent to create voiceprints of children, building up detailed profiles without parental permission. Amazon denies the allegations, stating it only stores data with device-owner permission and allows parents to delete recordings.
Engineer.ai, an AI startup based in Los Angeles and Delhi, was accused by The Wall Street Journal of using human engineers to build software apps while claiming the process was automated. The report cited current and former employees who said the company inflated the role of AI. Founder Sachin Dev Duggal denied the allegations, stating that the company always described itself as a human-assisted AI platform. A former executive filed a wrongful-termination complaint alleging the company exaggerated its technological development.
Two artists and an advertising company created a deepfake video of Mark Zuckerberg using CannyAI's video dialogue replacement technology and uploaded it to Instagram. The video shows Zuckerberg saying things he never said about controlling data. Facebook stated it would treat the content as misinformation and de-prioritize it. The incident tests Facebook's policies on manipulated media.
Domino's Pizza introduced the Dom Pizza Checker, a machine-learning camera system, in its Australia and New Zealand stores. The system checks whether pizzas meet order and quality standards and orders workers to remake any that do not. It is also to be used in a franchise scorecard to identify underperforming stores. Domino's said the tool was for training team members, not for punishing mistakes.
Each incident is written from the reporting or the account it cites. A report is an allegation, not a finding.
