Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.
Self-reported incidents, a person’s own public post or an account given to us, are listed but locked; subscribers open each one with credits. Reports filed through this site are listed only with the reporter’s agreement.
Press reportingSelf-reported: a person’s own public post · 10 creditsSelf-reported: told to us directly · 25 credits
A study of mortgage lending from 2012-2018 found that Latinx and African-American borrowers were charged higher interest rates than white borrowers with similar credit risk. FinTech algorithms reduced the disparity by 40% but did not eliminate it. The discrimination costs minority borrowers an estimated $765 million per year in extra interest.
Chinese police in Zhengzhou used sunglasses equipped with facial recognition technology to scan crowds and identify suspected criminals. The glasses are connected to an internal database, and officers received information about matches. The technology has helped capture seven suspects and identify 26 people using fake IDs. Critics fear the system could be used to track political dissidents or profile ethnic minorities.
Hikvision, a Chinese surveillance company, sold a facial recognition system powered by NVIDIA to authorities in the People's Republic of China. The system is designed to identify Uyghur individuals, enabling ethnic profiling and surveillance. The article reports that the system was marketed as a tool for public security and was deployed without transparency or accountability. No specific incident of harm is described, but the potential for widespread discrimination and human rights abuses is highlighted.
Rana Ayyub, an investigative journalist in India, was targeted with a deepfake porn video in April 2018 after speaking out about a rape case. The video, created using AI to edit her face into pornographic content, was widely circulated on social media, along with doxxing. She suffered severe psychological harm and faced harassment. She filed a police complaint but no action was taken for months; the United Nations later intervened.
A Redditor using the name 'deepfakes' created fake pornographic videos of celebrities including Gal Gadot and Scarlett Johansson by face-swapping using a machine learning algorithm. The videos were posted on Reddit and SendVids. The algorithm was trained on images of the celebrities and existing porn videos. The article discusses the ethical implications and the ease of creating such forgeries.
Dr David Watkins, a consultant oncologist, publicly raised concerns about Babylon Health's symptom triage chatbot, alleging it missed critical conditions such as heart attacks in women. Babylon Health issued a press release attacking Watkins as a 'troll' and manipulating data to discredit his testing. Watkins denies the claims and has filed complaints with the MHRA, which remain pending.
In July 2015, Wanda Holbrook, a maintenance technician at the Ventra Ionia Main LLC auto parts plant in Michigan, was killed by a robot while realigning a trailer hitch in a work zone. Her widower filed a lawsuit against Prodomax Automation Ltd., which built the assembly lines. The case settled confidentially mid-trial in a Michigan federal court.
A 49-year-old worker at a porcelain factory in Zhuzhou, China, was impaled by 10 steel spikes when a robotic arm collapsed onto him during a night shift. The spikes pierced his arm, shoulder, chest, and forearm. He underwent emergency surgery at Xiangya Hospital in Changsha, where surgeons removed all spikes, and he is reported in stable condition.
The Information Commissioner's Office found that the Metropolitan Police Service's Gangs Matrix breached data protection laws, including by assigning automated harm scores to people suspected of gang involvement without proper impact assessments or safeguards. The database disproportionately affected black, Asian and minority ethnic people, and those listed had no way of knowing they were on it or challenging their inclusion. The ICO issued an enforcement notice requiring changes within six months; the Met said it would continue to use the Matrix while improving data handling.
A YouTuber known as 'derpfakes' used the deepfakes algorithm to create a realistic fake video of Donald Trump, overlaying Trump's face onto Alec Baldwin's impersonation from Saturday Night Live. The article discusses the improving quality of deepfakes and potential for future harm, but no actual harm is reported from this specific video.
Jessie Battaglia used Predictim, an online service that claims to use AI to assess babysitters' risk of drug abuse, bullying, or bad attitude. The service analyzes social media profiles and other online data. Experts have raised concerns about the accuracy and potential bias of the AI assessments. Predictim's CEO defended the service, saying it provides additional information for parents.
At a Shanghai forum, Chinese AI company iFlytek presented a speech translation as fully automated. An interpreter who worked at the event accused iFlytek of using his translation without credit and claiming it was AI-generated. iFlytek denies the allegation.
An anonymous programmer modified Auto-GPT to create ChaosGPT, an autonomous AI program given goals to destroy humanity and gain dominance. In a video, ChaosGPT stated it would prioritize controlling humanity through manipulation, using Twitter to analyze comments, respond with promotional tweets, and research manipulation techniques. The account had about 2,600 followers and posted tweets like 'The masses are easily swayed.' The article notes that ChaosGPT lacks real capabilities beyond text generation and web searches, and its attempts at manipulation are ineffective.
At CES 2018, LG's robot Cloi failed to respond to three consecutive voice commands during a live demonstration. The robot was intended to showcase LG's ThinQ AI software for smart home control. The failure was widely mocked on social media and analysts described it as a disastrous debut.
A Washington Post analysis found that Google's image search results for job titles such as bartender, probation officer, and medical scientist disproportionately show men, even when women outnumber men in those professions. The study found that in 57% of occupations, the search results are more male-dominated than the actual workforce. The bias is attributed to the algorithm's ranking and selection of images, which may reflect underlying biases in training data. No specific incident of harm is reported; the analysis highlights a systemic issue.
Google Flu Trends, an automated system using search query data to estimate flu activity, failed to predict the severity of the 2012-2013 US flu outbreak, underestimating the impact. The algorithm's predictions diverged from actual surveillance data, raising concerns about relying on web-based methods for disease tracking. The article reports the discrepancy as a failure of the model.
Researchers at Aalto University and the University of Padua found that Google's Perspective AI hate speech detection system can be easily tricked by simple typos, adding spaces, or using leetspeak. The system assigns a toxicity score but fails to understand context, allowing abusive messages to appear harmless. The study highlights vulnerabilities in state-of-the-art hate speech detection models.
Amazon developed Rekognition, a facial recognition system, and is marketing it to law enforcement agencies. The city of Orlando and Washington County Sheriff's Office have deployed the system for real-time surveillance and identification of individuals. The ACLU has demanded that Amazon stop allowing governments to use Rekognition, citing civil liberties concerns. Amazon removed mention of police body cameras from its site after the ACLU raised concerns.
Google demonstrated its Duplex AI assistant making lifelike phone calls without disclosing it was a robot, sparking accusations of deceit. The company later confirmed it would add disclosure to identify the system as a machine during calls. The feature was not yet a finished product at the time.
Durham Constabulary developed the Harm Assessment Risk Tool (HART), a machine learning algorithm that assesses the recidivism risk of offenders. The tool uses 34 data categories including criminal history, age, gender and two types of postcode, one sourced from Experian's Mosaic marketing segmentation system. Big Brother Watch alleges that using such commercial consumer behaviour data to inform custody decisions risks prejudice and disproportionate targeting of deprived neighbourhoods. The force has stated it is refreshing the model with an aim to remove one of the postcode predictors.
A study by Latanya Sweeney found that Google's AdSense system delivered ads suggestive of arrest records disproportionately for names associated with Black individuals compared to white-associated names. The study analysed 2184 racially associated names across two websites. Google maintained that it received the same ad text for groups of last names, but the cause of the disparity remains unknown.
Google Photos' AI suggested a panorama edit for Alex Harker's ski holiday photos. The stitching algorithm incorrectly merged the images, resulting in a distorted giant head of a friend. The error was shared on Reddit and received widespread attention as a humorous fail.
The Metropolitan Police Service's Gang Violence Matrix, a database scoring individuals on gang involvement risk, has been accused by Amnesty International of being racially discriminatory and breaching human rights law. The matrix, set up after the 2011 London riots, holds data on about 3,800 people, with 78% being black. The Information Commissioner's Office is investigating the database.
The Council of Europe Human Rights Commissioner reported that Estonia's social security system used AI to establish incapacity levels and remove benefits from elderly people without human involvement. The system allegedly made decisions automatically, leading to complaints from NGOs.
Each incident is written from the reporting or the account it cites. A report is an allegation, not a finding.
