The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

92 incidents closest to “FACE Plus” · matched on meaning · public reporting

WF-J6K8T44 Nov 2017

Apple Face ID fails to distinguish between siblings in video

A video posted on Reddit shows Apple's Face ID unlocking an iPhone X for a sibling who was not the enrolled user, after he put on glasses similar to his brother's. The system had been set up by the other brother, and the video demonstrates that Face ID can be fooled by siblings with similar facial features. Apple has acknowledged that Face ID may be less secure for siblings with similar appearances.

Company involved
Apple
AI system involved
Face ID

9 source articles · read the reporting →

WF-PA1ZLG1 Apr 2020

Workaround compiled Instagram selfies into face-mask facial recognition training set

Workaround, an AI startup, published the COVID19 Mask Image Dataset in April 2020, containing more than 1,200 photographs of people wearing face masks scraped from Instagram without their knowledge or consent. The dataset was intended to help train facial recognition algorithms so they could identify people whose faces are partly covered during the COVID-19 pandemic. The firm's chief executive acknowledged that the people pictured were not asked for permission and were unaware they were included. Facebook said it was investigating and does not allow third parties to use photos in this way without consent.

Company involved
Workaround
AI system involved
COVID19 Mask Image Dataset

10 source articles · read the reporting →

WF-IKJ9FY20 May 2018

Belgian political party sp.a created deep fake video of President Trump

In May 2018, the Belgian political party sp.a released a deep fake video appearing to show President Donald Trump telling Belgium to withdraw from the Paris Climate Agreement. The video was created using lip-synching technology and was intended as a parody to promote the party's climate petition. The party confirmed the video was a deep fake and said it was legally sound.

Company involved
sp.a (Socialistische Partij Anders)

10 source articles · read the reporting →

WF-BFYN1E22 Sep 2020

Chinese propaganda network used AI-generated faces on Facebook

Facebook removed two networks of fake accounts spreading propaganda, one originating from China and one from the Philippines. The Chinese network used AI-generated faces created through Generative Adversarial Networks (GANs) to create fake profiles and evade detection. The campaign targeted the US election and Southeast Asia, pushing pro-China content. Facebook took down the accounts and published a report with Graphika.

Company involved
Chinese propaganda network (Fujian province)
AI system involved
GANs (Generative Adversarial Networks)

10 source articles · read the reporting →

WF-P2YW671 Jan 2019

DiveFace face-recognition dataset reuses Flickr photos despite licence restrictions

DiveFace is a face-recognition dataset published in 2019 with 139,677 images of about 24,000 people. The authors say the images came from Flickr via the MegaFace and YFCC100M datasets and were automatically labelled by gender and ethnicity. Exposing.ai found that many of the photos carry Creative Commons licences which prohibit derivative use, suggesting the dataset may have been assembled without proper permission. The dataset remains available from the authors' GitHub page.

AI system involved
DiveFace

3 source articles · read the reporting →

WF-9PEI6Z29 Aug 2019

Zao deepfake app sparks privacy and fraud fears in China

The Zao face-swapping app, developed by Momo, sparked privacy and fraud fears after its launch in China on 29 August 2019. The app's terms and conditions initially gave the developer permanent rights to use users' images, which experts said broke Chinese law. Momo subsequently apologized and deleted the controversial clause. Alipay assured users that deepfakes could not bypass its facial recognition payment system.

Company involved
Momo
AI system involved
Zao

10 source articles · read the reporting →

Clearview AI tested facial recognition surveillance cameras with UFT and Rudin

Clearview AI, the facial recognition company that scraped billions of photos from social media, developed a surveillance camera system under the name Insight Camera. The system was tested by the United Federation of Teachers and Rudin Management in New York City. The UFT used it to identify individuals who had made threats and prevent them from entering its offices. Clearview did not respond to requests for comment.

Company involved
Clearview AI
AI system involved
Insight Camera

9 source articles · read the reporting →

WF-D1F8DA1 Apr 2016

FindFace facial recognition app used to out sex workers in Russia

The FindFace facial recognition app is being used by users of the Russian website Dvach to identify porn actresses and sex workers from images and match them to their VKontakte accounts. The users then spam the victims' families and friends with the information, aiming to out them. Several victims complained to FindFace, but the founder stated there was no way to prevent the misuse.

Company involved
FindFace
AI system involved
FindFace

7 source articles · read the reporting →

WF-04CE0N17 Jan 2021

Faces of the Riot publishes extracted faces from Parler Capitol riot videos

The website Faces of the Riot used open source facial recognition software to extract and publish over 6,000 faces from Parler videos of the January 6 Capitol riot. The creator, an anonymous college student, aimed to help identify participants in the insurrection. The site has been criticised for privacy violations and potential vigilante misuse. The creator says they are working to remove non-rioter faces.

Company involved
Faces of the Riot
AI system involved
Faces of the Riot

8 source articles · read the reporting →

WF-52Q1NZ26 Feb 2021

Qoves facial assessment tool ranks journalist's attractiveness and suggests procedures

Tate Ryan-Mosley, a journalist at MIT Technology Review, used Qoves Studio's AI facial assessment tool, which automatically scored her face and listed perceived flaws such as smile lines and under-eye bags. The tool recommended skin-care products and paid surgical consultation reports. She also tried Megvii's Face++ beauty-scoring system, which gave her two percentage attractiveness scores. The article raises concerns that such algorithms are inaccurate, biased and not transparently used by social media platforms.

Company involved
Qoves Studio
AI system involved
Facial Assessment Tool

9 source articles · read the reporting →

WF-UXMPE77 Oct 2020

Facebook lifts ban on Newfoundland seed advertisement after AI flagged it as sexual

Facebook's AI content moderation system banned an advertisement for seeds from The Seed Company by E.W. Gaze, deeming the image of onions as 'overtly sexual'. The company contacted Facebook, and the ban was lifted after human review. Facebook apologized for the error.

Company involved
Facebook

8 source articles · read the reporting →

WF-2ZNE3K8 Jan 2018

Huawei and Megvii tested Uyghur alarm facial recognition system

Huawei and Megvii conducted an interoperability test in 2018 that verified a 'Uyghur alarm' function in Megvii's facial recognition system, according to a confidential report found by IPVM. The system could identify Uyghur minorities and alert police. Huawei denied real-world application and deleted the report after IPVM inquired. Megvii denied targeting ethnic groups. The test report was publicly accessible on Huawei's European website.

Company involved
Huawei
AI system involved
Megvii Dynamic Face Recognition

10 source articles · read the reporting →

WF-SKZU1O25 May 2021

TikTok's beauty filter changed users' faces without their consent

TikTok users reported that the app applied an unwanted beauty filter to their videos, altering the shape of their faces. The effect, which appeared on some Android devices, could not be turned off and caused distress, including gender dysphoria for one user. TikTok acknowledged the issue and said it had been resolved, but did not explain the cause.

Company involved
TikTok
AI system involved
TikTok's beauty filter

9 source articles · read the reporting →

WF-W32GV626 Apr 2023

Plastic Forte fined for using facial recognition on workers without notice

The Spanish data protection agency AEPD fined Plastic Forte, a plastics manufacturer in Alicante, €20,000 for using facial recognition to record employees' working hours without informing them. A worker requested information about his personal data and discovered the biometric processing. The company initially argued it was only for time tracking but later acknowledged responsibility, resulting in a reduced fine of €12,000. The AEPD deemed facial recognition for time control as highly intrusive and requiring prior impact assessment.

Company involved
Plastic Forte

7 source articles · read the reporting →

WF-J617041 Apr 2018

Facebook fined for using facial recognition without consent in South Korea

Facebook created and stored facial recognition templates of 200,000 South Korean users without their consent between April 2018 and September 2019. South Korea's Personal Information Protection Commission (PIPC) fined Facebook $5.5 million and ordered it to destroy the collected data and obtain proper consent. The PIPC also fined Netflix and issued a warning to Google for other privacy violations.

Company involved
Facebook
AI system involved
Facial recognition system

8 source articles · read the reporting →

WF-1IONT71 Sep 2021

Facebook AI mislabels video of Black men as Primates

Facebook's AI recommendation system labeled a video of Black men with a prompt asking users if they wanted to keep seeing videos about Primates. The social network apologized for the 'unacceptable error' and disabled the feature. The incident highlights ongoing issues with racial bias in AI systems.

Company involved
Facebook

9 source articles · read the reporting →

WF-VIFDEV19 Jun 2023

Douyin content creator used deepfake to fake Ukraine soldier identity

A content creator on Douyin used deepfake technology to pretend to be a Chechen soldier in Ukraine, posting fabricated videos for months. The account gained nearly 400,000 followers and sold items via e-commerce. Users discovered the IP address was in Henan, China, revealing the deception. Douyin suspended the account for spreading misinformation and banned it from making money.

10 source articles · read the reporting →

WF-J5IU7Z3 Dec 2024

FTC settles with IntelliVision over deceptive facial recognition claims

The Federal Trade Commission (FTC) took action against IntelliVision Technologies Corp. for making false, misleading, or unsubstantiated claims about its AI-powered facial recognition software. The company allegedly claimed its software had one of the highest accuracy rates on the market and was free of gender or racial bias, without supporting evidence. The FTC also alleged that IntelliVision did not train its software on millions of faces as claimed, but on images of approximately 100,000 individuals. Under a proposed consent order, IntelliVision is prohibited from making such misrepresentations without competent and reliable testing.

Company involved
IntelliVision Technologies Corp.
AI system involved
IntelliVision facial recognition software

9 source articles · read the reporting →

WF-U4WH351 Jun 2020

ScaleFactor reportedly failed to deliver promised automated bookkeeping software

ScaleFactor, an Austin-based startup, is reported to have failed to deliver the automated, real-time bookkeeping tools it promised customers, instead relying on human bookkeepers and a Filipino contract accounting firm. The company told Forbes in June that it was shutting down, initially blaming the pandemic, but Forbes later reported that its problems predated Covid-19. Investors reportedly came to see the company as more of a services business than a software platform, and pulled funding after a pivot to a marketplace model. No legal or regulatory action is reported.

Company involved
ScaleFactor

10 source articles · read the reporting →

Face recognition bypass used to register shell companies for tax fraud in Shanghai

In a major tax fraud case in Shanghai, criminals used face recognition cracking techniques to register shell companies on government platforms. They purchased high-resolution photos and used apps to create fake videos, then used modified phones to bypass face recognition authentication. The scheme resulted in fraudulent invoices totaling over 500 million yuan. The case was prosecuted by the Shanghai procuratorate.

7 source articles · read the reporting →

WF-VIE4ZV13 Aug 2024

Deep-Live-Cam enables real-time face swapping, raising fraud concerns

A software package called Deep-Live-Cam has gone viral for enabling real-time face swapping on webcams using a single photo. The tool, which is free and open source, allows anyone to impersonate another person's appearance in a video chat, raising concerns about remote deception and fraud. The article notes that similar technology has been used in a $25 million heist in Hong Kong and in audio deepfake scams.

AI system involved
Deep-Live-Cam

7 source articles · read the reporting →

New Jersey court rules police must disclose facial recognition algorithms used to identify defendant

Francisco Arteaga was charged with armed robbery after a facial recognition search by the New York Police Department identified him as a possible match. The New Jersey police used this match to obtain witness identifications. Arteaga requested information about the facial recognition software, including its source code and error rate, but the trial court denied his request. The New Jersey appellate court ruled that the prosecution must disclose the algorithms, citing due process and the Brady rule.

Company involved
New Jersey Police

7 source articles · read the reporting →

WF-S2D5111 Nov 2020

Dahua facial recognition technology can sort by race and alert police when detecting Uighurs

Dahua Technologies, a Chinese surveillance camera company, developed facial recognition software that can classify individuals by race and send real-time warnings to police when it identifies Uighur people. Documents and code revealed the capability, though it is unclear if it has been deployed. Human rights advocates and Uighur individuals have expressed concern about potential surveillance and discrimination. The company did not respond to requests for comment.

Company involved
Dahua Technologies
AI system involved
Dahua facial recognition software

10 source articles · read the reporting →

WF-LEN91Y1 May 2021

US CBP deploys CBP One app using facial recognition for asylum seekers amid privacy concerns

The article reports that U.S. Customs and Border Protection quietly deployed the CBP One mobile app at the Mexico border. The app uses facial recognition and geolocation to collect and verify information on asylum seekers before they enter the United States. Privacy experts warn that the app poses risks of persistent surveillance and that the facial recognition algorithm is unreliable for people of colour. A previous CBP facial recognition pilot was hacked, exposing images. CBP says the app is voluntary and data is secure.

Company involved
U.S. Customs and Border Protection
AI system involved
CBP One

8 source articles · read the reporting →

← Newerpage 3 of 4Older →