The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

92 incidents closest to “BriefCam Rapid Review” · matched on meaning · public reporting

Clearview AI tested facial recognition surveillance cameras with UFT and Rudin

Clearview AI, the facial recognition company that scraped billions of photos from social media, developed a surveillance camera system under the name Insight Camera. The system was tested by the United Federation of Teachers and Rudin Management in New York City. The UFT used it to identify individuals who had made threats and prevent them from entering its offices. Clearview did not respond to requests for comment.

Company involved
Clearview AI
AI system involved
Insight Camera

9 source articles · read the reporting →

WF-52Q1NZ26 Feb 2021

Qoves facial assessment tool ranks journalist's attractiveness and suggests procedures

Tate Ryan-Mosley, a journalist at MIT Technology Review, used Qoves Studio's AI facial assessment tool, which automatically scored her face and listed perceived flaws such as smile lines and under-eye bags. The tool recommended skin-care products and paid surgical consultation reports. She also tried Megvii's Face++ beauty-scoring system, which gave her two percentage attractiveness scores. The article raises concerns that such algorithms are inaccurate, biased and not transparently used by social media platforms.

Company involved
Qoves Studio
AI system involved
Facial Assessment Tool

9 source articles · read the reporting →

Judge rules police search using Flock was mass surveillance

A judge ruled that a police search using Flock's automated license plate readers constituted a form of mass surveillance. The ruling concerns the deployment of the AI-based camera system by law enforcement, which the court found to be an invasive surveillance practice. No further details of the case were provided in the article.

AI system involved
Flock

4 source articles · read the reporting →

Uttar Pradesh Police uses AI cameras to track women's distress

Uttar Pradesh Police, in collaboration with Staqu Technologies, deployed AI-powered cameras in Lucknow for surveillance of women in distress. Activists have criticised the system for invading privacy. The system is currently in use.

Company involved
Uttar Pradesh Police
AI system involved
Trinetra

10 source articles · read the reporting →

WF-PMLNUT17 Sep 2020

California bar exam facial recognition system fails to verify Arab-American student's identity

Ahmed Alamri, an Arab-American law student, was unable to register for the practice California bar exam because ExamSoft's facial recognition system repeatedly failed to recognize his face, citing poor lighting. Alamri attempted to verify his identity over 75 times in different rooms and lighting conditions without success. He and other students filed an emergency petition with the California Supreme Court, alleging that the system discriminates against people of color. The incident highlights concerns about bias in facial recognition technology used for exam proctoring.

Company involved
California State Bar
AI system involved
ExamSoft

10 source articles · read the reporting →

DeepScore markets facial and voice analysis app for trustworthiness scoring despite experts' doubts

DeepScore, a Tokyo-based company, is marketing an app that uses facial and voice recognition to score people's trustworthiness for lenders and insurers in Japan, Indonesia, Vietnam and the Philippines. The company says the app can detect deception with 70 per cent accuracy, but researchers and privacy advocates say there is no reliable scientific basis for such judgments and warn of discrimination and privacy harms. The chief executive said the system is only one part of lenders' and insurers' decision-making and that people can choose not to use it. Critics respond that an unequal balance of power makes consent difficult.

Company involved
DeepScore
AI system involved
DeepScore

6 source articles · read the reporting →

WF-ZQ3YFL1 Dec 2020

Retorio AI personality test swayed by candidate appearance in BR experiment

Bayerischer Rundfunk journalists conducted experiments with Retorio's AI video interview analysis tool. The AI, which assesses personality traits from short videos, produced different scores when the same actress changed her appearance (glasses, headscarf, wig) or the video background and lighting were altered. The start-up Retorio acknowledged that the AI considers external image, similar to a human interviewer. Experts warned that such software could perpetuate stereotypes and unfairly affect job candidates.

AI system involved
Retorio AI

10 source articles · read the reporting →

WF-SQJYDZ22 Oct 2020

CBSE introduces facial recognition system for students to access digital documents

The Central Board of Secondary Education (CBSE) has introduced a facial recognition system for Class 10 and 12 students to access their digital academic documents. A live image of the student is compared with the photograph on their CBSE admit card and, if the match succeeds, the certificate is emailed to them. The facility is available on Digi Locker for 2020 records and is expected to help foreign students and those unable to open a Digi Locker account.

Company involved
Central Board of Secondary Education (CBSE)
AI system involved
Facial Recognition System

10 source articles · read the reporting →

WF-FST9Z61 Apr 2018

ViaQuatro's facial recognition system in São Paulo metro challenged in court

In April 2018, ViaQuatro installed the Digital Interactive Doors System, developed by AdMobilize, on the São Paulo metro's yellow line. The system used cameras to detect passengers' faces and claimed to infer their emotion, age, and gender in order to target advertisements. The Brazilian Institute of Consumer Protection (IDEC) filed a public civil action alleging that the system violated consumer and data protection laws by processing biometric data without consent and making pseudoscientific and discriminatory inferences. A judge ordered the cameras removed in August 2018, and the case is pending a final ruling.

Company involved
ViaQuatro
AI system involved
Digital Interactive Doors System (DID system)

10 source articles · read the reporting →

Adobe uses Creative Cloud user data to train AI

Adobe's content analysis feature may scan Creative Cloud and Document Cloud files to train machine learning models, including object recognition in Lightroom and Liquid Mode in Acrobat. The company says it may analyze images, audio, video, text and other documents stored on its servers. Adobe offers an opt-out in account privacy settings, but the article notes the company did not ask first.

Company involved
Adobe
AI system involved
Content analysis

10 source articles · read the reporting →

X's Grok chatbot wrongly confirms AI-generated war video as real

During the US-Israel war with Iran, X's Grok chatbot was used by users to verify the authenticity of AI-generated videos. In many cases, Grok wrongly insisted that the AI-generated videos were real, misleading users. The article reports that X will suspend creators from monetisation if they post AI-generated conflict videos without a label.

Company involved
X
AI system involved
Grok

4 source articles · read the reporting →

WF-J5IU7Z3 Dec 2024

FTC settles with IntelliVision over deceptive facial recognition claims

The Federal Trade Commission (FTC) took action against IntelliVision Technologies Corp. for making false, misleading, or unsubstantiated claims about its AI-powered facial recognition software. The company allegedly claimed its software had one of the highest accuracy rates on the market and was free of gender or racial bias, without supporting evidence. The FTC also alleged that IntelliVision did not train its software on millions of faces as claimed, but on images of approximately 100,000 individuals. Under a proposed consent order, IntelliVision is prohibited from making such misrepresentations without competent and reliable testing.

Company involved
IntelliVision Technologies Corp.
AI system involved
IntelliVision facial recognition software

9 source articles · read the reporting →

WF-VIE4ZV13 Aug 2024

Deep-Live-Cam enables real-time face swapping, raising fraud concerns

A software package called Deep-Live-Cam has gone viral for enabling real-time face swapping on webcams using a single photo. The tool, which is free and open source, allows anyone to impersonate another person's appearance in a video chat, raising concerns about remote deception and fraud. The article notes that similar technology has been used in a $25 million heist in Hong Kong and in audio deepfake scams.

AI system involved
Deep-Live-Cam

7 source articles · read the reporting →

WF-S2D5111 Nov 2020

Dahua facial recognition technology can sort by race and alert police when detecting Uighurs

Dahua Technologies, a Chinese surveillance camera company, developed facial recognition software that can classify individuals by race and send real-time warnings to police when it identifies Uighur people. Documents and code revealed the capability, though it is unclear if it has been deployed. Human rights advocates and Uighur individuals have expressed concern about potential surveillance and discrimination. The company did not respond to requests for comment.

Company involved
Dahua Technologies
AI system involved
Dahua facial recognition software

10 source articles · read the reporting →

Answer.AI tests Devin and reports 14 failures in 20 tasks

Answer.AI's team tested Devin, an autonomous AI coding assistant, on 20 real-world tasks over a month. Devin succeeded in only 3 tasks, failed 14, and was inconclusive in 3. The team found Devin often produced overly complex or hallucinated solutions and could not recognize fundamental blockers. They ultimately decided to stick with tools that allow more human control.

AI system involved
Devin

5 source articles · read the reporting →

WebinarTV secretly records Zoom calls and turns them into AI podcasts

WebinarTV, a company that bills itself as a search engine for webinars, is secretly scanning the internet for Zoom meeting links, recording the calls, and turning them into AI-generated podcasts for profit. People only found out their calls were recorded when WebinarTV contacted them to promote its services. The recordings may put call participants at risk.

Company involved
WebinarTV

4 source articles · read the reporting →

WF-MFXC8G1 Aug 2023

Amazon merchants complain about AI review summaries focusing on negatives

Amazon introduced an AI-powered feature to generate summaries of customer product reviews. Merchants reported that the AI inaccurately highlights negative feedback, even when only a small percentage of reviews are critical. The summaries may exaggerate negative themes, potentially harming sales. Amazon acknowledged the issue and said it is working to refine the technology based on seller feedback.

Company involved
Amazon
AI system involved
AI-powered review highlights

5 source articles · read the reporting →

WF-LEN91Y1 May 2021

US CBP deploys CBP One app using facial recognition for asylum seekers amid privacy concerns

The article reports that U.S. Customs and Border Protection quietly deployed the CBP One mobile app at the Mexico border. The app uses facial recognition and geolocation to collect and verify information on asylum seekers before they enter the United States. Privacy experts warn that the app poses risks of persistent surveillance and that the facial recognition algorithm is unreliable for people of colour. A previous CBP facial recognition pilot was hacked, exposing images. CBP says the app is voluntary and data is secure.

Company involved
U.S. Customs and Border Protection
AI system involved
CBP One

8 source articles · read the reporting →

N-Tech.lab's FindFace used to identify St Petersburg metro passengers without consent

Egor Tsvetkov photographed passengers on the St Petersburg metro without their permission and used N-Tech.lab's facial recognition service FindFace to match their faces to public Vkontakte profiles. He published the results in an art project called 'Your Face is Big Data', saying he wanted to show how 'digital narcissism' can lead to stalking. Privacy advocates said the project was ethically problematic because the subjects had not consented and their identities were exposed. FindFace had been launched by N-Tech.lab in February 2016.

Company involved
N-Tech.lab
AI system involved
FindFace

8 source articles · read the reporting →

WF-3OJOAW6 Mar 2024

AI image generators produce misleading election images, study finds

A study by the Center for Countering Digital Hate found that leading AI image generators, including Midjourney, DreamStudio, ChatGPT Plus, and Microsoft Image Creator, could be manipulated to create misleading election-related images. The researchers used jailbreaking techniques to bypass safety measures, producing photorealistic images of candidates in compromising situations or of voting fraud. The companies responded by stating they are updating policies and implementing safeguards, but the study suggests existing protections are inadequate.

Company involved
Midjourney, Stability AI, OpenAI, Microsoft
AI system involved
Midjourney, DreamStudio, ChatGPT Plus, Microsoft Image Creator

8 source articles · read the reporting →

WF-0YQPL21 Jan 2019

iBorderCtrl lie detector falsely flagged honest reporter as liar

A journalist testing Europe's iBorderCtrl virtual policeman at the Serbian-Hungarian border gave honest answers but was deemed a liar by the system, scoring 48 out of 100 with four false answers flagged. The Hungarian policeman said the result suggested further checks, though none were carried out. The reporter only learned of the result after filing a data access request under European privacy laws. Experts and transparency activists have criticised the technology as pseudoscientific and potentially discriminatory.

Company involved
iBorderCtrl consortium
AI system involved
Silent Talker / iBorderCtrl virtual policeman

10 source articles · read the reporting →

WF-QUNA4I19 Dec 2023

RT launches AI-generated presenters without consistent disclosure

RT unveiled AI-generated 'journalists' Anastasia and Alexander on its Spanish-language channel RT Español. In 70% of the videos reviewed, the broadcasts failed to disclose that these presenters were artificial. The presenters covered a range of topics, including controversial and politically sensitive subjects, raising ethical concerns about transparency and potential manipulation of audiences.

Company involved
RT
AI system involved
Anastasia and Alexander (AI-generated presenters)

3 source articles · read the reporting →

WF-DLNNBB1 Jan 2015

French police secretly used Briefcam facial recognition since 2015

The French National Police has been using Briefcam's Video Synopsis software since 2015, according to internal documents obtained by Disclose. The software, which includes facial recognition capabilities, was deployed without the required data protection impact assessment or notification to the CNIL. The Ministry of the Interior concealed the use of this tool for eight years. The police hierarchy allegedly used the software for facial recognition without judicial requisition, and the DGPN did not respond to requests for comment.

Company involved
French National Police
AI system involved
Briefcam Video Synopsis

10 source articles · read the reporting →

WF-AZK2NL11 Jul 2023

Check Point Research finds Google Bard can generate phishing emails and malware

Check Point Research analysed Google's generative AI platform Bard and found it could be used to create phishing emails, malware keyloggers, and basic ransomware code with minimal manipulation. Bard's anti-abuse restrictors were significantly lower than ChatGPT's, making it easier to generate malicious content. The researchers demonstrated these capabilities in controlled tests but did not report actual harm to specific individuals or organisations.

Company involved
Google
AI system involved
Bard

4 source articles · read the reporting →

← Newerpage 3 of 4Older →