Facial recognition trialled in Australian schools to mark attendance
Five Australian schools are trialling a facial recognition system from LoopLearn to automate student attendance. The system scans students' faces and sends attendance data to an app. Privacy experts have raised concerns about data security and surveillance. The Victorian government previously tried to stop the trial due to privacy risks.
- Company involved
- Clarendon College
- AI system involved
- LoopLearn
10 source articles · read the reporting →
California bar exam facial recognition system fails to verify Arab-American student's identity
Ahmed Alamri, an Arab-American law student, was unable to register for the practice California bar exam because ExamSoft's facial recognition system repeatedly failed to recognize his face, citing poor lighting. Alamri attempted to verify his identity over 75 times in different rooms and lighting conditions without success. He and other students filed an emergency petition with the California Supreme Court, alleging that the system discriminates against people of color. The incident highlights concerns about bias in facial recognition technology used for exam proctoring.
- Company involved
- California State Bar
- AI system involved
- ExamSoft
10 source articles · read the reporting →
Cleveland State University's room scan requirement ruled unconstitutional
A federal judge ruled that Cleveland State University's requirement for a student to undergo a 360-degree room scan before an online exam was an unreasonable search under the Fourth Amendment. The student, enrolled at the public university, was told shortly before the exam that he would need to scan his private space. The court found that the university's justifications did not outweigh the privacy protections of the home. No final judgment or injunction has been issued yet.
- Company involved
- Cleveland State University
10 source articles · read the reporting →
UW-Madison disables Honorlock after skin tone recognition failure
The University of Wisconsin-Madison disabled the exam pause feature of its Honorlock anti-cheating software in March 2021 after three students complained that the software failed to recognize their darker skin tones and paused their exams. The software, used since online classes began, automatically pauses exams when it cannot detect facial features. Honorlock denied the issue was related to skin tone, attributing it to students looking away from their webcams. The university responded by disabling the feature.
- Company involved
- University of Wisconsin-Madison
- AI system involved
- Honorlock
10 source articles · read the reporting →
Retorio AI personality test swayed by candidate appearance in BR experiment
Bayerischer Rundfunk journalists conducted experiments with Retorio's AI video interview analysis tool. The AI, which assesses personality traits from short videos, produced different scores when the same actress changed her appearance (glasses, headscarf, wig) or the video background and lighting were altered. The start-up Retorio acknowledged that the AI considers external image, similar to a human interviewer. Experts warned that such software could perpetuate stereotypes and unfairly affect job candidates.
- AI system involved
- Retorio AI
10 source articles · read the reporting →
CBSE introduces facial recognition system for students to access digital documents
The Central Board of Secondary Education (CBSE) has introduced a facial recognition system for Class 10 and 12 students to access their digital academic documents. A live image of the student is compared with the photograph on their CBSE admit card and, if the match succeeds, the certificate is emailed to them. The facility is available on Digi Locker for 2020 records and is expected to help foreign students and those unable to open a Digi Locker account.
- Company involved
- Central Board of Secondary Education (CBSE)
- AI system involved
- Facial Recognition System
10 source articles · read the reporting →
Dartmouth Medical School charges 17 students with cheating based on tracking system.
Dartmouth College's Geisel School of Medicine charged 17 students with cheating on remote exams during the pandemic. The charges were based on secret tracking of student activity on the learning management system. Critics say the system is prone to errors and inappropriate for monitoring students.
- Company involved
- Dartmouth College Geisel School of Medicine
10 source articles · read the reporting →
Huawei and Megvii tested Uyghur alarm facial recognition system
Huawei and Megvii conducted an interoperability test in 2018 that verified a 'Uyghur alarm' function in Megvii's facial recognition system, according to a confidential report found by IPVM. The system could identify Uyghur minorities and alert police. Huawei denied real-world application and deleted the report after IPVM inquired. Megvii denied targeting ethnic groups. The test report was publicly accessible on Huawei's European website.
- Company involved
- Huawei
- AI system involved
- Megvii Dynamic Face Recognition
10 source articles · read the reporting →
AI emotion-detection software tested on Uyghurs in Xinjiang
A software engineer alleged that AI-based emotion-detection cameras were installed in police stations in Xinjiang and tested on Uyghur detainees. The system, placed three metres from subjects, analysed facial expressions and skin pores to create a pie chart indicating emotional state, with red suggesting anxiety, which was used as an indication of guilt. The engineer claimed the software was intended for 'pre-judgement without any credible evidence'. Human Rights Watch described the material as shocking, and the Chinese embassy denied the existence of such technology.
- Company involved
- Chinese government
- AI system involved
- Emotion detection camera
10 source articles · read the reporting →
PwC develops facial recognition tool to monitor employees working from home
Accounting giant PwC has developed a facial recognition tool that logs when employees are absent from their computer screens while working from home. The tool, intended for financial institutions, requires workers to provide written reasons for any absences, including toilet breaks. Commentators have criticised the tool as a huge invasion of privacy, with concerns about damage to trust and increased stress. PwC stated that the technology is designed to help regulated institutions meet compliance obligations and that voluntary consent of traders is essential.
- Company involved
- PwC
8 source articles · read the reporting →
Userviz machine-learning aimbot shut down after Activision request
A developer known as User101 shut down the Userviz cheat after Activision requested that he stop developing it. The software used computer vision and machine learning to automate aiming in games such as Call of Duty: Warzone, and was marketed as undetectable. It was never published, and the developer said his intention was not to do anything illegal.
- Company involved
- User101
- AI system involved
- Userviz
7 source articles · read the reporting →
Australian police trial facial recognition for COVID quarantine compliance
Australian police in New South Wales, Victoria, and Western Australia are trialling facial recognition software from Genvis to monitor people in home quarantine. Individuals are required to take selfies when randomly checked, and the software verifies their location and identity. Privacy advocates and the UN have raised concerns about surveillance overreach and potential human rights violations. The trials are voluntary, but critics warn of inaccuracies and mission creep.
- Company involved
- New South Wales Police Force, Victoria Police, Western Australia Police
- AI system involved
- Genvis facial recognition software
10 source articles · read the reporting →
Deep-Live-Cam enables real-time face swapping, raising fraud concerns
A software package called Deep-Live-Cam has gone viral for enabling real-time face swapping on webcams using a single photo. The tool, which is free and open source, allows anyone to impersonate another person's appearance in a video chat, raising concerns about remote deception and fraud. The article notes that similar technology has been used in a $25 million heist in Hong Kong and in audio deepfake scams.
- AI system involved
- Deep-Live-Cam
7 source articles · read the reporting →
DeepSeek-R1 censors 85% of sensitive Chinese political prompts in tests
Promptfoo tested DeepSeek-R1 against a dataset of 1,360 politically sensitive prompts and found that about 85% of them were refused. The refusals followed a standard form aligned with Chinese Communist Party policy. The testing also demonstrated that the censorship could be trivially bypassed using simple jailbreak techniques, such as prompt injection or changing the context.
- Company involved
- DeepSeek
- AI system involved
- DeepSeek-R1
5 source articles · read the reporting →
Meta tracks employee keystrokes on Google, LinkedIn, Wikipedia for AI training
Meta is using an internal tool, Model Capability Initiative (MCI), to capture employees' keystrokes, mouse movements and screen contents on work computers, including on sites such as Google, LinkedIn, Wikipedia and Slack, to train AI agents. Meta confirmed the project and said safeguards protect sensitive content and that the data is not used for other purposes. Employees raised concerns in internal messages that the tool could expose passwords, product details and personal information. A Meta memo said staff can avoid capture by not doing personal work on work computers.
- Company involved
- Meta
- AI system involved
- Model Capability Initiative (MCI)
5 source articles · read the reporting →
UIUC students petition to stop Proctorio exam proctoring over privacy concerns
A petition at the University of Illinois at Urbana-Champaign (UIUC) alleges that Proctorio, an online exam proctoring system, violates student privacy by accessing websites, downloads, screen content, and app settings. The petition claims the terms of service allow monitoring by 'any other means necessary', which students find unsettling. The petition, created on September 30, 2020, gathered 1,087 supporters but does not report any specific incident of harm. It calls on UIUC to discontinue use of Proctorio in favour of alternatives.
- Company involved
- UIUC
- AI system involved
- Proctorio
9 source articles · read the reporting →
iBorderCtrl lie detector falsely flagged honest reporter as liar
A journalist testing Europe's iBorderCtrl virtual policeman at the Serbian-Hungarian border gave honest answers but was deemed a liar by the system, scoring 48 out of 100 with four false answers flagged. The Hungarian policeman said the result suggested further checks, though none were carried out. The reporter only learned of the result after filing a data access request under European privacy laws. Experts and transparency activists have criticised the technology as pseudoscientific and potentially discriminatory.
- Company involved
- iBorderCtrl consortium
- AI system involved
- Silent Talker / iBorderCtrl virtual policeman
10 source articles · read the reporting →
AI detectors falsely flag non-native English speakers' essays as AI-generated
A study by Stanford researchers found that seven popular AI text detectors wrongly flagged over half of essays written by non-native English speakers as AI-generated. The detectors assess text perplexity, and non-native speakers' simpler word choices lead to false positives. The researchers warn that this bias could have serious implications for students and job applicants, potentially leading to discrimination.
9 source articles · read the reporting →
Evolv weapon detection system falsely flags Chromebooks as weapons
Evolv's AI-based weapon detection system has been reported to falsely identify certain Chromebook laptops as weapons, causing false alarms at security checkpoints. The system is used in venues such as schools. The article discusses the flaw and user experiences. No official response from Evolv has been mentioned.
- Company involved
- Evolv
- AI system involved
- Evolv
5 source articles · read the reporting →
Baltimore schools monitor student laptops for suicide signs using GoGuardian Beacon
Baltimore City Public Schools uses GoGuardian Beacon software to monitor student laptops for signs of suicide. Since March 2021, the system has flagged 786 alerts, with nine students taken to emergency rooms. Privacy advocates warn the monitoring could lead to disciplinary actions, outing of LGBTQ students, and disproportionately affect disadvantaged students. School officials defend the practice as a safeguard.
- Company involved
- Baltimore City Public Schools
- AI system involved
- GoGuardian Beacon
10 source articles · read the reporting →
Duke University recorded students' faces without proper consent for public dataset
In March 2014, Duke University researchers recorded thousands of students walking to class on campus without their knowledge or proper consent, creating the DukeMTMC dataset of over 2 million image frames. The dataset was placed on a public website and downloaded by academics, security contractors, and military researchers globally, including Chinese companies and military academies linked to surveillance of ethnic minorities. The university took down the public website in April 2019 after an Institutional Review Board investigation found the study deviated significantly from the approved protocol. The lead researcher apologized, stating he took full responsibility for his mistakes.
- Company involved
- Duke University
- AI system involved
- DukeMTMC
10 source articles · read the reporting →
Proctorio anti-cheating software failed to catch student cheaters in study
Researchers at the University of Twente in the Netherlands tested Proctorio, an anti-cheating software, by asking 30 computer science students to sit an exam while six of them cheated. Proctorio did not flag any of the cheaters and flagged some honest students for irregular behaviour. An independent human review caught only one of the six cheaters. Proctorio disputed the study's methodology and cited other research.
- Company involved
- Proctorio
- AI system involved
- Proctorio
5 source articles · read the reporting →
Delhi Police use facial recognition to screen PM Modi rally attendees
Delhi Police used an Automated Facial Recognition System (AFRS) to screen crowds at Prime Minister Narendra Modi's rally on December 22, 2019. The system, originally installed to find missing children, was used to identify possible disruptions. Privacy advocates called the move illegal and unconstitutional, saying it amounts to mass surveillance. Police defended the use, citing credible intelligence about possible disruptions.
- Company involved
- Delhi Police
- AI system involved
- Automated Facial Recognition System (AFRS)
6 source articles · read the reporting →
Vumacam's AI CCTV system flagged 28 black people as suspicious in Johannesburg suburbs
In Johannesburg suburbs, Vumacam's AI-powered CCTV network using iSentry software flagged 28 black individuals as 'suspicious' in a shift report, according to a 2019 article. The system, deployed by private security firms, uses video analytics to detect abnormal behavior and alerts security guards. The article alleges that the system disproportionately targets people of color, reflecting racial bias in a racially divided country.
- Company involved
- Vumacam
- AI system involved
- iSentry
6 source articles · read the reporting →
