ExamSoft failure during Michigan Bar Exam
On July 28, 2020, ExamSoft's software platform failed during the Michigan Bar Exam, preventing some test takers from completing the exam. ExamSoft posted a statement on X (formerly Twitter) with the hashtag #MichiganBarExam, but critics noted the company deleted and reposted the statement multiple times, allegedly to hide negative comments. The incident affected a group of examinees and caused disruption to their ability to take the exam.
- Company involved
- ExamSoft
- AI system involved
- ExamSoft platform
10 source articles · read the reporting →
Wisconsin’s dropout prediction algorithm labels students high risk with racial bias
Wisconsin’s Dropout Early Warning System (DEWS) uses machine learning to predict middle school students' likelihood of graduating on time, relying on factors including race and income. The Markup found the algorithm frequently mislabels students, with a higher false alarm rate for Black and Hispanic students. The Wisconsin Department of Public Instruction has not informed schools of the racial disparities and continues to use the system.
- Company involved
- Wisconsin Department of Public Instruction
- AI system involved
- Dropout Early Warning System (DEWS)
10 source articles · read the reporting →
Nevada's AI grad score model cuts funding for low-income students
Nevada replaced its income-based school funding formula with a machine learning model developed by Infinite Campus that generates a 'grad score' for each student. The model reduced the number of students eligible for supplemental funding from 288,000 to 63,000, disproportionately affecting low-income students. Critics argue the model lacks transparency and de-emphasizes economic status, potentially leaving many high-poverty schools underfunded.
- Company involved
- Nevada Department of Education
- AI system involved
- Grad score model
7 source articles · read the reporting →
Qoves facial assessment tool ranks journalist's attractiveness and suggests procedures
Tate Ryan-Mosley, a journalist at MIT Technology Review, used Qoves Studio's AI facial assessment tool, which automatically scored her face and listed perceived flaws such as smile lines and under-eye bags. The tool recommended skin-care products and paid surgical consultation reports. She also tried Megvii's Face++ beauty-scoring system, which gave her two percentage attractiveness scores. The article raises concerns that such algorithms are inaccurate, biased and not transparently used by social media platforms.
- Company involved
- Qoves Studio
- AI system involved
- Facial Assessment Tool
9 source articles · read the reporting →
California bar exam facial recognition system fails to verify Arab-American student's identity
Ahmed Alamri, an Arab-American law student, was unable to register for the practice California bar exam because ExamSoft's facial recognition system repeatedly failed to recognize his face, citing poor lighting. Alamri attempted to verify his identity over 75 times in different rooms and lighting conditions without success. He and other students filed an emergency petition with the California Supreme Court, alleging that the system discriminates against people of color. The incident highlights concerns about bias in facial recognition technology used for exam proctoring.
- Company involved
- California State Bar
- AI system involved
- ExamSoft
10 source articles · read the reporting →
Cleveland State University's room scan requirement ruled unconstitutional
A federal judge ruled that Cleveland State University's requirement for a student to undergo a 360-degree room scan before an online exam was an unreasonable search under the Fourth Amendment. The student, enrolled at the public university, was told shortly before the exam that he would need to scan his private space. The court found that the university's justifications did not outweigh the privacy protections of the home. No final judgment or injunction has been issued yet.
- Company involved
- Cleveland State University
10 source articles · read the reporting →
DeepScore markets facial and voice analysis app for trustworthiness scoring despite experts' doubts
DeepScore, a Tokyo-based company, is marketing an app that uses facial and voice recognition to score people's trustworthiness for lenders and insurers in Japan, Indonesia, Vietnam and the Philippines. The company says the app can detect deception with 70 per cent accuracy, but researchers and privacy advocates say there is no reliable scientific basis for such judgments and warn of discrimination and privacy harms. The chief executive said the system is only one part of lenders' and insurers' decision-making and that people can choose not to use it. Critics respond that an unequal balance of power makes consent difficult.
- Company involved
- DeepScore
- AI system involved
- DeepScore
6 source articles · read the reporting →
UW-Madison disables Honorlock after skin tone recognition failure
The University of Wisconsin-Madison disabled the exam pause feature of its Honorlock anti-cheating software in March 2021 after three students complained that the software failed to recognize their darker skin tones and paused their exams. The software, used since online classes began, automatically pauses exams when it cannot detect facial features. Honorlock denied the issue was related to skin tone, attributing it to students looking away from their webcams. The university responded by disabling the feature.
- Company involved
- University of Wisconsin-Madison
- AI system involved
- Honorlock
10 source articles · read the reporting →
Retorio AI personality test swayed by candidate appearance in BR experiment
Bayerischer Rundfunk journalists conducted experiments with Retorio's AI video interview analysis tool. The AI, which assesses personality traits from short videos, produced different scores when the same actress changed her appearance (glasses, headscarf, wig) or the video background and lighting were altered. The start-up Retorio acknowledged that the AI considers external image, similar to a human interviewer. Experts warned that such software could perpetuate stereotypes and unfairly affect job candidates.
- AI system involved
- Retorio AI
10 source articles · read the reporting →
CBSE introduces facial recognition system for students to access digital documents
The Central Board of Secondary Education (CBSE) has introduced a facial recognition system for Class 10 and 12 students to access their digital academic documents. A live image of the student is compared with the photograph on their CBSE admit card and, if the match succeeds, the certificate is emailed to them. The facility is available on Digi Locker for 2020 records and is expected to help foreign students and those unable to open a Digi Locker account.
- Company involved
- Central Board of Secondary Education (CBSE)
- AI system involved
- Facial Recognition System
10 source articles · read the reporting →
Wisconsin court used secret COMPAS algorithm to sentence Loomis to harsher term
In Loomis v. Wisconsin, a judge used a COMPAS risk score from Northpointe to sentence a defendant to a harsher punishment. The algorithm was kept secret as a trade secret, preventing the defendant from challenging its accuracy. The Wisconsin Supreme Court upheld the sentence, ruling that the score was only one part of the rationale. The case raises concerns about due process and the use of secret algorithms in criminal sentencing.
- Company involved
- State of Wisconsin
- AI system involved
- COMPAS
10 source articles · read the reporting →
Hospitals use Epic AI to predict Covid-19 decline without validation
Dozens of hospitals across the US are using Epic's deterioration index AI system to predict which Covid-19 patients will become critically ill, despite the tool not being validated for the new disease. The rapid deployment during the pandemic bypassed normal testing and validation processes.
- AI system involved
- Deterioration index
9 source articles · read the reporting →
PwC develops facial recognition tool to monitor employees working from home
Accounting giant PwC has developed a facial recognition tool that logs when employees are absent from their computer screens while working from home. The tool, intended for financial institutions, requires workers to provide written reasons for any absences, including toilet breaks. Commentators have criticised the tool as a huge invasion of privacy, with concerns about damage to trust and increased stress. PwC stated that the technology is designed to help regulated institutions meet compliance obligations and that voluntary consent of traders is essential.
- Company involved
- PwC
8 source articles · read the reporting →
DCPS used allegedly false test scores in teacher value-added evaluations
The District of Columbia Public Schools used student test scores from schools under investigation for cheating in value-added calculations for teacher evaluations. More than 200 teachers were terminated based on these evaluations. Teachers can appeal their ratings to the chancellor, but decisions will not be made before the next school year. The school system removes affected scores only when cheating is confirmed.
- Company involved
- District of Columbia Public Schools
- AI system involved
- value-added model
10 source articles · read the reporting →
Answer.AI tests Devin and reports 14 failures in 20 tasks
Answer.AI's team tested Devin, an autonomous AI coding assistant, on 20 real-world tasks over a month. Devin succeeded in only 3 tasks, failed 14, and was inconclusive in 3. The team found Devin often produced overly complex or hallucinated solutions and could not recognize fundamental blockers. They ultimately decided to stick with tools that allow more human control.
- AI system involved
- Devin
5 source articles · read the reporting →
IRCC uses AI triage for Temporary Resident Visa applications
Immigration, Refugees and Citizenship Canada (IRCC) uses an AI system called Advanced Analytics to triage Temporary Resident Visa applications from India and China. The system categorizes applications into tiers, with Tier 1 approved automatically and others sent to human officers. Critics allege the system lacks transparency and may introduce bias, leading to visa refusals without clear rationale. The author, a Canadian immigration lawyer, is filing Federal Court cases on behalf of clients affected by refusals.
- Company involved
- Immigration, Refugees and Citizenship Canada (IRCC)
- AI system involved
- Advanced Analytics Triage of Overseas Temporary Resident Visa Applications
10 source articles · read the reporting →
Cigna StressWaves Test found unreliable and invalid in independent study
A study published in Scientific Reports evaluated the Cigna StressWaves Test (CSWT), an AI tool that claims to assess psychological stress from speech. The study found that the CSWT had poor test-retest reliability and poor validity compared to the Perceived Stress Scale. The authors warned that widespread availability of the tool could lead to misleading results and negative consequences for users making healthcare decisions. Cigna has not publicly responded to the findings.
- Company involved
- Cigna
- AI system involved
- Cigna StressWaves Test
4 source articles · read the reporting →
State Bar of California admits using AI to develop bar exam questions
The State Bar of California admitted that it used artificial intelligence to develop multiple-choice questions for the February 2025 bar exam. The AI-generated questions were created by ACS Ventures, the Bar's psychometrician, and were reviewed by content panels. Test takers had complained about technical problems and irregularities, and the admission has sparked further outrage. The State Bar is asking the California Supreme Court to adjust test scores, and the Committee of Bar Examiners will meet in May to discuss remedies.
- Company involved
- State Bar of California
7 source articles · read the reporting →
Texas uses AI to grade student STAAR test answers
The Texas Education Agency will use an automated scoring engine to grade written answers on the 2023 STAAR tests, replacing thousands of human graders. The system uses natural language processing and will initially score all responses, with a quarter rescored by humans. Educators have expressed concerns about the system's fairness and the potential for errors, especially for creative or non-standard answers.
- Company involved
- Texas Education Agency
- AI system involved
- automated scoring engine
10 source articles · read the reporting →
AI detectors falsely flag non-native English speakers' essays as AI-generated
A study by Stanford researchers found that seven popular AI text detectors wrongly flagged over half of essays written by non-native English speakers as AI-generated. The detectors assess text perplexity, and non-native speakers' simpler word choices lead to false positives. The researchers warn that this bias could have serious implications for students and job applicants, potentially leading to discrimination.
9 source articles · read the reporting →
Baltimore schools monitor student laptops for suicide signs using GoGuardian Beacon
Baltimore City Public Schools uses GoGuardian Beacon software to monitor student laptops for signs of suicide. Since March 2021, the system has flagged 786 alerts, with nine students taken to emergency rooms. Privacy advocates warn the monitoring could lead to disciplinary actions, outing of LGBTQ students, and disproportionately affect disadvantaged students. School officials defend the practice as a safeguard.
- Company involved
- Baltimore City Public Schools
- AI system involved
- GoGuardian Beacon
10 source articles · read the reporting →
Audit of LAION-400M finds sexual violence, racial slurs, and stereotypes in dataset
An audit of the LAION-400M dataset by Abeba Birhane and colleagues at University College Dublin and University of Edinburgh found that its automated curation using CLIP failed to remove sexually explicit images, racial slurs, and stereotypes. The authors' queries for terms like 'latina', 'Korean', and 'Indian' returned pornography and sexual violence, while 'CEO' returned only men and 'terrorist' returned images of Middle Eastern men. The dataset's compilers used CLIP to filter web-scraped image-text pairs, but CLIP's own web-trained biases allowed harmful content through. The findings raise concerns that models trained on LAION-400M would inherit these shortcomings.
- Company involved
- LAION-400M team
- AI system involved
- LAION-400M
7 source articles · read the reporting →
CBSE OnMark portal vulnerability exposed student data to Google Gemini
A 19-year-old ethical hacker, Nisarga Adhikary, claimed to have hacked the CBSE's digital evaluation ecosystem, revealing that personal information of students was processed by Google's Gemini in automation scripts. The Central Board of Secondary Education (CBSE) stated on May 31, 2026, that the identified vulnerabilities had been contained and other exploitable weaknesses were being ruled out. The board expressed gratitude to alert citizens and ethical hackers who pointed out the weaknesses. No actual data breach was confirmed, but the incident raised concerns about student privacy.
- Company involved
- Central Board of Secondary Education (CBSE)
- AI system involved
- OnMark
1 source article · read the reporting →
Audit of RisCanvi finds biases and reliability issues in criminal justice system
Eticas conducted an adversarial audit of RisCanvi, an AI risk assessment tool used in Catalonia's criminal justice system. The audit uncovered biases in risk classifications against specific demographics and significant reliability issues. The findings call for fairer practices in criminal justice AI.
- Company involved
- Catalonia's criminal justice system
- AI system involved
- RisCanvi
4 source articles · read the reporting →
