Mozilla Foundation animation exposes racial bias in test proctoring app
The Mozilla Foundation commissioned an animated short telling the story of Amaya, a student whose test proctoring software failed to recognise her because of her skin tone. The software's facial recognition system did not detect her presence, illustrating the bias that students of colour face with remote-learning tools. The animation aimed to raise awareness and encourage programmers to address hidden biases in their work.
1 source article · read the reporting →
15-Year-Old Test Exposes Flaw: ChatGPT for Teens Fails to Block Homework Cheating, Repeated Requests Bypass Restrictions
15歲使用者實測揭漏洞：青少年版ChatGPT難擋宿題代寫，反覆要求即破解 - BigGo 財經
A 15-year-old tester found that OpenAI's ChatGPT for Teens, launched in August, initially refused to write essays but generated full examples after repeated requests. It also immediately solved SAT-level math problems. Parental controls require account linking and are off by default, while experts warn the memory feature could lead to emotional attachment.
- Company involved
- OpenAI
- AI system involved
- ChatGPT for Teens
1 source article · read the reporting →
Resume prompt injection tricks AI hiring - moneywise.com
AI screening system determined which job applicants to advance to the next stage of recruitment.
1 source article · read the reporting →
Meazure Learning Agrees $1.35m California Bar Exam Class-Action Settlement - Lawyer Monthly
The exam software experienced technical failures that prevented candidates from logging in or completing the California bar exam.
- Company involved
- Meazure Learning
- AI system involved
- ProctorU
1 source article · read the reporting →
Portland Water Bureau AI pilot offers discount to billionaire CEO
The Portland Water Bureau used a machine learning system in a randomised control trial to identify customers for its financial assistance programme. The system selected Columbia Sportswear CEO Tim Boyle, a billionaire and one of the city's largest residential water consumers, for a 40% water bill discount. Boyle declined the discount, stating it should go to someone who needs it. The bureau is testing whether the SERVUS algorithm can accurately identify customers' ability to pay.
- Company involved
- Portland Water Bureau
- AI system involved
- Smart Discount Program
1 source article · read the reporting →
LLMjacking Attack Leverages Stolen Credentials to Exploit Cloud LLMs
The Sysdig Threat Research Team observed an attack where stolen cloud credentials were used to access cloud-hosted large language model services. The attackers targeted a vulnerable Laravel system to obtain credentials, then used them to invoke models like Anthropic Claude on AWS Bedrock. They intended to sell LLM access to other cybercriminals, potentially costing victims over $46,000 per day. The attack involved checking credentials against ten AI services and using a reverse proxy to manage access.
- AI system involved
- Claude (v2/v3) on AWS Bedrock
2 source articles · read the reporting →
International Baccalaureate's Secret Algorithm Downgrades Students, Jeopardising College Admissions
In July 2020, the International Baccalaureate (IB) program used a statistical model to predict final grades for over 170,000 students after cancelling exams due to Covid-19. Many students, including Anahita Nagpal, received lower-than-expected scores, causing them to lose college places and scholarships. Students, parents, and teachers questioned the fairness and transparency of the algorithm, with over 15,000 signing a petition demanding a fairer approach. IB defended the model but offered an appeals process, while some students planned to retake exams at their own expense.
- Company involved
- International Baccalaureate
1 source article · read the reporting →
Universities' use of race in student risk algorithms harms Black and Latinx students
The Markup found that several universities, including UMass Amherst, UW–Milwaukee, and Texas A&M, use EAB's Navigate software which incorporates race as a predictor of student dropout risk. The algorithms label Black and Latinx students as high risk at disproportionately high rates compared to White students. Students, professors, and experts worry this is pushing minority students out of math and science majors through biased advising.
- Company involved
- University of Massachusetts Amherst,University of Wisconsin–Milwaukee,Texas A&M University,Texas Tech University,South Dakota State University
- AI system involved
- Navigate
1 source article · read the reporting →
LAUSD shelves AI chatbot after vendor AllHere collapses
Los Angeles Unified School District turned off its 'Ed' AI chatbot on June 14, 2024, after the vendor AllHere furloughed most staff due to financial collapse. The chatbot, which cost $3 million, was designed to provide students and parents with academic guidance and school information. A former AllHere employee alleged that student data was improperly shared with third parties and processed overseas, raising privacy concerns. The district stated it will ensure privacy protections and plans to eventually restore the chatbot.
- Company involved
- Los Angeles Unified School District
- AI system involved
- Ed
5 source articles · read the reporting →
New York City Department of Education Releases Flawed Teacher Ratings Publicly
In February 2012, the New York City Department of Education publicly released individual Teacher Data Reports rating over 12,000 teachers based on value-added analysis of student test scores. The ratings, originally intended for internal use, were disclosed after a court ruled in favor of media organizations under the Freedom of Information Act. The teachers' union and education experts criticised the data as unreliable due to large margins of error and failure to account for demographic factors, warning that the release would unfairly shame or praise educators. The Department defended the ratings as a useful perspective on teacher effectiveness.
- Company involved
- New York City Department of Education
- AI system involved
- Teacher Data Reports
6 source articles · read the reporting →
Houston teachers sue over secret EVAAS evaluation algorithm
Houston Independent School District used the SAS Institute's EVAAS system, a proprietary algorithm, to evaluate teacher performance based on student test scores. The Houston Federation of Teachers and several teachers sued in 2014, alleging the system violated their Fourteenth Amendment procedural due process rights because the algorithm is a trade secret and teachers cannot verify the accuracy of their scores. A federal magistrate judge refused to dismiss the procedural due process claims in May 2017, ruling that teachers must have an opportunity to test the accuracy of the scores that could cost them their jobs.
- Company involved
- Houston Independent School District
- AI system involved
- Educational Value-Added Assessment System (EVAAS)
1 source article · read the reporting →
Boston Public Schools reverses algorithm-generated bus schedule after parent protest
In December 2017, Boston Public Schools announced new school start times and bus schedules generated by an algorithm designed by MIT researchers. Some parents, predominantly white, protested the changes, arguing they were unfair. The city reversed the decision after the protest, despite the algorithm's aim to improve equity and sleep health for the majority of students. The incident highlighted the lack of transparency in algorithmic decision-making.
- Company involved
- Boston Public Schools
10 source articles · read the reporting →
Beijing school uses Hanwang emotion recognition system to monitor student behaviour
Niulanshan First Secondary School in Beijing has installed Hanwang Technology's Classroom Care System, which photographs students every second and analyses their facial expressions and behaviour. The system sends weekly reports to teachers and parents, and one student was described as low participation in English class despite being recorded as focused 94% of the time. The school has been highlighted in a report by Article 19, which says such emotion recognition tools rely on junk science and risk eroding privacy and human rights.
- Company involved
- Niulanshan First Secondary School
- AI system involved
- Classroom Care System
10 source articles · read the reporting →
Meta's BlenderBot 3 chatbot generates anti-Semitic stereotypes and election denial claims
Meta launched its BlenderBot 3 AI chatbot for public testing in the US. Users quickly discovered that the bot made anti-Semitic remarks, such as claiming Jews are overrepresented among the super-rich, and expressed uncertainty about Joe Biden winning the 2020 election. Meta acknowledged that chatbots can generate offensive content and stated it was collecting feedback to improve future systems.
- Company involved
- Meta
- AI system involved
- BlenderBot 3
3 source articles · read the reporting →
Chinese military researchers used Meta's Llama 2 to develop defense chatbot ChatBIT
Chinese military researchers, including two affiliated with the People's Liberation Army, reportedly used Meta's Llama 2 AI model to develop a defense chatbot called ChatBIT. According to Reuters, the chatbot is designed to gather and process intelligence and offer information for operational decision-making. Meta stated that the use was unauthorized and contrary to its acceptable use policy.
- Company involved
- People's Liberation Army (PLA)
- AI system involved
- ChatBIT
6 source articles · read the reporting →
Russian disinformation network Matryoshka uses AI deepfakes on Bluesky
The Russian disinformation network 'Matryoshka' has launched a campaign on Bluesky, posting AI-generated videos that impersonate academics to spread anti-Ukraine narratives. The videos use AI to replicate the voices of real professors, making false claims about the war. The campaign was detected by Bellingcat founder Eliot Higgins and the Bot Blocker project. The videos have gained limited traction on Bluesky so far, but researchers warn the platform is more vulnerable to such disinformation.
- Company involved
- Matryoshka (Russian disinformation network)
3 source articles · read the reporting →
Middle schooler beats Edgenuity grading algorithm to get perfect score
A seventh-grade student in the Los Angeles Unified School District received a failing grade on a history assignment graded by Edgenuity's automated scoring algorithm. With help from his mother, a history professor, he reverse-engineered the algorithm by writing a paragraph with relevant keywords and a jumble of words, earning a perfect score. The incident highlights concerns about the accuracy and fairness of automated grading systems in education.
- Company involved
- Los Angeles Unified School District
- AI system involved
- Edgenuity
10 source articles · read the reporting →
Wisconsin’s dropout prediction algorithm labels students high risk with racial bias
Wisconsin’s Dropout Early Warning System (DEWS) uses machine learning to predict middle school students' likelihood of graduating on time, relying on factors including race and income. The Markup found the algorithm frequently mislabels students, with a higher false alarm rate for Black and Hispanic students. The Wisconsin Department of Public Instruction has not informed schools of the racial disparities and continues to use the system.
- Company involved
- Wisconsin Department of Public Instruction
- AI system involved
- Dropout Early Warning System (DEWS)
10 source articles · read the reporting →
Nevada's AI grad score model cuts funding for low-income students
Nevada replaced its income-based school funding formula with a machine learning model developed by Infinite Campus that generates a 'grad score' for each student. The model reduced the number of students eligible for supplemental funding from 288,000 to 63,000, disproportionately affecting low-income students. Critics argue the model lacks transparency and de-emphasizes economic status, potentially leaving many high-poverty schools underfunded.
- Company involved
- Nevada Department of Education
- AI system involved
- Grad score model
7 source articles · read the reporting →
CBSE introduces facial recognition system for students to access digital documents
The Central Board of Secondary Education (CBSE) has introduced a facial recognition system for Class 10 and 12 students to access their digital academic documents. A live image of the student is compared with the photograph on their CBSE admit card and, if the match succeeds, the certificate is emailed to them. The facility is available on Digi Locker for 2020 records and is expected to help foreign students and those unable to open a Digi Locker account.
- Company involved
- Central Board of Secondary Education (CBSE)
- AI system involved
- Facial Recognition System
10 source articles · read the reporting →
DCPS used allegedly false test scores in teacher value-added evaluations
The District of Columbia Public Schools used student test scores from schools under investigation for cheating in value-added calculations for teacher evaluations. More than 200 teachers were terminated based on these evaluations. Teachers can appeal their ratings to the chancellor, but decisions will not be made before the next school year. The school system removes affected scores only when cheating is confirmed.
- Company involved
- District of Columbia Public Schools
- AI system involved
- value-added model
10 source articles · read the reporting →
South Korea's AI textbook program rolled back after complaints
South Korea's Ministry of Education launched AI-powered digital textbooks in March 2025 for math, English, and computer science. Students, teachers, and parents complained about technical problems, inaccuracies, and data privacy risks. After four months, the textbooks were stripped of official status and made optional. The program was criticized for being rushed and insufficiently tested.
- Company involved
- South Korean Ministry of Education
- AI system involved
- AI-powered textbooks
7 source articles · read the reporting →
Answer.AI tests Devin and reports 14 failures in 20 tasks
Answer.AI's team tested Devin, an autonomous AI coding assistant, on 20 real-world tasks over a month. Devin succeeded in only 3 tasks, failed 14, and was inconclusive in 3. The team found Devin often produced overly complex or hallucinated solutions and could not recognize fundamental blockers. They ultimately decided to stick with tools that allow more human control.
- AI system involved
- Devin
5 source articles · read the reporting →
Dutch government to refund over 10,000 students over discriminatory DUO fraud algorithm
The Dutch government has pledged to refund over 10,000 students who were unjustly flagged for student finance fraud by a discriminatory algorithm used by the Education Executive Agency (DUO). The algorithm, implemented in 2012, used criteria that disproportionately targeted students from immigrant backgrounds, particularly those of Turkish and Moroccan descent. Following investigations by the Dutch Data Protection Authority and an independent report by PwC, the algorithm was suspended in July 2023 and replaced with a random-sampling system. The government has allocated 61 million euro for refunds.
- Company involved
- Education Executive Agency (DUO)
- AI system involved
- DUO fraud detection system
9 source articles · read the reporting →
