Udemy automatically opts instructors into AI training with limited opt-out window
Udemy automatically opted instructors into having their classes used to train its generative AI program. Instructors were given a three-week window to opt out, which has now passed, leaving some unable to remove their content. Katie Stegs, an instructor, said she only found out via an email welcoming her to the program and found the opt-out option greyed out. Udemy defended the policy, citing the technical difficulty of removing data from AI models, while some instructors have left the platform in protest.
- Company involved
- Udemy
- AI system involved
- Generative AI Program (GenAI Program)
6 source articles · read the reporting →
Mozilla Foundation animation exposes racial bias in test proctoring app
The Mozilla Foundation commissioned an animated short telling the story of Amaya, a student whose test proctoring software failed to recognise her because of her skin tone. The software's facial recognition system did not detect her presence, illustrating the bias that students of colour face with remote-learning tools. The animation aimed to raise awareness and encourage programmers to address hidden biases in their work.
1 source article · read the reporting →
Fired Teacher Says AI Tainted Review And Arbitration - law360.com
The AI-generated performance evaluation contributed to the teacher's termination.
- Company involved
- Concord Public Schools
1 source article · read the reporting →
Meazure Learning Agrees $1.35m California Bar Exam Class-Action Settlement - Lawyer Monthly
The exam software experienced technical failures that prevented candidates from logging in or completing the California bar exam.
- Company involved
- Meazure Learning
- AI system involved
- ProctorU
1 source article · read the reporting →
University of Illinois Ends Proctorio Contract Over Accessibility Concerns
The University of Illinois at Urbana-Champaign announced it will not renew its emergency contract with Proctorio, an online proctoring software, citing significant accessibility, privacy, and equity concerns. Students reported that the AI system unfairly flagged individuals with disabilities, nonwhite students, and those wearing headscarves as suspicious during exams. The decision follows student petitions and campaigns against the software, which they described as invasive spyware. The university is exploring alternative assessment methods.
- Company involved
- University of Illinois at Urbana-Champaign
- AI system involved
- Proctorio
6 source articles · read the reporting →
University of Toronto exam monitoring software disadvantages BIPOC students
The University of Toronto continued using ProctorU and Examity exam monitoring software despite reports that the facial recognition technology fails to identify BIPOC students, causing stress and delays. Students like Maame Adjoa experienced the AI system being unable to verify their identity, requiring manual intervention. The university acknowledged the equity concerns but has not banned the software, while ProctorU stated that human proctors make final decisions. The issue has drawn criticism from US senators and researchers who highlight built-in bias in AI facial recognition.
- Company involved
- University of Toronto
- AI system involved
- ProctorU, Examity
1 source article · read the reporting →
University of Texas at Austin Stops Using Biased Ph.D. Admissions Algorithm
The University of Texas at Austin's computer science department used a machine-learning system called GRADE from 2013 to 2020 to score Ph.D. applicants. Critics argued the system perpetuated bias by training on past admissions decisions and using features like institution prestige and biased language in recommendation letters. The department discontinued the system in 2020, citing maintenance difficulties, but the controversy highlighted concerns about algorithmic fairness in admissions. The system was never disclosed to applicants, and its impact on underrepresented groups remains unknown.
- Company involved
- University of Texas at Austin
- AI system involved
- GRADE
2 source articles · read the reporting →
California State Bar Flags Over 3,000 Online Bar Exam Takers for Possible Cheating
In October 2020, the California State Bar administered its first online bar exam using ExamSoft's AI-enabled proctoring software. The software flagged 3,190 out of 9,301 test takers for potential rule violations such as gazing off-screen or having food. The state bar is reviewing each case, which could require affected individuals to retake the exam, delaying their ability to practice law. Test takers have expressed distress and hired attorneys to contest the allegations.
- Company involved
- California State Bar
- AI system involved
- ExamSoft
2 source articles · read the reporting →
LLMjacking Attack Leverages Stolen Credentials to Exploit Cloud LLMs
The Sysdig Threat Research Team observed an attack where stolen cloud credentials were used to access cloud-hosted large language model services. The attackers targeted a vulnerable Laravel system to obtain credentials, then used them to invoke models like Anthropic Claude on AWS Bedrock. They intended to sell LLM access to other cybercriminals, potentially costing victims over $46,000 per day. The attack involved checking credentials against ten AI services and using a reverse proxy to manage access.
- AI system involved
- Claude (v2/v3) on AWS Bedrock
2 source articles · read the reporting →
AI detector falsely accuses student of cheating at Central Methodist University
Moira Olmsted, a student at Central Methodist University, submitted a written assignment and received a zero grade after her professor said an AI detection tool flagged her work as AI-generated. Olmsted alleges the accusation was false and that her writing had been flagged at least once before. The incident highlights the risk of AI detectors falsely accusing students of cheating.
- Company involved
- Central Methodist University
1 source article · read the reporting →
Blind people and advocates criticise AccessiBe for worsening website access
Blind users and disability advocates say AccessiBe, an automated web accessibility overlay, disrupts screen readers on websites, making some sites unnavigable and preventing users from paying rent, shopping or teaching. More than 400 people signed an open letter asking companies to stop using automated overlays. Some blind users have brought ADA lawsuits against companies that use AccessiBe; one case was referred to mediation and another was settled. AccessiBe denies that it was at fault, saying critics do not give specific examples.
- Company involved
- AccessiBe
- AI system involved
- AccessiBe
10 source articles · read the reporting →
Universities' use of race in student risk algorithms harms Black and Latinx students
The Markup found that several universities, including UMass Amherst, UW–Milwaukee, and Texas A&M, use EAB's Navigate software which incorporates race as a predictor of student dropout risk. The algorithms label Black and Latinx students as high risk at disproportionately high rates compared to White students. Students, professors, and experts worry this is pushing minority students out of math and science majors through biased advising.
- Company involved
- University of Massachusetts Amherst,University of Wisconsin–Milwaukee,Texas A&M University,Texas Tech University,South Dakota State University
- AI system involved
- Navigate
1 source article · read the reporting →
LAUSD shelves AI chatbot after vendor AllHere collapses
Los Angeles Unified School District turned off its 'Ed' AI chatbot on June 14, 2024, after the vendor AllHere furloughed most staff due to financial collapse. The chatbot, which cost $3 million, was designed to provide students and parents with academic guidance and school information. A former AllHere employee alleged that student data was improperly shared with third parties and processed overseas, raising privacy concerns. The district stated it will ensure privacy protections and plans to eventually restore the chatbot.
- Company involved
- Los Angeles Unified School District
- AI system involved
- Ed
5 source articles · read the reporting →
Houston teachers sue over secret EVAAS evaluation algorithm
Houston Independent School District used the SAS Institute's EVAAS system, a proprietary algorithm, to evaluate teacher performance based on student test scores. The Houston Federation of Teachers and several teachers sued in 2014, alleging the system violated their Fourteenth Amendment procedural due process rights because the algorithm is a trade secret and teachers cannot verify the accuracy of their scores. A federal magistrate judge refused to dismiss the procedural due process claims in May 2017, ruling that teachers must have an opportunity to test the accuracy of the scores that could cost them their jobs.
- Company involved
- Houston Independent School District
- AI system involved
- Educational Value-Added Assessment System (EVAAS)
1 source article · read the reporting →
Teachers find no value in SAS EVAAS system in Southwest School District
A study examined the SAS Education Value-Added Assessment System (EVAAS®) used by the Southwest School District (SSD) to evaluate teacher effectiveness for high-stakes consequences. Teachers reported that the system produced inconsistent results, was biased by student factors, and reduced morale and collaboration. The study found that teachers did not use the data formatively as intended, and unintended consequences included heightened pressure and teaching to the test.
- Company involved
- Southwest School District (SSD)
- AI system involved
- SAS Education Value-Added Assessment System (EVAAS®)
10 source articles · read the reporting →
Texas A&M Professor Uses ChatGPT to Falsely Accuse Students of Cheating
Jared Mumm, an instructor at Texas A&M University–Commerce, used ChatGPT to check if students' writing was AI-generated, leading him to accuse them of using the chatbot. He informed students they would receive an incomplete grade and those deemed guilty would get a zero, causing distress and temporarily withholding one student's diploma. The university investigated and stated that no students failed or were barred from graduating, and the professor is working individually with students to resolve the issue.
- Company involved
- Texas A&M University–Commerce
- AI system involved
- ChatGPT
5 source articles · read the reporting →
Beijing school uses Hanwang emotion recognition system to monitor student behaviour
Niulanshan First Secondary School in Beijing has installed Hanwang Technology's Classroom Care System, which photographs students every second and analyses their facial expressions and behaviour. The system sends weekly reports to teachers and parents, and one student was described as low participation in English class despite being recorded as focused 94% of the time. The school has been highlighted in a report by Article 19, which says such emotion recognition tools rely on junk science and risk eroding privacy and human rights.
- Company involved
- Niulanshan First Secondary School
- AI system involved
- Classroom Care System
10 source articles · read the reporting →
Chinese university deploys facial recognition for attendance monitoring
A university in eastern China has installed a facial recognition system at its entrance and in two classrooms to monitor student attendance and behaviour. The system automatically captures students' faces without their cooperation. The pilot project is intended to improve attendance rates. No specific incidents of harm have been reported.
10 source articles · read the reporting →
UBC students raise concerns over Proctorio's privacy breach and discrimination
The University of British Columbia (UBC) uses Proctorio, an algorithmic test proctoring software, for remote exams. In June 2020, a UBC student reported an issue with Proctorio, and the company's CEO released the student's support chat logs, sparking privacy concerns. The AMS of UBC published an open letter alleging that Proctorio discriminates against students of colour, those with disabilities, and other groups by flagging 'abnormal' behaviours and denying access. The letter calls on UBC to end its contract with Proctorio and conduct an external audit.
- Company involved
- University of British Columbia
- AI system involved
- Proctorio
10 source articles · read the reporting →
Middle schooler beats Edgenuity grading algorithm to get perfect score
A seventh-grade student in the Los Angeles Unified School District received a failing grade on a history assignment graded by Edgenuity's automated scoring algorithm. With help from his mother, a history professor, he reverse-engineered the algorithm by writing a paragraph with relevant keywords and a jumble of words, earning a perfect score. The incident highlights concerns about the accuracy and fairness of automated grading systems in education.
- Company involved
- Los Angeles Unified School District
- AI system involved
- Edgenuity
10 source articles · read the reporting →
ProctorU threatens UC Santa Barbara faculty over privacy criticism
ProctorU, an online proctoring service, sent a legal threat letter to the University of California Santa Barbara faculty association after the association expressed concerns about ProctorU's data privacy policies. The letter, from ProctorU's lawyer, invoked defamation, copyright, and trademark claims, and was copied to state and federal prosecutors. The faculty association had asked the university to reconsider its relationship with ProctorU due to concerns about student data sharing.
- Company involved
- ProctorU
- AI system involved
- ProctorU
10 source articles · read the reporting →
ProctorU data breach exposes 444,000 user records
ProctorU, an online exam proctoring service, confirmed a data breach after a threat actor leaked a database of user records on a hacker forum. The database contained email addresses, names, addresses, phone numbers, and hashed passwords for approximately 444,000 users, including students from universities such as UCLA, Princeton, and Harvard, as well as U.S. military members. ProctorU stated that the breach involved records from 2014 and that they are investigating the incident.
- Company involved
- ProctorU
- AI system involved
- ProctorU
8 source articles · read the reporting →
Proctortrack data breach exposed student data from online proctoring
Proctortrack, an online proctoring service used by universities, suffered a data breach in September 2020 when its source code was leaked online. An analysis by Consumer Reports found that the code contained hard-coded passwords and exposed the names and email addresses of over 150 students. The company acknowledged the leak but said no harm resulted. Students had been required to use the software, which performed facial recognition and recorded video during exams.
- Company involved
- Proctortrack
- AI system involved
- Proctortrack
10 source articles · read the reporting →
Proctorio's exam proctoring system allows human agents to review student feeds despite privacy claims
A security researcher analyzed Proctorio's Chrome extension and found evidence that the system allows human agents to review students' room scans and live ID checks, contrary to Proctorio's claims that only professors can access recordings. The system flags students based on behavioral metrics and can terminate exams. The researcher also raised concerns about discrimination against low-income students and privacy violations.
- Company involved
- Proctorio
- AI system involved
- Proctorio
10 source articles · read the reporting →
