Proctorio uses 'racist' algorithms to detect students' faces
A facial detection system run by online proctoring service Proctorio w as found to be 'racist' . College student Lucy Satheesan reverse-engineered the exam software and discovered that Proctorio's software had been using a facial detection model that failed to recognise Black faces 57 percent of the time. By assessing the code behind Proctorio’s extension for the Chrome web browser, c ollege student Lucy Satheesan discovered that the file names associated with the tool’s 'proprietary' facial detection function were identical to those published by open-source computer vision software library OpenCV. Tbe discovery appeared to confirm complaints by students that Proctorio's face detection system is inaccurate and unreliable. The company uses the technology to see if a student is looking away from their screen, leaves the room, or if there’s another person in the frame - any of which could indicate cheating. The Electronic Privacy Information Center (EPIC) had earlier filed a complaint accusing Proctorio and four other online test proctoring services of unfair and deceptive trade practices. EPIC said it is preparing to file a lawsuit unless they change their practices. System 🤖 Proctorio Developer: Proctorio Country: USA Sector: Education Purpose: Detect faces Technology: Facial detection; Computer vision; Machi ne learning Issue: Fairness; Transparency Resources 📃 Proctorio CEO letter to US legislators Legal, regulatory 👩🏼⚖️ EPIC (2020). In re Online Test Proctoring Companies EPIC (2020). Letter to Proctorio EPIC (2020). Complaint and Request for Investigation, Injunction, and Other Relief Senator Richard Blumenthal (2020). Blumenthal Leads Call for Virtual Exam Software Companies to Improve Equity, Accessibility & Privacy for Students Amid Troubling Reports
- Date it happened
- 2021-04-01
- Organisation involved
- Miami University; University of British Columbia; University of Illinois
- Product, system or model
- OpenCV
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.