Proctorio fails to recognise Vrije Universiteit Black student
Technology company Proctorio was accused of racial discrimination after its e xam cheating system failed to recognise Robin Pocornie, a Black student at Vrije Universiteit Amsterdam, when she was taking exams during the COVID-19 pandemic. Pocornie noticed the software often appeared to struggle to properly 'process' her face, resulting in her resorting to sitting through her exams with a bright light shining on her face. In 2022, Pocornie filed a complaint (pdf) with the Netherlands Institute for Human Rights, which concluded she had presented sufficient facts for a presumption of discrimination, and that the university had to prove otherwise. In October 2023, the Institute ruled that Vrije Universiteit demonstrated that Pocornie did not experience more log-in issues than other students, and that her unstable internet connection or the fact that she was wearing glasses were to blame. It also said that 'it is entirely possible that the use of Proctorio or similar AI software could indeed lead to discrimination in a different situation.' The incident raised questions about the accuracy of the system, and led to a legal challenge. Proctorio had earlier responded by saying an audit of its facial detection model by US-based consultancy BABL AI found 'no significant bias toward anyone'. The company has not made the audit results public. System 🤖 Proctorio 🔗 Documents 📃 Proctorio (2023). Response to RTL Nieuws Operator: Vrije Universiteit Developer: Proctorio Country: Netherlands Sector: Education Purpose: Detect exam cheating Technology: Facial detection; Gaze detection; Machine learning Issue: Bias/discrimination - race; Transparency I nvestigations, assessments, audits 🧐 RTL Nederland (2023). Verantwoording onderzoek RTL Nieuws naar Procotrio-software (pdf)
- Date it happened
- 2022-01-01
- Organisation involved
- Vrije Universiteit
- Product, system or model
- Proctorio
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.