Flawed AI algorithms grade student essays in multiple US states

AI-powered systems used to score the essay portions of standardised tests in the US regularly suffer from bias and other issues, according to a media investigation. What happened Citing research studies, Motherboard reported that so-called 'automated essay scoring engines' used by 21 US states demonstrated bias against certain demographic groups. These included e-rater, an engine developed and run by the nonprofit Educational Testing Service (ETS), which researchers found to have given higher scores to some students, particularly those from mainland China, than did expert human graders. It also tended to underscore African-American students and Arabic, Spanish and Hindi speakers. Meantime, ACCUPLACER, a machine-scored test owned by the College Board, failed to reliably predict female, Asian, Hispanic, and African-American students’ eventual writing grades. Furthermore, some systems can be fooled by nonsense essays with sophisticated vocabulary, whilst most stuggle to judge more nuanced aspects of writing, like creativity , experts say. Why it happened Automated scoring is proving popular with schools as it is cheaper and faster than human grading. But concerns about accuracy and fairness persist. These systems use natural language processing (NLP) to predict scores based on patterns from previously graded essays, but they are prone to biases against certain demographic groups. The algorithms focus on measurable aspects like vocabulary and sentence length, which may disadvantage English language learners and others who write differently. What it means Experts suggest that while machine scoring can be useful, it should always be accompanied by human grading to ensure quality control. Automated essay scoring Automated essay scoring (AES) is the use of specialized computer programs to assign grades to essays written in an educational setting. It is a form of educational assessment and an application of natural language processing . Source: Wikipedia 🔗 System 🤖 ACCUPLACER 🔗 E-rater scoring engine 🔗 Operator: Developer: ACCUPLACER; American Institutes for Research (AIR); Educational Testing Service (ETS) Country: USA Sector: Education Purpose: Assess and score student essays Technology: NLP/text analysis; Machine learning Issue: Accuracy/reliability; Bias/discrimination - race, ethnicity; Ethics/values Transparency : Govern ance

Date it happened

2019-08-01

Product, system or model

E-rater

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.