Allegheny child neglect screening system unfairly flags Blacks for investigation
An algorithm used to tell when kids are in danger of abuse or neglect was found to disproportionately flag Black children for “mandatory” investigations compared to white children. Research from Carnegie Mellon University revealed (pdf) that Allegheny County's Family Screening Too l recommended investigating two-thirds of Black children reported, compared to about half of all other children, raising concerns about it's accuracy and tendency for bias. The researchers also found (pdf) that social workers disagreed with the risk scores the algorithm produced about one-third of the time. Allegheny County officials dismissed the research as 'hypothetical', based on old data, and noted that social workers can always override the tool, which was never intended to be used on its own. The findings prompted questions about whether the tool improved decision-making or reduced racial disparities in child welfare investigations. It also surfaced frustrations about the secrecy surrounding the algorithm and how risk scores are calculated - something considered important given the sensitivity of the tool and its impact on the lives of children and their families. ➕ June 2022. Oregon dropped its Allegheny County's Family Screening Too l-inspired Safety at Screening tool. ➕ January 2023 . T he AP reported that the Allegheny algorithm could harden bias against people with disabilities, including families with mental health issues. System 🤖 Allegheny County Family Screening Tool Operator: Allegheny County Children and Youth Services Developer: Rhema Vaithianathan; Emily Putnam-Hornstein; Irene de Haan; Marianne Bitler; Tim Maloney; Nan Jiang Country: USA Sector: Govt - welfare Purpose: Predict child neglect/abuse Technology: Prediction algorithm Issue: Accuracy/reliability; Bias/discrimination - race, ethnicity
- Date it happened
- 2022-04-01
- Organisation involved
- Allegheny County Children and Youth Services
- Product, system or model
- Allegheny Family Screening Tool (AFST)
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.