Allegheny child neglect screening tool may harden bias against people with disabilities
Concerns were raised about potential discrimination against families with disabilities caused by a child abuse and neglect screening tool used by Allegheny County, Pennsylvania. The US Department of Justice is reportedly examining the tool over concerns that it may violate the Americans with Disabilities Act by discriminating against families with disabilities, including those with mental health issues. The algorithm incorporates data points tied to disabilities in children, parents, and other household members. This includes information from Supplemental Security Income (SSI) and diagnoses for mental, behavioural and neurodevelopmental disorders. Critics worry that including disability-related data could automate discrimination against families based on disabilities or other external characteristics, and lead to a higher likelihood of investigations for families with disabilities. Allegheny County officials maintained that the tool helps predict outcomes and that parents with disabilities may need additional support. They also state that the algorithm has been updated several times, sometimes removing disability-related data points. The debate highlighted a broader discussion about the use of algorithms in child welfare decisions, with supporters arguing for their efficiency and critics warning about potential biases and discrimination. System 🤖 Allegheny County Family Screening Tool Operator: Allegheny County Children and Youth Services Developer: Rhema Vaithianathan; Emily Putnam-Hornstein; Irene de Haan; Marianne Bitler; Tim Maloney; Nan Jiang Country: USA Sector: Govt - welfare Purpose: Predict child neglect/abuse Technology: Prediction algorithm Issue: Accuracy/reliability; Bias/discrimination - disability Regulation ⚖️ The Americans with Disabilities Act Investigations, assessments, audits 🧐 AP (2023). Not magic: Opaque AI tool may flag parents with disabilities
- Date it happened
- 2023-01-01
- Organisation involved
- Allegheny County Children and Youth Services
- Product, system or model
- Allegheny Family Screening Tool (AFST)
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.