Predictive Risk Models in Child Welfare

Since 2016, the Allegheny County Department of Human Services (DHS) in Pennsylvania has utilized the Allegheny Family Screening Tool (AFST), which assists child welfare call screening caseworkers in their assessment of general protective service (GPS) referrals regarding potential child maltreatment.

What is this report about?

This report reviews the research evidence on algorithms in child welfare, specifically focusing on the causal impacts of the AFST and comparable predictive risk models (PRMs). It begins by summarizing the influence of these tools on child welfare decisions. The report then explores the discrepancies between perceived and actual effects of these models, highlighting the importance of bridging the gap between perception and reality to alleviate concerns and maximize the effectiveness of these tools.

What are the takeaways?

The impacts of predictive risk models in child welfare must be compared with alternative approaches to augmenting call screening caseworker decision-making. Traditional risk assessments in child welfare have been largely manual, prone to inconsistencies, and often omit critical information. Before implementing the AFST, Allegheny County did not employ any structured risk assessment.

The main conclusions from recent research on the AFST are:

The AFST changed the composition of investigated referrals. The introduction of the AFST decreased the probability of investigation for referrals with low risk of removal and increased the probability of investigation among referrals with high risk of removal. The introduction of the AFST also reduced the racial gap in investigation rates, particularly among higher risk referrals.
The AFST is reducing, not increasing, racial disparities. Researchers found that the introduction of the model reduced racial disparities in investigation rates across AFST scores, although the size and precision of the reduction varied. The AFST reduced the racial disparity in investigation rates for the highest risk referrals by 83%, from 10.6% to 1.8%. The researchers estimated that the AFST reduced the Black–White gap in removal rates of screened-in referrals by 73%, from 4.3% to 1.2%.
Screeners use the algorithm but with caution. Researchers found that call screening case workers are integrating information from the AFST effectively, aligning their decisions more closely with predicted removal risk compared with the period before the tool’s introduction. The tool is seen as a helpful source of additional information rather than a replacement for professional judgment.

How is this report being used?

The application of algorithms to support decision-making, especially in sensitive areas like child welfare, mandates high transparency. It is critical that the complexities of predictive risk models are communicated clearly to all stakeholders to maintain trust and prevent misuse. DHS is committed to keeping the public informed about the use and impact of algorithms at the Department and draw upon current research to shape the implementation of these tools in the field.