Predictive Risk Models in Child Welfare

Current information

Since 2016, the Allegheny County Department of Human Services (DHS) in Pennsylvania has utilized the Allegheny Family Screening Tool (AFST), which assists child welfare call screening caseworkers in their assessment of general protective service (GPS) referrals regarding potential child maltreatment.

What is this report about?

This report reviews the research evidence on algorithms in child welfare, specifically focusing on the causal impacts of the AFST and comparable predictive risk models (PRMs). It begins by summarizing the influence of these tools on child welfare decisions. The report then explores the discrepancies between perceived and actual effects of these models, highlighting the importance of bridging the gap between perception and reality to alleviate concerns and maximize the effectiveness of these tools.

What are the takeaways?

The impacts of predictive risk models in child welfare must be compared with alternative approaches to augmenting call screening caseworker decision-making. Traditional risk assessments in child welfare have been largely manual, prone to inconsistencies, and often omit critical information. Before implementing the AFST, Allegheny County did not employ any structured risk assessment.

The main conclusions from recent research on the AFST are:

  • The AFST changed the composition of investigated referrals. The introduction of the AFST decreased the probability of investigation for referrals with low risk of removal and increased the probability of investigation among referrals with high risk of removal. The introduction of the AFST also reduced the racial gap in investigation rates, particularly among higher risk referrals.
  • The AFST is reducing, not increasing, racial disparities. Researchers found that the introduction of the model reduced racial disparities in investigation rates across AFST scores, although the size and precision of the reduction varied. The AFST reduced the racial disparity in investigation rates for the highest risk referrals by 83%, from 10.6% to 1.8%. The researchers estimated that the AFST reduced the Black–White gap in removal rates of screened-in referrals by 73%, from 4.3% to 1.2%.
  • Screeners use the algorithm but with caution. Researchers found that call screening case workers are integrating information from the AFST effectively, aligning their decisions more closely with predicted removal risk compared with the period before the tool’s introduction. The tool is seen as a helpful source of additional information rather than a replacement for professional judgment. 

How is this report being used?

The application of algorithms to support decision-making, especially in sensitive areas like child welfare, mandates high transparency. It is critical that the complexities of predictive risk models are communicated clearly to all stakeholders to maintain trust and prevent misuse. DHS is committed to keeping the public informed about the use and impact of algorithms at the Department and draw upon current research to shape the implementation of these tools in the field.

Read more about AFST here.

Generating a Fully Synthetic Human Services Dataset

In 2022, staff at the Urban Institute partnered with the Allegheny County DHS and the Western Pennsylvania Regional Data Center (WPRDC) to pilot synthetic data generation at the local level, to help understand the unique challenges that might face state and local governments in generating synthetic data. Each record in the synthetic dataset represents a simulated individual, or record, who received at least one service from the Allegheny County DHS in 2021. The synthetic data were designed such that records aggregated by service represent the original data. Read more here about synthetic data.

Why create a synthetic dataset?

The Department of Human Services (DHS) in Allegheny County, Pennsylvania, serves one in five residents of the county every year through child welfare services, behavioral health services, aging services, developmental support services, homeless and housing supports, and family strengthening and youth supports. In the process, data are collected about these services and the population using them. These data are integrated at the individual level to allow for better care coordination, operational improvements, and program evaluation. Because of the dataset’s sensitive nature, it cannot be widely shared at an individual level, so synthetic data are used in the real dataset’s place—allowing the data to be publicly shared and helping stakeholders, including researchers, service providers, and members of the public, understand these populations better.

From June 2018 to December 2020, the Urban Institute conducted a systemwide assessment of the system response in Allegheny County, PA to intimate partner violence (IPV) to better understand the system as a whole and operations of some key agencies

What is this report about ?

Urban Institute presents the findings from their systemwide assessment. The goals of this assessment were to 1) examine how IPV cases enter the justice and child welfare systems in Allegheny county, 2) analyze agencies’ processes for responding to IPV and 3) recommend ways the county can improve responses to IPV.

What are the recommendations?

  • Have county leaders prioritize IPV
  • Shift the focus from case outcomes to people’s experiences, especially during early encounters with formal services.
  • Reinstate and sustain IPV-focused fatality reviews and ensure they embrace a non-blaming culture.
  • Establish a specialized IPV unit in the Allegheny County Public Defender office
  • Differentiate IPV from DV throughout all systems.
  • Record survivor information consistently and securely share it when possible.
  • Prioritize and improve referrals to batterers’ intervention programs
  • Create a mechanism to consistently track aggressors’ and survivors’ experiences at system entry points.

 

How is this report being used?

The county executive and Mayor of the City of Pittsburgh created an IPV Reform Leadership task force in May 2022 to actively work on addressing these recommendations and improving the system.

In July 2013, the Center for the Study of Social Policy (CSSP) and the Allegheny County Department of Human Services (DHS) launched a partnership to better support child welfare-involved youth achieve healthy sexual and identity development. This institutional analysis prepared by CSSP used data analysis, case reviews, and interviews to understand current experiences of LGBTQ+ children and families who interact with child welfare as well as cultural and practice changes that have occurred since the initiative began.

Click here to read the report.

How is the Allegheny County Department of Human Services (DHS) able to link data across programs and with providers, given privacy law restrictions?

Erin Dalton, Director of DHS, and Brian Bell, a supervisor and privacy officer, share their insights on the Gov Innovator podcast. Hosted by Andy Feldman of the Brooking’s Institution, the podcast shares insights and strategies from leaders in the public sector. This talk builds on an earlier conversation with Dalton about the Data Warehouse.

Click here to access the interview.

U.S. Government Accountability Office

The U.S. Government Accountability Office examined four innovative data sharing practices (including DHS’s) to determine (1) how selected states or localities have shared data across programs to improve the administration of human services, (2) challenges state and local human services agencies face in balancing privacy protections with greater data sharing, and (3) actions that the federal government could take to help address these challenges.

Click to read the full report. 

Addressing the specific needs of youth across multiple life domains as they transition from the children’s behavioral health treatment system to the adult treatment system is critically important. To help quantify these issues and contribute to discussions related to system interventions locally, Allegheny Health Choices, Inc. (AHCI) identified a cohort of youth who turned 17 between January 2007 and December 2009 and used behavioral health services while 17. This report describes their service use characteristics and involvement with other systems during their 17th year compared to their 18th year.

Click to read the full report.