Of the thousands of calls to the state’s and Allegheny County’s child protection hotlines, alleging potential child neglect, call screeners often have to make life-altering decisions without much information.
Without the resources to investigate every single call, which ones should be prioritized? While all calls alleging physical or sexual abuse are investigated, other allegations, such as those pertaining to unsafe or unsanitary living conditions, can be harder to gauge.
Allegheny County human service officials and a team of researchers have created the Allegheny Family Screening Tool -— a computer tool that aims to help child protection staff make better decisions. Child welfare officials say they want to make sure they are focusing their limited resources on the most at-risk children.
The so-called “predictive risk modeling tool” uses more than 100 variables -— things like previous child welfare involvement and previous criminal justice system involvement — to generate a score from one to 20, with 20 being the highest risk, to predict the likelihood of a re-referral or home removal for a child.
This score is then used to screen calls in (meaning they are investigated) or out (meaning they are not).
The screening tool is only used for calls pertaining to general protective services issues, not child protective services calls that involve alleged physical or sexual abuse — by law, all these calls must be investigated.
General protective services calls involving non-abuse cases still warrant an assessment of the child’s well-being. Such calls would be for inappropriate supervision, unsafe or unsanitary living conditions, truancy or substance abuse by the parent. In 2016, the agency had more than 10,000 phone calls for general protective services issues. These issues can sometimes point to serious problems for children in a family, but that’s not always the case. Sometimes they are due to problems of poverty and are not necessarily indicators of abuse.
Child protective services calls involve allegations that appear to rise to the level of legally defined child abuse, such as physical or sexual.
Allegheny County officials emphasize that the algorithm they are using is not intended to replace human decision-making, but to assist and improve it.
Having some kind of tool to assist with child welfare decisions at call screening is not unusual, though the volume of data available locally and the algorithm is fairly unique, according to Allegheny County officials.
(Click image for larger version)
The tool pulls information from the Allegheny County Department of Human Services’ “Data Warehouse” — social service data representing 29 different human service program areas, with more than 640 million records for about 1 million distinct clients. It includes data on child welfare services, mental health services, drug and alcohol services, juvenile justice involvement, jail involvement, public benefits and information from the largest local public school districts.
If a call is investigated, the score generated is not shared with investigators who might go to the home, so as not to bias them.
“What we're really worried about is harm to kids, serious abuse and neglect. There's not a perfect predictor to predict for that,” said Erin Dalton, deputy director, Office of Data Analysis, Research and Evaluation for the Allegheny County Department of Human Services. “The researchers built the best model based on our data to predict those two things [re-referral and home removal].”
Researchers Rhema Vaithianathan of Auckland University of Technology in New Zealand, Emily Putnam-Hornstein from the University of Southern California, Irene de Haan from the University of Auckland, Marianne Bitler from University of California-Irvine, and Tim Maloney and Nan Jiang from Auckland University of Technology, used historical data for all child protective services and general protective services referrals made to Allegheny County from September 2008 to April 2016 to build the computer model.
The tool was rolled out in August.
Local child welfare advocates generally have praised the county’s transparency and approach to gathering input from others about how to best implement the changes.
“This wasn't just pronounced. It was vetted with a lot of advisory groups and community groups,” said Scott Hollander, executive director of KidsVoice, which advocates for children involved in the child welfare system. “I was at a number of those meetings and heard the comments, and I think people took them seriously.”
“I think they did a really good job of making sure it was done the right way,” by seeking community input, said Dr. Rachel Berger, Chief Division of Child Advocacy at UPMC Children's Hospital.
Officials also said they are working to minimize whether the data might inadvertently contain biases such as racial or economic disparities.
“People have raised concerns about those things. We raised those concerns ourselves,” Ms. Dalton said. Researchers have said they will conduct regular quality assurance checks to see how the tool is performing, and that it could evolve with additional data sources.
The tool did not require state approval, Pennsylvania Human Services Secretary Ted Dallas said, but officials were aware of what Allegheny County was working on and supportive.
“I think all of us are looking to use data to better guide decision-making processes,” Mr. Dallas said.
Cathleen Palm, founder of the Center for Children's Justice, based in Berks County, said if the tool is leading to better safety outcomes for children in Allegheny County, it would be good to find a way to apply it elsewhere in Pennsylvania, though Ms. Palm said realistically, most other counties do not have the same level of integrated data as Allegheny County does.
“You would be hard-pressed to find this in other communities, particularly on this scale,” she said.
The total cost for research and development was $781,073, funded through the DHS and the Richard King Mellon Foundation. An additional $224,858 in public and private dollars are funding evaluations.
Kate Giammarise: kgiammarise@post-gazette.com or 412-263-3909; Twitter @KateGiammarise.
First Published: April 9, 2017, 4:00 a.m.
