Case Study
Generating On-Demand, Actionable Evidence: First Place for Youth and Gemma Services
Download PDF
First Place for Youth helps youth who have aged out of the child welfare system build the skills they need to make a successful transition to self-sufficiency and responsible adulthood, and Gemma Services is a social service agency that operates a long-term residential psychiatric care program for youth. A few years ago, both organizations realized that their administrative data systems, while extensive, generated little data that could be used by front-line practitioners as they worked directly with youth and families.
To address this limitation, First Place and Gemma partnered with BCT Partners, an evaluation and data science firm, to institute “precision analytics,” which trains machine learning algorithms to build predictive, prescriptive, and evaluation models that can be technologically implemented to generate actionable evidence, on demand. Using program administration and case assessment data, this process learns from differences in outcomes among similar youth who experienced different treatment patterns. The goal of this process is to understand what has worked for specific participant populations in the past and then to translate those findings into actionable information for front-line staff so that they can connect each youth with the services and supports that are most likely to lead to success.
This case describes the steps the partners at both organizations took to develop their learning systems and user dashboards; the resources and capacity required, including the roles of evaluators, practice experts, data scientists, and practitioners; and the resulting changes in practice for front-line staff, managers, and leadership. It also addresses steps the partners took to minimize bias in the development of algorithms.
This case is one in a series commissioned by the Actionable Evidence Initiative in 2020 and 2021. The series illustrates how researchers, evaluators, practitioners, funders, and policymakers across the country are exemplifying principles of the Actionable Evidence framework. It profiles a range of settings, actors, learning questions, methods, and products, unified by a commitment to practitioner-centered, timely, practical, equitable, and inclusive evidence building. Each case describes the origins, development, and results of a research or evaluation project, along with the authors’ reflections on their experiences. Our hope is that these cases will provide both inspiration and practical guidance for those interested in generating and using evidence that leads to better and more equitable outcomes for youth and communities.