Tiered-evidence frameworks were designed to help public and private funders make evidence-based policy and resource allocation decisions. Generally speaking, tiered frameworks are used to award larger investments to organizations and programs with mature evidence of effectiveness to help them replicate and scale, while those with earlier-stage evidence receive smaller awards to help them test, innovate, and develop. As a part of the evidence-based policy movement — which started under the George W. Bush administration and grew during the Barack Obama administration, as well as among private philanthropies — tiered frameworks have been increasingly adopted. 

However, it is time to modernize. While tiered-evidence frameworks can provide useful decision-making guidelines, their top-down design and rigid definitions have resulted in artificially narrow considerations that can privilege select organizations or domains and constrain innovation – contributing to evidence building and use that is short-sighted and inequitable.

I’ve come to think of this problem as the “tier-anny of evidence.”

Given the disruption caused by the ongoing pandemic – the toll it is taking on students, teachers, and front-line workers, and the challenges it is posing to funders and policymakers – now is an optimal time to reconsider whether tiered evidence frameworks can address our most pressing challenges. Real-world conditions demand a more nuanced and practical approach to building and using evidence, which in turn means we must consider more diverse definitions of what counts as evidence. This is not to dilute integrity, rigor, or quality, but rather to widen the aperture so that more surgical and less blunt tools may be applied. Otherwise, we are limiting who participates in innovating and risking a collective loss of knowledge.

Having worked on both public and private funding decisions using tiered-evidence frameworks, I can say frankly that they limit our inclusivity, our ability to engage in real testing and learning, and our opportunity to support a diverse set of leaders and organizations. Quite simply, most tiered-evidence frameworks were not designed with equity, research and development, and practitioner-centric evidence building in mind. They do not provide capacity-building support to organizations without evidence (or the right kind of evidence). They prioritize funding a narrow set of existing interventions rather than strategically and intentionally supporting innovation and focusing on areas where evidence is necessary but nascent. And they are unable to factor in important structural, financial, and systemic considerations. As we have learned from COVID-19 — where evidence was needed to make swift, actionable decisions to support those hardest hit by the pandemic — sometimes our innovation and problem-solving contexts require us to take a more nimble, flexible, and holistic approach in order to make more equitable decisions.

The Families First legislation passed in 2018 illustrated the pitfalls of trying to apply a tiered-evidence framework to an area with nascent evidence. Families First set out to fundamentally rethink the child welfare system and provided funding based on evidence tiers. The program encouraged child- and family-serving federal agencies, funders, and a diverse set of community stakeholders to work together in new ways. But for that very reason, many organizations and programs lacked established evidence to meet the eligibility requirements. The tiered framework was a ladder without lower rungs, leaving many organizations without the ability to climb it. When we do not create space for joining and ascending, and rely instead on the established and dominant, we limit our ability to develop new solutions to our most pressing challenges.

Without innovating on contemporary definitions of evidence – of what counts for decision making – we risk a “tier-anny of evidence” that can perpetuate an inequitable distribution of resources and a skewed understanding of evidence building. In this scenario, funders and policymakers continue to award grants to large, well-funded organizations able to meet higher evidence tiers without a close assessment of what other evidence might be generated to help address challenges – especially those that are emergent and gnarly. Practitioners fall into a “just get me on the list” mentality where evidence is built primarily to meet funding eligibility or compliance requirements, rather than used to learn, improve, and achieve stronger outcomes for program participants. And both practitioners and funders adopt a mindset where evidence building is thought of as a linear process, rather than one that is constantly adapting to changing needs and contexts in order to achieve actionable, relevant information for communities.

We must broaden our thinking around how we use data and evidence beyond the constraints of tiered frameworks, and to reconceive evidence tiers with a stronger equity focus and in service of actionable evidence building. The antidote to the “tier-anny of evidence” is inclusivity and practicality — broadening what we mean by evidence, being clear on its purpose, considering cost, and expanding the list of who gets to participate in the evidence-building process. As the Equitable Evaluation Initiative points out, “interpreting and drawing conclusions from data without the participation of those engaged in and affected by the work inequitably takes ownership of knowledge and decision making power out of their hands.”

It’s on all of us to take advantage of new opportunities created by the Biden administration to expand the use of data and evidence across government, including entitlement, formula, and competitive grant programs. Laudable work is currently underway by federal agencies to build learning agendas and evidence plans to comply with the Evidence Act. The Biden administration’s executive orders on racial equity will strengthen this work by intentionally building equity into Evidence Act requirements. Together, these steps will lead to stronger use of data and evidence to drive more equitable spending and outcomes across sectors.

For policymakers and public and private funders who are building new tiered evidence programs, we offer four suggestions to make them more inclusive, equitable, and rigorous:

Include evidence reviews and needs assessments as criteria for lower-tier funding. Most early-stage organizations have not yet systematically collected the evidence needed to substantiate hypotheses around program impacts, nor identified the appropriate set of metrics or outcomes to be studied. Furthermore, tiered frameworks assume fund-seeking organizations already know what interventions and supports are needed by the communities in which they work. While many tiered frameworks require a logic model for the lowest tiers, we believe that they should also require an evidence review and a needs assessment that meaningfully incorporates input from program participants. This will encourage practitioners to begin using data and evidence at a program’s earliest stages and will better allow funders to judge the strength of a program’s potential.

Provide technical assistance focused on strengthening support throughout the continuum, with added resources for early-stage evidence-building capacity of practitioners. By providing supports to get more organizations onto the on-ramp — rather than only funding organizations with the highest level of existing evidence — we can build a more inclusive pipeline of programs that better represent the needs and voices of communities. We can also deliberately counter patterns in which resources flow disproportionately to leaders, organizations, and communities that are already more highly resourced. This technical assistance could focus on developing learning agendas and theories of change, data collection and systems, data analysis and use, capacity assessments, or other needs.

Strengthen tiers by requiring that proven impacts have practical significance. For stronger and more equitable outcomes for communities, we must encourage practitioners to build actionable evidence that is not focused primarily on academic relevance but on community impact. Practicality should be a principle throughout all tiers, and top-tier funding decisions should not be based solely on statistical significance without taking into account community context and practical relevance to policy and practice.

For all tiers, policymakers should explore methods and funding options to help organizations keep testing and learning in order to avoid a “one-and-done” mentality. Evidence goes stale, and to stay relevant to communities, evidence building must be a continuous process — think R&D, not compliance — that supports practitioners and policymakers to innovate and meet emerging needs.

Importantly, these proposed revisions do not discard rigor, but rather attempt to reclaim rigor with practical relevance and credibility. Rigor and credibility are important in all evidence building, whether the evidence is used to inform public spending or to support continuous improvement. But for too long, we have falsely equated rigor with randomized control trials (RCTs) alone, when in reality, rigor applies throughout the tiers, from early-stage evidence gathering to large-scale evaluation design and implementation. We believe that practitioners should strive to build evidence that is seen as credible not just in the eyes of researchers, but also in the eyes of those who are most proximate to the challenges being addressed and often the ones providing the data.

At Project Evident, we are committed to building a more diverse pipeline of evidence-based programs that provide meaningful solutions for communities, along with more consistent and broader use of data and evidence to improve social and education programs. Through our Next Generation of Evidence Campaign and Actionable Evidence Initiative, we’re advancing a new approach to evidence building that is practitioner-centric, continuous, practical, equitable, and inclusive. We are currently developing an Equitable Recovery Wallet concept to support actionable technical assistance in data and evidence for those with limited access. And we are not alone – Equitable Evaluation Initiative, PolicyLink, We All Count, and others are working to center equity and community voice in how we gather and use evidence.

Tiered-evidence frameworks made evidence a central component for funding and policy decisions. However, the frameworks’ rigidity and lack of focus on equity — the “tier-anny of evidence” — have perpetuated funding inequities and resulted in a number of unintended consequences for practitioners. To unlock the potential of evidence to guide stronger and more equitable decision-making, we must ensure our ladders include more lower rungs so that more can participate. The time has come for a next generation of evidence that expands on what we’ve learned, intentionally centering community needs, broadening the definition of what counts as evidence, and developing more inclusive and rigorous criteria in our frameworks to allow a wider array of voices to participate in evidence building.