Q&A with Kumasi Vines, Baltimore Public Schools, Sara K. Cooper, the Annie E. Casey Foundation and Bi Vuong, Project Evident

Project Evident recently hosted a conversation with Baltimore City Public Schools’ Kumasi Vines, the Annie E. Casey Foundation’s Sara K. Cooper, and Project Evident’s Bi Vuong to discuss how Baltimore City Schools worked with Project Evident to use data and evidence to improve their CTE program outcomes, in order to ensure students have access to life long living wage employment. The piece below represents highlights from the webinar. The full Actionable Evidence case study is available here.

KELLY FITZSIMMONS: In 2020, two reports were released that showed quality and equity concerns with CTE programming in Baltimore Public Schools. Students in higher-performing schools had access to the most rigorous career readiness pathways, while students at schools with lower levels of academic achievement had access to pathways associated with lower-level job opportunities. BCPS partnered with Project Evident to create a data-driven strategic plan to improve its CTE programming and reduce these inequities. To do this work well, Baltimore City Public Schools held nearly 120 community meetings with school stakeholders in order to begin the implementation phase of the work. Could you speak to some of the capacities or behavior shifts that came from this process of hearing from stakeholders and sharing back on the project, as it was unfolding?

KUMASI VINES: Community engagement was key to ensuring that my office wasn’t making a decision for a district wide body of students by itself. We held meetings with key stakeholders, including teachers, students and families, local businesses and the community at large, to gain an understanding of their needs but also to communicate where we wanted to go to move forward.

We wanted our staff to understand why we were planning to make decisions based on what the data was telling us, while communicating that it isn’t about blame, it’s about understanding where we are and what strategies we will use to move forward positively and together. We had some very difficult conversations, because based on labor market demand and student interest, we had to end some programs while expanding others. But we worked to be transparent in making those decisions. 

“We wanted our staff to understand why we were planning to make decisions based on what the data was telling us, while communicating that it isn’t about blame, it’s about understanding where we are and what strategies we will use to move forward positively and together.”

Kumasi Vines, Baltimore City Public Schools

And we, of course, involved students and their families. This shed light on the realities of changing programming for students and enabled us to take their feedback to heart, so that  student engagement could help inform decision making. We also involved the business community to ask: What are you looking for in an employee? What skills do they need? What types of training and certifications are directly correlated to high paying jobs in your industry? We didn’t want to guess at the answers to these questions, and it informed our decision making. 

We also involved the community at large, letting them know what we are trying to do and why we’re doing it: to help better our students. 

KELLY FITZSIMMONS: This project was a little unique in how the funding was structured. The Casey Foundation provided funding to The Fund for Educational Excellence which then provided funding and infrastructure support to City Schools. This provided City Schools with more autonomy in the work, having the ability to choose the right partner for the project and had the ability to hire or fire that partner if they did not meet City Schools’ expectations. Sara, were there moments in which it was challenging for the philanthropic community to stick with the work or was there anything that was different about this from your perspective?

SARA COOPER: Seeing the incremental progress that the district has made after each quarter has been incredible to listen to. Funders typically ask what can we do? And how much would that cost? But recognizing that the process is the process, all of those community engagement meetings had to happen, for all the reasons that we have mentioned. So we did have to put the brakes on a little to figure out the right approach. We’ll get there, we just have to trust the process. 

We are pleased to continue to support City Schools’ other needs that have come up. One in particular is if City Schools need an intermediary with some additional capacity to support work based learning, apprenticeship, things related to CTE. We’ve been able to provide additional support to City Schools to help them think through if an intermediary partner could be supportive of this sort of ecosystem beyond even CTE that they’re building out. Because you do need to address capacity issues as they come up.

KELLY FITZSIMMONS: Coming back to implementation – Bi, were there questions you and the Project Evident team were confronted with that were a little different than you thought they would be going in? And were there questions you ended up needing to ask and explore as a part of that?

BI VUONG: A couple of things came up during the implementation of the engagement between City Schools and Project Evident. Community engagement was not part of the original scope of work, but it became clear through the process that there was a need to engage the community, and as such, we were able to provide City Schools with the support they needed. We both agreed that we’ll figure out the money later. We were able to do this because of the trusted relationship between all parties involved. Without this, the original scope of work wouldn’t have evolved the way that it did. We were able to lean-in to what City Schools needed at that time. 

Another area where we provided support that was not originally envisioned was in the area of human capital. To ensure that resources were used effectively, we looked into whether enrollment was aligned with teacher allocation. Specifically, we worked with City Schools to answer: Where are we over-resourced, where are we under-resourced and how are we meeting student demand? We didn’t have this data during the initial round of data transfer, however, because of the nature of our contract, City Schools was able to transfer the necessary data to us to help them with the analysis.

“One question we hear a lot is why are you doing this? Why now? Why are you doing so much? Why not wait and see? The simple answer to that is our kids can’t wait and see. We have to do it now; we have to act, even if it leads to some uncomfortable conversations. The beauty of it is we have the data to support not just our actions but we also have that engagement piece with the community.”

Kumasi Vines, Baltimore City Public Schools

KUMASI VINES: One question we hear a lot is why are you doing this? Why now? Why are you doing so much? Why not wait and see? The simple answer to that is our kids can’t wait and see. We have to do it now; we have to act, even if it leads to some uncomfortable conversations. The beauty of it is we have the data to support not just our actions but we also have that engagement piece with the community.  It’s not just mandates coming out of a shadow office, it is transparent: here is what we’re thinking. And understanding that there may be emotion involved, even if the data says one thing, and while we may not necessarily agree on the plans all the time, we are ultimately guided by our North Star. 

Another question we were asked a lot is how are you going to fund this work? The strategy that Project Evident helped us develop as we mentioned earlier, kept us primarily budget neutral, at least with the funds we were requesting from the district. And if you’re making changes while remaining budget neutral as a school district, then you’re doing something great. When we go to other funders for needs outside of our internal funding, we’re prepared; we have data, we have an implementation plan to show we’re not just asking for money to try something random; we have a plan. 

“When we go to other funders for needs outside of our internal funding, we’re prepared; we have data, we have an implementation plan to show we’re not just asking for money to try something random; we have a plan.“

Kumasi Vines, Baltimore City Public Schools

KELLY FITZSIMMONS: There is often an unnecessary tension with research and evaluation work, especially at R1 universities (institutions with very high research activity) or larger research firms, between rigor (strict adherence to research methodology) and practical operational applications (i.e. budget, staff allocations, physical infrastructure, etc). We often see departments (i.e. operations, finance, evaluation) largely siloed from each other. With this work with City Schools and Project Evident, there was integration across multiple areas to be able to problem solve effectively and strengthen the CTE program. Where do you see these dynamics playing out? Will they strengthen the work or cause tensions in it?

BI VUONG: Improvement does not occur in a vacuum and so the work to improve cannot either.  Fundamentally, our work is focused on how do we help a school district think through not only how it is currently performing, but what policy, practice, and resource changes does it need to make changes to improve student opportunities and outcomes? Therefore, while we always look at student performance we also consider whether the strategy a district seeks to pursue can be implemented. For example, City Schools may have wanted to open a new program in one of its schools to ensure more equitable access to certain pathways. However, opening a new program requires City Schools to look at its facility, resources, and timeline for implementation.  Even if its facilities could fit a new program, does it have the $350,000 to pay for renovation costs, and can the improvement occur before school starts?  We have all of these outcomes we hope to achieve, but in order to get there we have to consider other factors at play like finance and operations. Successful implementation requires us to reduce the fragmentation and friction that tends to exist within a system.

KUMASI VINES: The way we bucketed out the work with different work streams, it makes it easier to understand that even if we want to remain siloed we can’t. Because all of these things tie together whether it’s the facilities, the budget, the human capital impact, or professional development. If we change the pathways, we change the curriculum, we need to make sure that teachers understand how to deliver the curriculum. So all of these things tie together internally and I think that has forced us to really embrace this as an office. 

For us, equity and access is so important and we have been collaborating with our Special Ed Office and Our English Language Learners Office.

KELLY FITZSIMMONS: Are you thinking about what is next? For some of these core elements of the work, do some stand out more than others? And how are you thinking about building on them in the next phase of the work?

KUMASI VINES: We have our financial plan for the next few years and a yearly plan that outlines  how we make changes, but the world doesn’t stop. Mandates from the state don’t stop and our kids are in programs currently so we have to be nimble, we have to adjust in order to get to the end goal for our programming. 

We are also deepening our connection in our attention to our special populations of students. For three of our largest concentrations of CTE programming, the district removed the entrance criteria for those schools at the beginning of the school year. So we’re projecting drastic changes in the demographics, meaning we need to work side by side with our colleagues that provide services for our special populations of students because whatever the demographic is, we want to make sure our students are successful in our programming. We are also looking to continue to engage with the industry, to ensure they understand our plan and are informing us of changes on their end, so we can make sure we’re translating that in the classroom.

KELLY FITZSIMMONS: Is that a capacity you built through this project? Or were there new lessons that you are carrying forward?

KUMASI VINES: We’re putting a lot of effort into growing our work based learning opportunities, internships and apprenticeships. When it comes to how we’ve grown, I think intellectually and operationally we are thinking differently. We are taking a step back and seeing the big picture and how things connect together and we are engaging as we should be. But also understanding the limitations. I have a small team and our programming is still current, meaning we’re doing all of these things on top of what we already were doing before and it stretches people thin. There is only so much I can ask the team. But there isn’t the space to create four or five more positions in the office so that is why working with Project Evident closely has been so vital. It has helped us focus more on our mission, while identifying other areas that are connected to what we want to do but where maybe we aren’t the experts and shouldn’t be. It really just highlighted those areas of our need, limits and capacity, and started identifying ways to address them.

“But there isn’t the space to create four or five more positions in the office so that is why working with Project Evident closely has been so vital. It has helped us focus more on our mission, while identifying other areas that are connected to what we want to do but where maybe we aren’t the experts and shouldn’t be. It really just highlighted those areas of our need, limits and capacity, and started identifying ways to address them.“

Kumasi Vines, Baltimore City Public Schools

KELLY FITZSIMMONS: Can you say a little bit more about how you’re tracking student outcomes at the school level? And how successful you’ve been at collecting real time data on enrollment persistence and credential entertainment and then, finally, how that information has led to changes in strategy to increase student success? 

KUMASI VINES: Data is another one of our work streams. We are mandated to report on CTE programs to both the district and the state anyway. Through our work with Project Evident, we realized we need to make sure we’re building the infrastructure for data analysis internally, asking the right questions and pulling the right data in order to inform not only our long term strategy but also our day to day operations. To better inform how we can support our students, how can we support teachers and how can we provide instructional support? So we’re internally building that infrastructure to think on the instructional side and the teacher side, but also on the funding side to help inform decisions on how we are going to spend our very limited financial resources.

KELLY FITZSIMMONS: Kumasi, this work is relatively recent. Can you share a few indicators of success that are building confidence that you are on the right path towards the student success outcomes?

KUMASI VINES: Some of the earlier indicators have been the engagement of the business community, and the multiple opportunities to increase and enhance work-based learning. The excitement in both our teachers and students is another one. Letting them know we’re in this together with them, and as time goes on you can fill in whatever is missing and you will get that. And having the data and metrics to confirm that what we’re betting on now is going to pay off in the future. There is also anecdotal stuff– just the energy and feedback we’re getting about how we move forward.

“The excitement in both our teachers and students is another one. Letting them know we’re in this together with them, and as time goes on you can fill in whatever is missing and you will get that. And having the data and metrics to confirm that what we’re betting on now is going to pay off in the future.“

Kumasi Vines, Baltimore City Public Schools

KELLY FITZSIMMONS: Sara, what is next? How are you thinking about the future?

SARA K. COOPER: There are two things. One is just the continued thought partnership with Kumasi and his team. Just being there as an ongoing support to whatever City Schools needs. We’re more than happy to do that. And we’re thinking about how we can influence the whole career readiness ecosystem that City Schools is trying to build. It is so critical to Baltimore’s workforce ecosystem and our economy, so how to influence and get the positive word out in different spaces and at different tables. I’m sure in so many communities it can be very, very siloed. But we’re trying to draw the connections and exert our influence that way, to bring everyone together.

KELLY FITZSIMMONS: What was your most powerful learning moment that you would offer others as they think about their own data and evidence journeys?

SARA K. COOPER: I think it’s already been shared, but I just want to emphasize the combination of quantitative and qualitative data. And how important it is to have storytelling and as a funder that’s really important to not make funding decisions, or really any decisions, in a vacuum. 

KUMASI VINES: Another thing we learned is how to take a step back, look at the bigger picture. And while keeping that bigger picture in mind, we can start focusing and narrowing down when we need to and then just understanding the reality of capacity or capacity building. Building internal capacity required asking for additional help and just moving through that process. A lot of folks on my team want to save the world. And we all will one day, but you get a better understanding, a more realistic understanding about capacity and limitations. 

BI VUONG: It was really nice to have a cash flow from Project Evident that enabled my team to do things in good faith with Baltimore City Public Schools, regardless of contracting timelines. It’s also important to have a partner who is willing to be “in it” with you. We might disagree but we acknowledge that our back-and-forth is done to figure out what might be best for students. We go into the work knowing that we may have to pivot to achieve the outcome we want to achieve. Transparency and honesty is core to the relationship.