August 2018 began the fourth academic year of OnTrack Greenville (OTG), the broad collaborative to boost middle grade success at Berea, Lakeview, and Tanglewood Middle Schools and Greenville Early College. It also began the freshman year of high school for the first group of students who’d experienced OnTrack Greenville for their full middle school experience, many of them at Carolina High School.
Fittingly, in August, Carolina High School came on board as a school using nearly the full OnTrack Greenville model, including the early warning and response system, School Based Health Center, and a Communities In Schools School Support Specialist for ninth grade. The start of the school year at Carolina with so many new partners and functions was smooth, thanks to all of the evaluation and learning about OnTrack Greenville over the last three years.
This mid-year report for funders explores the themes of leadership change, scaling and sustainability, evaluation, adaptation, and alignment.
Leadership change
Over the last three academic years, there have been various personnel changes within the OnTrack Greenville schools and partner organizations. Because of the attention to culture and partnership, these transitions have had a minimal impact on the work with students.
The changes in leadership at the start of 2018-19 are positively building upon the good work already underway.
In early 2018, Meghan Barp came on board as the President and CEO of United Way of Greenville County, OTG’s fiscal agent. Meghan came from Greater Twin Cities United Way, which was also a SIF grant recipient, therefore she understands the value and responsibility of having a SIF grant. She also has a depth of knowledge on Collective Impact and the significant role OnTrack Greenville plays in the future of United Way. This provides a level of leadership and support that will be of great value as we move forward.
Berea Middle School is now led by a new principal. Robin Mill, who’d served as Berea’s principal since the launch of OTG retired after successfully overseeing the first three years of implementation in that middle school. New principal Corey Collington grew up in Greenville County Schools and was previously assistant principal at Carolina High, so he deeply understands the goals of OTG and brings new passion to the work.
Scaling and Sustainability
The commitment to scale is a significant success. Not only has OTG expanded in its full form to Carolina High, the district has rolled out the Early Warning Data Systems and Early Warning Teams to all elementary, middle, and high schools because of its effectiveness in identifying which students are off track and possibly in need of intervention. Yet there are numerous decisions and considerations in how OnTrack Greenville is scaled. Since the data system and teams have been scaled to all schools, OTG partners are now considering which interventions should be scaled and to what schools. There is much to decide, including how to scale with fidelity, do we scale for volume or equity, and do we scale specific models or types of interventions? Further, we suspect that every school having an early warning team will increase the identification of students with challenges, therefore potentially straining the supports available through the non-profit community. There is then the question of what form of shared leadership model best supports this scaling and how will the current roles of backbone or stakeholders need to change or evolve to meet the scale. We have reengaged a consultant who helped with our stakeholder agreement to facilitate and guide us through this weighty decision-making process.
Our other challenge is related to keeping OnTrack Greevnille a priority and in the forefront of the minds of our funders and partners, as leadership changes and community priorities shift; we are always subject to losing momentum or having funding reallocated. Additionally, as new principals are brought on board, the time required to orient and work with them to ensure they fully embrace the model can be challenging. Principals appropriately have a vision for their schools, and we have to work diligently to ensure that they see OnTrack as a way to strengthen or deliver their vision, which requires constant and effective communication and relationship-building.
The other significant issue is funding sustainability. As scaling becomes the focus, we must also remember the necessity of sustaining the foundational work in our pilot schools while also developing a scaling plan with funding in mind. The United Way is excited to raise funds for OnTrack Greenville and there is potential to attract national funding. Nonetheless, these will be lofty goals to reach.
Evaluation
Data continues to be the driver for decision making and activities in OnTrack Greenville. Partners use their evaluation reports to guide program changes and build upon what’s working.
Promising findings in the last period include:
70% of students in BELL’s summer program reported positive change in social and emotional learning (as measure by the PEAR Holistic Student Assessment). BELL successfully met and exceeded all academic, social emotional learning, enrichment, and partnership goals.
97% of students going to the School Based Health Center returned to class and they were slightly less likely to seek medical treatment at the Emergency Room and more likely to seek primary medical care in the school health room.
More than 90% of Communities in Schools students worked on their academic, personal, and behavioral goals with program staff.
Teen Leadership class participants reported significantly stronger relationships with teachers and caring adults when compared to matched comparison students at non-treatment schools. They also reported higher levels of school engagement, school belonging, and valuing education. They were less likely to have had one or more in-school suspensions and had fewer total hours of in-school suspensions and fewer total hours of out-of-school suspensions.
Partners are now building their own evaluation capacity. Greenville County Schools, for instance, was puzzled by some of the findings in their evaluation report on the Teen Leadership classes. This spurred the Data and Quality Specialist to consult with the district’s internal Evaluation Specialist. In order to have greater impact, the district specialists facilitated workshop discussion to increase teacher capacity for evaluation. With guidance, the teachers generated ideas for what data they could collect and how they could collect that data. They identified that student portfolios can be used to collect data, in a way that is meaningful for students and that lends itself to qualitative analysis. For example, ideas for student portfolio artifacts that could be data sources include: teacher interviews of students; and, students identifying a specific behavior they want to improve, monitoring the behavior, and reflecting on their progress.
In late spring, The Riley Institute will publish interim findings from the 2017-2018 academic year and we look forward to sharing those findings with funders, as well as progress to date.
Adaptation
Thanks to the robust evaluation of process, implementation, and impact, partners have had data with which to plan and make decisions. As has been stated in previous reports, evaluation is a learning tool and not a report card. A few examples of adaption in OTG and partners:
Communities In Schools has conducted an internal restructuring to focus efforts, streamline tasks, and firmly account for the work required behind the scenes in support of their amazing work with students. Over the past five years, CIS has increased its operating budget 63% and its front line staff by 70%, but its administrative staff have not grown commensurately. Its restructuring gives it more capacity to address needs and allow for better continuity of programming from elementary to high school feeder patterns.
Tanglewood’s principal replaced one of the Teen Leadership class periods with a planning period for the instructor. This instructor has been tasked with strengthen the presence of the Capturing Kids’ Hearts processes to build a positive, student-centered culture throughout the school.
Greenville Health System used their evaluation data to restructure the consent form distribution and collection process, which resulted in the highest number of consents collected and the highest percentage of students at the OnTrack schools with a consent on file to date.
As was previously reported, Public Education Partners determined from its evaluation that it could be more impactful as a strategic partner to OTG rather than as an implementation partner. PEP and the United Way built a Memorandum of Understanding to further define how PEP can serve OnTrack as an intermediary. As a strategic partner for OnTrack, PEP can leverage its role and function as Greenville’s K-12 intermediary to provide knowledge, capacity, and thought-partnership to OnTrack to improve its collective practice, integration with Greenville County Schools, and planning for sustainability.
OTG backbone staff learned from process evaluations that there was a need to strengthen external communications about the collaboration. Thanks to a grant from the Jolley Foundation, OTG contracted with a communications professional to increase its social media presence, build awareness of OTG, and uniting our partners around shared marketing.
Alignment and Vision
Thanks to intentional work on establishing the collaborative, building trust, creating shared norms and values, and continuous work to sustain them, OTG’s partners have always been well-aligned. But now that they are facing the end of the initial five-year vision and the work has begun scaling, the partnership needs to revision the future beyond the SIF-driven goals and model.
As was mentioned on the first page in the section on scaling, we have brought in the facilitator to help us work through this re-visioning. Key stakeholders have met several times and have identified opportunities to leverage and challenges to mitigate in the scaling of OnTrack Greenville. Over the next six to nine months, stakeholders will engage in collaborative design processes to:
- Design a universal framework all Greenville County Schools can use to identify areas of need and align internal and external interventions to support students.
- Design a scaling plan for the OnTrack Greenville model focused on high-priority schools.
- Evolving the collective impact model to support a scaled initiative, including considering a robust cradle to career model, evolved goals and milestones, and realigned roles.
Conclusion
OnTrack Greenville has exceeded the expectations of so many of us. The impact on students is apparent each day in schools, as well as measured in the data. It has transformed the way Greenville County Schools and the partners view students in poverty and how we address their needs as well as the systemic poverty itself. Furthermore, the impact that the collaborative has had on the way we approach evaluation, learning, adaptation, and partnership is undeniable.
As we knew from the beginning, OnTrack Greenville is a long-term systemic effort that requires patient and sustained commitment.
All of us involved with OnTrack Greenville continue to be tremendously grateful to the funding partners who have trusted the process and joined in the journey with us. The work continues!
