An Evaluation of
The Sustainable Schools Project
2002-2003
Prepared for
Shelburne Farms and
The Vermont Education for Sustainability Project
by
Amy L. Powers
Program Evaluation and Educational Research Associates
July 1, 2003
Executive Summary follows. The full report is attached as a pdf file.
EXECUTIVE SUMMARY
Sustainable Schools Project Overview
The Sustainable Schools Project (SSP) is a partnership program of Shelburne Farms and Vermont Education for Sustainability (VT EFS). The project uses sustainability as the integrating context on a school-wide level, seeking to connect science and literacy, ecology and community, and beginning inquiry and practice to improve the quality of life for all. SSP focused its first school year (2002-2003) piloting its program in one urban elementary school in Burlington, VT.
SSP staff developed and implemented a two-day summer institute for the teachers of Champlain Elementary, and have since provided on-going support to the school in the form of: attending grade level meetings and assisting teachers with curriculum planning; attending and facilitating sustainability-themed staff meetings; assisting teachers in the classroom working with small and large groups of students; providing teachers with curriculum and content resources; facilitating connections between community and parent volunteers and classrooms; and meeting with teachers and administrators individually to coordinate the effort.
In 2002, SSP joined efforts with a several other New England place-based education programs to strengthen their evaluation methods, develop their programs and contribute to the field of place-based education. This program is one of four programs evaluated as part of this evaluation collaborative. The four programs were each evaluated individually, and an additional analysis of cross-program themes was performed and is documented in a different report.
Evaluation Methods
This year’s program evaluation sought to understand the successes and challenges of SSP’s process of program development and implementation, and to measure the degree to which its short-term projected outcomes were attained. The evaluation was understood as a tool for better documenting the process and outcomes of the developing model and providing information for stakeholders to assist with program development, justification and refinement. While it is primarily a formative evaluation, the report provides preliminary evidence of positive outcomes as well as suggestions for program improvements and is intended to be useful to program managers, funders and interested stakeholders, and to contribute to the field of place-based education.
Evaluation questions focused on:
1. process effectiveness
2. teacher outcomes
3. student outcomes
The choice was made not to focus the evaluation specifically on school-wide change because of the newness of the project. However, findings in this realm were explicit and thus are reported in addition to impacts on teachers and students.
The table below depicts the types of evaluation procedures conducted.
nstrument Type and Number Administered
Interviews
22 Teacher/Administrator (7 were pre and post)
3 Community Partner
3 SSP Staff
Student (informal)
Teacher Written Surveys
11 Pre (grades 1-5)
13 Post (grades 1-5)
Student Written Activity
122 Pre
126 Post
Institute Evaluations
16
Observations
By evaluator:
4 classes
2 teacher institute days
3 staff meetings
1 grade-level planning meeting
1 school-wide sustainability fair day
Also collected and reviewed observation notes from SSP staff throughout the year.
Document Review Project fliers and brochures, logic model, curriculum rubrics, teacher-developed curricula, assessment and evaluation tools used by teachers, student work samples.
Findings and Discussion
Process strengths
The strengths of the SSP process are numerous. Highlighted here are the aspects of the Shelburne Farms/VT EFS project staff’s process implementation that were particularly successful and highly touted, as well as aspects of the pilot school that were deemed to be critical to success of this first-year project.
SSP implementation strengths:
· Skilled process facilitation
· Rapport and trust established
· Summer workshop promotes enthusiasm
· Guidance on curriculum planning
· Provision of resources and contacts
· Building capacity and providing support
· Role modeling and one-on-one coaching
Pilot school strengths:
· Steady administrative support
· Rapport among faculty
· Interest at district level
· Internal example of using the concept of sustainability for instruction
Process Challenges
Resounding praise for and satisfaction with the first year of the SSP model clearly indicates that the project was born of a strong foundation and met overwhelming success. One could argue that another testament to a projects’ success is its flexibility, malleability and openness to critique and evolution. Particularly in the first phases of a project, seeking feedback on challenges and barriers to success is critical.
Throughout the evaluation process, some barriers cited were school, community or grade-level specific while others address program functions. In the former case, they are important for program staff to be aware of while selecting schools and/or tailoring their work to particular sites. In the latter case, this evidence gives staff the opportunity to alter the model accordingly to better meet user needs. The following areas emerged as the greatest process challenges:
· Role of community advisory group
· Facilitating community partner relationships
· Limited time allocated for project
· Curriculum documentation
· Communication challenges
· Teachers’ other pressures
· Limited resources for younger grades
School-wide Outcomes
Although the evaluation questions focused on teacher-specific changes and process features, throughout the evaluation process it became clear that the SSP made clear strides at Champlain School in terms of whole-school change. Four school-wide impacts in particular merit discussion:
· More parental/community involvement: Growing school-community
connections
· Growing curricular integration
· Changes in school yard and physical space
· Increased use of Learning Center
Teacher Outcomes
Since teachers are a primary agent of change in the school setting, a predominant focus of the evaluation was measuring the degree to which teacher practice changed as a result of SSP involvement in the school. Clear changes were seen in the following areas:
· Increased collaboration among teachers
· Dialogue about sustainability created among faculty
· Enthusiasm and Personal growth
· Curriculum changes
o Teachers’ utilization of local place: schoolyard and community
o Interdisciplinary curricula
o Intentional planning
Student Outcomes
During interviews and observations, data were gathered as to teacher, staff and community partner perspectives on the impacts of the sustainability work on students. In addition, teachers conducted a flexible pre-post activity with individual classes to assess changes in students’ understanding of their school community over the course of the year. Though outcomes for students varied greatly by grade level and classroom, the following are themes that emerged as general student outcomes:
· Awareness of their community and natural world
· Effective strategy for including high-needs students
· Sense of school community and caring
· Attitudes toward school work
· Academic performance
Conclusions
As the SSP model emerged through a pilot program at Champlain Elementary School, the strengths of the newly forming and still flexible process were resounding. The project generated enthusiasm and positive learning outcomes for students and it seems to hold considerable promise as a whole-school change model.
The first year of implementation would indicate that SSP has successfully put into play its defined strategies. By assisting teachers with curriculum development during a summer institute and throughout the school year, SSP staff helped teachers move toward a curriculum that is not only more connected to the local community but also one that is more connected within the school among grade levels and, to some extent, throughout the school.
By effectively implementing its core strategies with the Champlain School, SSP is making in-roads toward meeting its broader goal of creating a truly sustainable school. Already there are signs as well that the goals are realistic and measurable. Though all programs have room for improvement, this year’s evaluation process yielded clear indications of a successful project.
Program Recommendations
The report contains recommendations that are intended to help SSP staff refine the project as it continues to grow and expand its reach. They are based on aggregated input from participants and the evaluator’s analysis of the total program. The recommendations fall into these eight categories:
o Program Content and Resource Provisions
o Project Start-up
o Communication
o Cultivating Community Partnerships
o Media and public exposure
o District-wide efforts
o Project documentation
o Evaluation
The full report is attached (below) as a pdf file.
| 