An Evaluation of
A Forest for Every Classroom:
Learning to make choices for the future of Vermont’s forests
Prepared for
Shelburne Farms
The Marsh-Billings-Rockefeller National Historical Park
The Conservation Study Institute
The Northeast Natural Resource Center of the
National Wildlife Federation
The United States Forest Service
and the Place-based Education Evaluation Collaborative
by
Amy L. Powers
Program Evaluation and Educational Research Associates
July 1, 2003
EXECUTIVE SUMMARY
(see attached pdf file below for complete report)
A Forest for Every Classroom Program Overview
A Forest for Every Classroom (FFEC) is a professional development program for educators developed by a unique partnership of public land management agencies and nonprofit organizations. In particular, FFEC’s focus has been working with teachers and school districts adjacent to the Green Mountain National Forest and Marsh-Billings-Rockefeller National Historical Park. The partners have worked together to provide two cohorts of teachers with a year-long workshop series in which they are exposed to new content for their teaching; discuss new ideas about how to link subjects to the local community through field experience with resource specialists; and support in curriculum development. Critical components of the FFEC model include an emphasis on place-based education, service-learning, educational use of community resources,
and civic participation.
Formed in 2000, the partnership consists of Shelburne Farms, The National Park Service’s Marsh-Billings-Rockefeller National Historical Park and the Conservation Study Institute, The Northeast Natural Resource Center of the National Wildlife Federation (NWF) and The Green Mountain National Forest. As with many partnerships, roles and responsibilities of the various partners have evolved over the course of their first two years together.
FFEC Common Vision and Project Goals
FFEC Common Vision: If today’s students are to become responsible environmental decision makers, they must understand the local ecosystems in which they live and they must have educational opportunities based on real life issues that encourage them to practice citizenship in their own communities.
FFEC’s goals are to:
· Cultivate an understanding of place by working with teachers and their students to experience and understand local forests as complex and dynamic systems of natural and cultural resources and increasing interaction between the school and community, building a stronger sense of place and stewardship of public lands;
· Provide resources for educators to meet state and national education standards while effectively integrating stewardship, citizenship and a sense of place into their curricula;
· Foster a strong network of teachers, partners, community members and natural and cultural resource specialists that will ensure an ongoing relationship of sharing of information, materials, and resources.
· Promote a balanced view of forest stewardship that not only teaches about the forest ecosystem, but also includes the spectrum of stewardship challenges faced by land management agencies (federal, state, local) and private forest landowners
· Build a strong partnership that helps to increase institutional capacity and further program needs.
FFEC Evaluation Methods 2002-2003
Evaluation of the Forest for Every Classroom program began at its inception in 2000. Project partners sought a comprehensive evaluation of their first two years of programming in order to better understand the successes and challenges of FFEC’s process of program development and implementation, and to measure the degree to which its projected outcomes were attained. Additionally, the evaluation was understood as a tool to better document the process and outcomes of the developing model, providing information for FFEC project partners and funders to assist with program development, justification and refinement. The evaluation process was participatory, encouraging input and reflective practice by teachers and amongst FFEC staff.
Evaluation questions were designed by looking at the goals, objectives and expected outcomes outlined by FFEC partners in their Logic Model and by meeting with program stakeholders. FFEC staff reviewed the questions and upon approval, appropriate research instruments were designed.
The evaluation questions focused on:
1. process effectiveness: major strengths and challenges of the program
2. teacher outcomes: results and impacts of the program on teachers
3. student outcomes: results and impacts of the program and teachers’ students
The following table lists the types of evaluation instruments used and participants involved in the evaluation:
Evaluation instruments and data collection methods
Instrument Type, Number, Brief Description of People involved
Focus Group Interviews
6 FFEC 1 teachers
11 FFEC 2 teachers
9 FFEC partners from 5 organizations
Students (informal)
Class/Field Observations (including interviews with teachers and informal interviews with students) 6 teachers on-site:
· 1st grade integrated
· 3rd grade integrated (2 teachers at 2 different schools)
· 3/4 grade integrated
· 7th grade math
· elementary school enrichment teacher
Teacher Written Surveys
16 PRE (94%); 12 POST (71%) (FFEC 1)
10 PRE (77%); 8 POST (62%) (FFEC 2)
Student Written Surveys
Insufficient data returned for reporting
Individual interviews
2 people who had experience with another comparable professional development program (7/8 science teacher and one project partner
Institute Evaluations
Written and verbal from both FFEC 1 and 2 sessions
Institute Observations
2 days of FFEC 2 summer institute
Document Review
Project fliers and brochures, grant proposals, logic model, participant correspondence, workshop materials, participant products such as interpretive trail guide, teacher-developed curricula and teaching materials, student work samples.
Findings and Discussion
Findings in terms of both process and outcomes were exceptionally positive overall and shed light on the unique and successful elements of the FFEC model in addition to a range of impacts experienced by participants. Furthermore, candid feedback on challenges experienced by participants are presented in the report in the hopes that they will inform the refinement of the model.
Process Strengths
A consistent theme conveyed by teachers during focus groups and in written surveys was unequivocal praise for the quality of the FFEC program. Comments such as, “this has been the best professional development I have done in my 20 years teaching” were common, and participants were able to easily substantiate their praise with descriptions of the most significant strengths of the program. Data gathered from teachers were augmented by evaluator observations and data gleaned from FFEC partners. The most salient themes that emerged are:
· Role modeling sound teaching practice
· Offering diverse and balanced perspectives
· Respecting and nurturing teachers as professionals
· Offering an organized, well-crafted program
· Commitment to long-term support
Process challenges
As with any evolving model, process challenges are inherent. To aid the partners in developing and refining the model, process challenges were documented. These challenges are divided into three general categories: those that pertain to the broader project or model, specific reservations about program content, and pressures faced by teachers that originate external to FFEC but impact its outcomes.
The process challenges that emerged were:
· Program implementation costs
· Follow-up visits underutilized
· Service-learning challenges
· Program areas to improve
o Defining central terms
o Help teachers understand “the How”
o Offer options for pursuing more depth
o Curriculum development expectations
· Teachers’ other pressures
· Clarity of partner roles
Project- and program-level recommendations provided in the last section of the full report are based on aggregated input from participants and the evaluator’s analysis of the program. Many of the recommendations are specifically linked to the process challenges noted and are intended to help FFEC partners refine the program as it continues to grow and expand its reach.
Teacher Outcomes
There was ample evidence that teachers gained new content, resources and inspiration from their participation in the FFEC series. The outcomes most consistently noted by or observed in teachers include:
· A change in teaching practice
· Using FFEC resources
· Building relationships with local natural and human resources, including public lands
· Creating a network of support
· Developing into teacher leaders
· Personal changes: knowledge, inspiration and rejuvenation
Student Outcomes
The student outcomes discussed are primarily a product of teacher reports and observation data. Though an attempt was made to survey students before and after their teachers implemented FFEC-related curricula, the response rate was so low that data can not be used. Teachers readily reported their observations of the effect of place-based education on their students, with seven themes emerging as the most commonly seen outcomes for students. They are:
· A growing relationship to local resources, both people and places, what might be though of as an “attachment to place”
· Building community in the classroom itself
· Student engagement in outdoor learning
· A positive influence on academic performance
· Positive influences and outcomes for students with special needs
· Evidence of civic engagement in students
Teacher Survey Results
Pre- and post-surveys administered before FFEC intervention and after the series ended asked teachers to report on:
1. their familiarity with forestry knowledge and skills;
2. their understanding of other FFEC content areas; and
3. their curriculum and teaching practice.
The questions in these areas were designed to measure change over time, and survey data were compared using t-tests. In all 22 areas surveyed, teachers showed improvements in their knowledge or practice. Areas that reached statistical significance at the p<.05 level were the following:
Forestry knowledge and skills
· conducting a forest inventory
· Current logging techniques and practices
· Use of Biltmore stick (greatest gains)
· Global forces in the wood product industry
· Sustainably forestry certification (greatest gains)
· “Multiple Use” issues
Content knowledge:
· Soil science
· Forest ecology
· Forest fragmentation
· Land use history
Curriculum and teaching practice:
· Teaching standards-based curricula
· Creating standards-based curricula
· Using hands-on science activities in the classroom
· Teaching students outdoors
· Promoting service learning opportunities for students
These consistent gains are a very strong confirmation of program success in these content and practice areas. Furthermore, perhaps the greatest testament to the successful execution of a program is the degree to which it meets its goals. In post-surveys, participants were asked to rate eight program goals:
· Link teachers to resource specialists
· Link teachers to local resources and places
· Provide teachers with useful printed resources or other media
· Assist teachers in meeting educational standards
· Increase teachers’ environmental awareness
· Increase teachers’ knowledge about forests
· Assist teachers in incorporating service learning into their curriculum
· Increase respect and caring for local forestry resources in students
For both FFEC 1 and FFEC 2 groups, mean scores showed that participants rated all eight goals as attained or nearly attained, the top categories available, another meaningful testament to the program’s success.
Conclusions and Recommendations
From a multi-year commitment to teachers to its individualized curriculum development focus, many features of the FFEC model distinguish it from a standard professional development program. But perhaps what truly sets FFEC apart from even the most innovative programs is the fact that it is the creation of a diverse partnership of public sector and non-profit organizations who bring a balance of skills, personalities and resources to the professional development series.
For change to come to schools, teachers must change the way they teach. And for teachers to change what and how they teach, they must have models, resources and the motivation to change. The nurturing and respect FFEC provides teachers motivates them to be fully engaged in the FFEC program, to utilize new resources—public lands, publications, people—and, ultimately, to change how they teach students.
On another level, one could argue that teachers must also experience a level of personal transformation in order to bring change to their teaching, and to most effectively convey their passion to students. FFEC provides teachers with the stimulation and challenge that encourages personal growth. Teachers reported becoming less judgmental, more respectful of others’ viewpoints, more knowledgeable of a diversity of issues behind what they teach and more connected to others in their profession and in some cases, more personally engaged as citizens in their own towns.
Enhancing a community’s understanding of and respect for its local heritage—both natural and cultural—is a large goal. By exposing teachers to public spaces, training them to access local resources, and offering them the skills to offer these to their students in meaningful ways, FFEC increases students’ understanding of and participation in their communities, a positive step toward a greater appreciation of public resources and enhanced civic engagement.
The full report is attached (below) as a pdf file.
| 