See below to download a pdf of the complete report.
Shelburne Farms
Forestry Program Evaluation
Final Report
August 2000
Amy Powers
Shelburne Farms
Report of results and recommendations for a program evaluation conducted in Fall 1999.
Introduction
The mission of Shelburne Farms is to cultivate a conservation ethic in children and young adults by teaching and demonstrating the stewardship of natural and agricultural resources. Program goals at the second grade level are to increase awareness and appreciation of natural and agricultural resources and the working landscape.
The goal of Shelburne Farms’ School programs is to increase awareness and appreciation of natural and agricultural resources and the working landscape; and to inspire active stewardship of our environment. It is the philosophy of the Education department to provide a nurturing environment for an individual to develop a strong sense of place and self, leading to empowerment and hope. Programs are conducted for pre-school through middle school students as well as pre-service and experienced educators. The programs include hands-on, age appropriate activities that are designed to develop skills and a knowledge base necessary for an individual to make informed decisions; to develop effective communication skills; and to provide experiences that lead an individual to develop a sense of one'’ place and comfort with
the earth.
Since 1978 the field trip program has offered teachers the opportunity to extend their students classroom studies with hands-on field trips. The structure of this offering has varied throughout the years. Currently, most teachers bring their class to Shelburne Farms for one field trip experience. There are several middle school programs that also offer a pre-visit to the school, and outreach programs are an option.
Shelburne Farms recognizes the need for a foundation of research to promote sound program development. The main purpose of this study was to determine if a one-day, three hour, Forestry field trip increases appreciation and awareness of natural resources in 2nd grade students. A secondary objective was to determine if a two-day field trip more effectively increases this appreciation and awareness than a one-day field trip. The results are intended to help the education staff design future programs based on a solid understanding of what educational content, methodologies and practices best accomplish Shelburne Farms’ mission. See Appendix A for field trip outline, including field trip objectives.
Research Context
Past research on the effectiveness of Environmental Education experiences was used to guide the development of this project. The subjects of many studies are middle school, high school and college students, though several studies of pre-school, kindergarten and elementary-age students have been reported. Leeming, et. al. provided a review of 34 environmental education studies published between 1974 and 1993 and draw the conclusion that researchers have largely neglected to work with children in the lower grades and suggest that the effects of environmental education with young children is a fruitful area of research.
In another study, The State Education and Environment Roundtable (SEER) of California published a report which reviewed and compiled studies of Environmental Education effectiveness (Hoody, 1995). Their review found that interdisciplinary-based research in EE is poorly represented in the literature and that assessment of student learning in the environmental education context is problematic because standardized testing does not readily measure hands-on, problem-solving educational methods. In response to this kind of complexity in measurement, several scales for evaluating the effects of environmental education have been developed and validity-tested, including the Children’s Attitudes toward the Environment Scale (CATES) and the Children’s Environmental Attitude and Knowledge Scale (CHEAKS) (Musser, 1994 and
Leeming, 1995). While these instruments assure test reliability and facility of statistical computation, they are not tailored to the type of study we are undertaking. The tests ask general questions about turning off lights, for instance, or river dams which are not pertinent to the experiences students have at Shelburne Farms.
Instead we have utilized the examples of approaches to program evaluation other than strictly quantitative measures, such as Thomas’ discussion of the case study method (1989-90) to design this program evaluation. In addition, personal communication with Anita Kraemer of eEvaluations and a review of her comprehensive evaluation of the Chesapeake Bay Foundation Education Program was instrumental in planning this study.
Other studies have acknowledged the importance of physical context in learning, a basic tenet of Shelburne Farms’ field trip programs (Falk, 2000). The conclusion is that since students have problems transferring information learned in one context into another, we must teach students about forests in the forest for the most concrete learning to occur (Ceci, 1994). Thus we speculate that perhaps two field trips to the study site will enhance understanding of the concept, in this case, forestry. This study aims to discover if that is true for the type of field trip that Shelburne Farms offers.
One longitudinal study at Bank Street College in New York City demonstrated that children remembered the most about museum visits when they were allowed multiple visits rather than single visits (Wolins, 1992). This type of study prompts further research to determine whether the same holds true for a non-museum-based forestry curriculum.
The above context is by no means an exhaustive literature review. Please see Appendix B for additional references and resources.
Purposes
The Shelburne Farms education department is committed to providing the best field trips possible for children visiting the Farm. The staff continually evaluates the effectiveness of field trips through less formal measures in order to fine tune field trips and at times make broad changes to field trip content or methodologies. On-going evaluation methods include:
Teachers are asked to complete a written evaluation of each field trip after the visit. A 20% return rate is estimated and methods of encouraging a higher return rate and more useful responses are being considered currently.
At the end of most field trips students are asked by Shelburne Farms staff to describe one thing they learned during the trip or to share their favorite part of the day.
Following each field trip, teaching staff gather to process the teaching day—what worked well, what new things they tried, what concept or activity did not seem to work well for this age group, etc.
Staff regularly receive “thank you” cards from students in which children detail what they remember most about the field trip.
It has become increasingly apparent, however, that Shelburne Farms would benefit from more solid research into program effectiveness, in terms of format and content. The purpose of conducting this more formal evaluation is two-fold:
To make decisions on how to best structure field trip or other programs as Shelburne Farms continues to grow and evolve as an education program;
To satisfy Shelburne Farms’ members’ and supporters’ interest in research documenting the effectiveness of its programs.
Evaluation Questions
The objectives for the second grade forestry field trip are that students will increase their awareness and appreciation of natural resources; put into proper sequence and identify the life cycle of a tree; find animal homes throughout the forest; name the parts of a tree (roots, trunk, branches, twigs, leaves); name some uses of wood from the forest. The study was designed to determine whether students increased in both awareness (knowledge) and appreciation (attitudes).
The instruments were designed to help us answer the following questions:
Knowledge
Do students learn and retain factual information from attending a one-day field trip at Shelburne Farms?
Do students learn and retain more factual information from attending a two-day field trip?
Were students more inquisitive immediately after the field trip?
Attitudes
Did students’ attitudes toward nature change as a result of participating in a one-day field trip?
Did students’ attitudes toward nature change more as a result of participating in a two-day field trip?
Was there a change in free-time preferences after the field trip?
Was there a change in how students would treat a forest creature? (More curiosity, respect?)
It is noteworthy to mention the seminal EE research by Hines (1986-87), Hungerford and Volk (1990) among others, that determined that responsible environmental behavior is associated with the following variables: knowledge of issues, knowledge of action strategies, locus of control, attitudes, verbal commitment, and an individual’s sense of responsibility. Shelburne Farms’ programs are designed with these variables in mind, and this study focuses in particular on the building blocks of knowledge and attitudes.
| 