 |
McBeth, B., Hungerford, H., Marcinkowski, T., Volk, T., and Meyers, R. (2008). National environmental literacy assessment project; Year 1, national baseline study of middle grades students: Final research project. U.S. Environmental Protection Agency; National Oceanic and Atmospheric Administration, U.S. Department of Commerce; and North American Association of Environmental Education: Grant # NA06SEC4690009.
The executive summary of this report is reproduced immediately below, and the complete report can be downloaded from the bottom of this page.
****************************************
EXECUTIVE SUMMARY
This research project was designed primarily to meet recommendation number five, (Support and strengthen long-term research initiatives), established by the U.S. Environmental Protection Agencyís (EPA) National Environmental Education Advisory Council (NEEAC) and tendered to Congress in the 2005 Report to Congress, Setting the Standard, Measuring Results, and Celebrating Successes. More specifically, the Action Items supporting this recommendation include: a national measure of environmental literacy; the development of a comprehensive, research-based instrument for this purpose, as well as for use in more specific state and programmatic assessments so that comparisons can be made to the national assessment data. Other actions such as to indentify ìprovenî and ìpromisingî programs and
the study of these programs to understand their relative effectiveness in meeting the goals of environmental education and environmental literacy will be dealt with in subsequent studies.
The project is important because it provides instrumentation to measure environmental literacy and baseline environmental literacy data for 6th and 8th graders across the United States. This information can be used eventually to assess program effectiveness in the hope of raising environmental literacy across the nation. Similarly, this project
supports and advances the goals of the National Environmental Education Act. More specifically it supports the EPA Office of Environmental Educationís (OEE) Strategic Planís research component recommending research that assesses the effectiveness of Environmental Education in meeting environmental protection and academic achievement goals. This project also addresses the National Oceanic and Atmospheric Administrationís (NOAA) vision to incorporate social sciences into its research strategies. The data generated from this project and subsequent programmatic data may well have a direct impact on the design of NOAAís educational programming. Opportunities will eventually exist to assist NOAA in designing educational programming that has the highest possibility of success at meeting its academic objectives.
By providing instrumentation and baseline data on environmental literacy, this project will help to fulfill the NAAEEís mission of advancing environmental literacy. The results of this project will provide resources and data to use in the development and advancement of quality environmental programming.
The administering partner for this project was the North American Association for Environmental Education (NAAEE). Key partners included researchers from the Center for Instruction, Staff Development and Evaluation; Florida Institute of Technology; the firm of Ron Meyers and Associates; and the University of Wisconsin - Platteville.
The Middle School Environmental Literacy Survey (MSELS) used in this study was developed and refined by Hungerford, Volk, Bluhm, McBeth, Meyers, and Marcinkowski. The MSELS includes several demographic items and measures of the following environmental literacy components: (a) ecological knowledge; (b) verbal commitment; (c) actual commitment, or environmental behavior; (d) environmental sensitivity; (e) issue identification and issue analysis skills; and (f) action planning. As such, it includes measures in each of the four domains that are critical to environmental literacy: Knowledge, Affect, Cognitive Skills, and Behavior. The MSELS contains multiple choice and Likert-type items, and was designed to be administered within a traditional 50-minute class period.
Three other survey forms were used in each school. Administrators were asked to complete School Information Forms and each cooperating teacher was asked to complete Program and Teacher Information Forms. A nationally stratified random sample of 6th and 8th grade classrooms was identified by theGfK Roper for this project. Then, in April, May and June of 2007, the National Middle School Student Environmental Literacy Assessment Project administered the instrument to the previously selected random sample in forty-eight schools across the United States. Dr. Meyers was the coordinator for the data collection.
Once collected, the data were sent to Dr. Marcinkowski at the Florida Institute of Technology for interpretation. Statistical software packages were used to calculate individual student scores for each section of the MSELS, to run statistical analyses on those scores, and to graph these results (e.g., the range, median, mean, standard deviation for each section of the MSELS). More specifically, four or more kinds of statistical analyses were conducted. First, these scores were analyzed at the school level, as this is the sampling unit for this phase of the research. Second, scores for all sixth grades on each section of the MSELS were analyzed using descriptive statistics in an effort to generate a national profile of environmental literacy for this grade level. The same was
done for scores for all eighth graders. Third, inferential statistics were used to compare sixth and eighth grade studentsí scores on sections of the MSELS to determine if there were any significant differences. Fourth, descriptive statistics were computed using information from the School, Program, and Teacher Forms, and from the demographic items in the MSELS (e.g., differences between classes from rural/suburban/urban schools, differences between classes based on primary program resources in use, differences between classes based on teachersí highest degree or workshops completed, differences on the basis of student gender or ethnic background). Finally, additional exploratory analyses are planned for a later time.
The research team expects to disseminate research findings in both ongoing and summative fashion through presentations at annual NAAEE conferences, at NAAEE's Research Commission gatherings, and at conferences sponsored by organizations related to environmental education; articles in environmental education journals and newsletters; articles in journals and newsletters in fields related to environmental education (e.g., outdoor education, interpretation, etc.); and articles in journals and publications within the larger educational community. The audience for these research findings includes professional educators and researchers as well as private and governmental agencies. It is hoped that other groups which interface EE would also take a serious interest in the findings of this study, e.g., classroom teachers, teacher
educators, staff of informal and non-formal programs, environmental scientists, conservation educators, and others.
As noted above, the results from this study included weighted scores for all sixth graders and for all eighth graders on each scale (or index) included in the MSELS. Weighted results such as these are important because weighting permits the results of this study to represent the national population of sixth and eighth graders in the U.S. Table 1 (p. x) summarizes these results. This table identifies the number of items in and possible range of scores for each scale, as well as the mean, median, and modal scores on each scale for each of these grade levels. However, as important as weighted scores such as these may be, it is often difficult to interpret or use them in this form. For this reason, these scores were adjusted and used to calculate composite scores (Table 2, p. xiii), which are summarized here
to ease interpretation of and facilitate use of these results.
Table 1.
Summary of Descriptive Statistics for Weighted Data, by Scale or Index, and by Grade
Figures 1 and 2 present the distributions of the environmental literacy composite scores (utilizing unweighted data) for all 6th grade students (Figure 1, p. xi) and for all 8th grade students (Figure 2, p. xi). Figure 3 (p. xii) presents the combined 6th and 8th grade studentsí composite score (also using unweighted data). Histograms are graphic representations of the distributions of scores. Considering a random sample such as ours, we would expect a normal (bell shaped) curve. The research team is pleased with these distributions.
Figure 3. Histogram of the environmental literacy composite scores for all 6th and 8th grade
students.
The normalcy of the distributions both validates the randomness of the sampling and adds to the validity of the MSELS as a measurement tool with the ability to discriminate levels of environmental literacy within these study populations.
In order to derive a composite score of all literacy components, scores on the individual sections of the MSELS were adjusted with multipliers so that the sum of each of the four Components of Environmental Literacy (Ecological Knowledge, Environmental Affect, Cognitive Skills, and Behavior) equated to 60. An Environmental Literacy Composite Score resulted from compiling the four Components of Environmental Literacy scores (possible composite score 240, with a range from 24 to 240). Table 2 (p. xiv) presents the environmental literacy composite scores. The 6th grade
composite score was 143.99 and that of the 8th grade was 140.19 with an overall composite score of 142.14. These scores all fall in the mid-range of possible scores (97 ñ 168). The team cautions the reader to bear-in-mind that these composite scores are reflections of the environmental literacy components and are limited to the range of possible scores on the instrument used in this research. Although these scores do not have inherent value, their utility lies in their potential for comparing this sample to others.
Table 2.
Components of Environmental Literacy and Composite Scores
Additional insight into environmental literacy can be gained from examining the scores associated with the four components of environmental literacy included in the instrument used in this research (each component with a possible value of 60). Of the four environmental literacy components, the highest scores (combined 6th and 8th grade adjusted grand mean)
were attained in Ecological Knowledge (40.34), with slightly lower scores in Environmental Affect and Behavior (39.40 and 36.84, respectively). The lowest scores were observed in the component of Cognitive Skills (25.56). In the components that focused on performance (i.e., which asked students to demonstrate knowledge or skills), the 8th graders out-scored the 6th graders. This might be expected due to developmental differences between the two age groups. In the components that relied on
self-reports (affect and behavior), the 6th graders outscored the 8th graders. Within the Cognitive Skills component, the highest values for both 6th and 8th graders were observed for Issue Analysis Skills, followed by Issue Identification Skills, and then by Action Planning Skills.
The development of environmental literacy is complex and can take many forms. This study provides us with a measure against which to compare future measures. Further analyses of these data (in particular with respect to the classroom and teacher information) might shed light on the impacts of environmental education efforts, where it was present in these classrooms. The research team anticipates an additional research effort that will purposively seek out and collect data from middle school settings where environmental education is in place. That will permit us to compare measures on environmental literacy variables across this representative sample of middle school classrooms and that purposive sample to determine if environmental education efforts can indeed make a difference in environmental literacy. It will also permit us to observe relative effectiveness of diverse environmental education programs and curriculums with respect to environmental literacy, in general, and
with respect to the four components of environmental literacy addressed by the MSELS. We are confident in our ability to collect data on environmental literacy variables and to compare, in meaningful ways, efforts to develop these variables in the adolescent populations in the United States.
| |
