This page presents three different formats describing an investigation into possible relationships between a place-based education project called CO-SEED and standardized test scores in Malden, Massachusetts.
1) Immediately below is the Executive Summary of the full report.
2) Downloadable from the bottom of this page is the full report.
3) Downloadable from the bottom of this page is also a one page "Eval Brief" which highlights key findings.
EXECUTIVE SUMMARY
The Community-based School Environmental Education Project (CO-SEED) is a program of Antioch New England Institute (ANEI), a consulting branch of Antioch University New England in Keene, New Hampshire. CO-SEED’s primary purpose is to help schools and communities work together to develop community- and place-based approaches to education while simultaneously increasing social capital and preserving the environment.
Each year, CO-SEED conducts extensive program evaluations and makes reports available on the web at http://www.peecworks.org/PEEC/PEEC_Reports/. As part of this year’s evaluation, PEER Associates, an external evaluation team, was asked to conduct a quantitative evaluation of the relationship between CO-SEED and academic achievement at the Beebe Environmental & Health Science Magnet School (Beebe School) in Malden, Massachusetts.
The Beebe School uses a school wide theme to connect health and environmental curriculum to real life issues in the surrounding community. The school’s theme involves students in hands-on, integrated learning and utilizes local agencies, business, and local natural settings for learning. CO-SEED worked with the Beebe School to enhance the effectiveness of its existing curricular theme from 2000 through 2005.
Measurement Strategies
In order to investigate possible relationships between Beebe students’ participation in the CO-SEED curriculum and their academic achievement, we used two different comparison strategies. The first was to compare average standardized test scores of Beebe cohorts to those of both the Malden School District and State during the same periods, using test scores from the Massachusetts Comprehensive Assessment System (MCAS). These benchmarks would illuminate patterns in Beebe performance that might deviate from typical trends. We tested predictions made by Beebe CO-SEED staff about subject areas in which they expected to see increased student achievement due to focused CO-SEED theme work. In addition, we sought to identify trends in achievement for traditionally underperforming groups.
In our second comparison strategy, we sought to capitalize on differences in the length of time that individual students had been enrolled in the Beebe School. The reasoning was that duration of enrollment could serve as a crude index of degree of exposure to the CO-SEED model of education since CO-SEED’s implementation focused on bring the school wide theme to life. If outcomes of the CO-SEED program are reflected in standardized test scores, we might see students with higher levels of exposure to the program also demonstrating higher scores. This is an application of a strategy for exploring program impacts that we have termed “dose-response” analyses, i.e. identifying variations in exposure to program elements, and determining whether higher exposure is reliably associated with higher levels of intended
program outcomes.
Results
We found moderate to strong support for five out of the six predictions for subjects areas in which there would be increased student achievement at Beebe due to focused theme work. These subjects included Math, Life science, Physical science, and Technology/Engineering, mostly for upper level grades. The strongest deviation from the typical pattern of achievement among Beebe, the District, and the State was seen in Math and Life Science for Grade 8.
In addition, the typical pattern of achievement shows that students not receiving free lunch regularly outperform free-lunch students on standardized tests. However, at Beebe the achievement gap was about three quarters of that of the District as a whole. This was entirely due to improved performance by free-lunch students, not by reduced performance of non-free-lunch students.
Conclusions & Implications for Practice
Overall, the analysis provided moderate support for the relationship between exposure to the CO-SEED program and academic achievement at Beebe School. Specifically,
· The Beebe CO-SEED program appears to be having a positive effect on upper level student achievement in the specific content areas of Math and Science.
· The CO-SEED program at Beebe may be having a targeted effect on traditionally underperforming students, such as those of lower socioeconomic status. It is possible that the highly experiential and innovative nature of the CO-SEED program in some way surpasses more traditional teaching methods, which do not serve the unique learning needs of underserved students as well.
· There is a somewhat reliable relationship between time in the Beebe school and performance on the MCAS, suggesting that students with more exposure to the CO-SEED program do appear to perform better on standardized tests.
The results presented in this study are fairly convincing, but not entirely specific enough to warrant more confident conclusions.
The measurement strategy used here was essentially a pilot, and so there were several lessons learned about the analysis process. Overall, the somewhat diffuse approach that was taken in this analysis does not appear to be a sufficiently sensitive enough measurement strategy to establish clear cut, causal connections among the data available. We recommend that future attempts at this type of analysis employ a strategy that more closely aligns a program’s place-based education activities with specific test outcomes.
In addition, a more complete data set would enrich the analysis and ability to make such conclusions. Ideally, in gathering data to establishing possible dose-response correlations, one would want both attendance records and achievement records in the same system or format. This would help to limit the amount of cases thrown out due to unavailable, incomplete, or incompatible data. The amount of variability in dose for the program also appears to have an impact on the ability to detect significant dose-response relationships.
| 
