State-Commissioned Study: Pre-K Boosts ‘School Readiness’; Long-Term Benefits ‘Negligible’

Executive Summary of “Assessing the Impact of Tennessee’s Pre-Kindergarten Program: Final Report,” Released May 31, 2011 by the Strategic Research Group:

In 2007, the Tennessee Office of the Comptroller contracted Strategic Research Group (SRG) to conduct a study to investigate the short- and long-term effects of state-funded Pre-Kindergarten (Pre- K) participation on academic outcomes in Kindergarten through Fifth Grade through an examination of existing school records (i.e., secondary data). The evaluation was structured to take place over a multi-year timeframe and in a series of reporting stages. The overarching goal of this effort over the series of reports submitted to date has been to identify Pre-K participants in existing school records and to determine, to the best possible extent given the data available for analysis, whether there is evidence to suggest that Pre-K participation is associated with a positive effect on student performance in Grades K-5 relative to students who did not participate in Pre-K.

On the whole, the results of analyses conducted to date in this series of analyses of outcomes in Grades K-5 point to an initial near-term advantage associated with Pre-K participation in Kindergarten and First Grade—primarily for students who received Free/Reduced Price Lunch (FRPL) or are considered “at-risk” due to socioeconomic status. Longitudinal analyses conducted in two previous reports have found that this initial advantage tends to be followed by a pattern of convergence, although a slight advantage of Pre-K participation appears to be maintained among economically disadvantaged students through the Second Grade. For students in Grades 3-5, analyses have found either no significant effect of Pre-K participation on assessment scores, or, in some cases, have found that students who attended Pre-K, on average, score lower than their non-Pre-K counterparts on some assessments.

The goal of this final report has been to maximize the number of student records that can be included in the analysis, providing opportunities for longitudinal analyses that were not possible for previous reports in this series. This provides valuable perspective considering that each previous report has varied in terms of the program years covered as well as the school years/grade levels incorporated into the analysis. Thus, this final report includes all possible student records from the years specified in the study period.

In order to evaluate the short- and long-term impact of Pre-K on student outcomes, a sample of non-Pre-K students was randomly selected that mirrored the Pre-K group with regard to school or school system, gender, race, and FRPL status.

Data were analyzed using random effects analysis of covariance models, also referred to more broadly as hierarchical linear models or multilevel models. Analyses controlled for demographic characteristics such as child race and gender, as well as FRP status, special education, English as a Second Language (ESL) status, and retention.

Combined results across ten cohorts of students who participated in Pre-K indicate that on standardized assessments in Kindergarten, Pre-K students—particularly those who experience economic disadvantage—perform better than students who did not participate in Pre-K. No overall differences were found between Pre-K and non-Pre-K students in First Grade, although again, Pre-K students who experience economic disadvantage tend to perform better than their non-Pre-K counterparts. However, this same pattern is not consistently observed for students who do not experience economic disadvantage, and the initial advantage attenuates and is largely diminished by the Second Grade. Among students who do not experience economic disadvantage, the initial advantage of Pre-K is less evident, and the models suggest that they may experience slower academic growth over time.

The results of the analyses of long-term effects (i.e., Grades 3-5) find that the differences between Pre-K students and non-Pre-K students are negligible, particularly when examining assessment outcomes for students who experienced economic disadvantage. By the third grade, students who did not experience economic disadvantage performed better on standardized assessments than Pre-K and non-Pre-K students who had received FRPL, although not as well as students who had not experienced any known risk factors.

This study has faced some challenges. One of the greatest is that no assessments were available for students as they began Kindergarten. Instead, assessments conducted end-of-year in Kindergarten are the earliest indicator available that we can use to gauge the most immediate impact of program participation. However, even this indicator is impacted by factors outside of the control of the Pre-K program, including the fact that these data were only available when school systems elected to administer assessments at the Kindergarten level. The majority of Kindergarten students did not complete standardized assessments; more students are assessed in First Grade, but still not the majority.

Arguably, the greatest limitation of this study is that educational records do not indicate whether students participated in any Pre-K program other than Tennessee’s Pre-K. Throughout this series of studies, analyses have not been able to determine whether students in the non-Pre-K group attended another type of Pre-K program, nor have the analyses conducted here been able to control for additional interventions students may have received (or have not received) beyond Pre-K. These remain the most significant issues in terms of interpretation of the results because it is quite likely that the benefits of Pre-K are underestimated in the models presented here.

Despite the limitations of this study, however, the overall conclusions to be drawn from this series of reports and the cumulative analyses presented in this final report have been consistent: students who participate in Pre-K reliably show better outcomes on Kindergarten assessments than students who do not participate in the Pre-K program. These results provide evidence that the objective of Tennessee’s Pre-K program – school readiness – is being met.