Categories
Press Releases

Vanderbilt Research Group Releases New Pre-K Study

Press Release from Vanderbilt University’s Peabody Research Institute, Feb. 24, 2011:

Children who attended state-funded prekindergarten classes gained an average of 82 percent more on early literacy and math skills than comparable children who did not attend, researchers from the Peabody Research Institute at Vanderbilt University have found.

The initial results are from the first rigorous longitudinal study that has been conducted on the effects of public prekindergarten attendance on a statewide scale.

“This research is difficult to do but critically important to evaluating the effects of Tennessee’s investment in pre-k,” study leaders Mark Lipsey and Dale Farran said. “Such evidence is especially important in the context of the current budgetary constraints in Tennessee and other states that have made commitments to pre-k education.”

For the study, 23 schools in 14 Tennessee school districts randomly admitted children to their pre-k program. All of the schools received applications from more students than they could accommodate. The children admitted to pre-k were then compared to the children whose families applied but were not admitted. A total of 303 children were involved in this phase of the study.

Assessments at the beginning and end of the prekindergarten year found that the pre-k children had a 98 percent greater gain in literacy skills than children who did not attend a state pre-k program, a 145 percent greater gain in vocabulary and a 109 percent greater gain in comprehension. They also made strong, but more moderate, gains in early math skills (33 percent to 63 percent greater gains). Overall, the average gain across the board was 82 percent more than for the children who did not attend state pre-k.

Results from a second parallel study corroborated these findings. That study compared 682 children who attended 36 pre-k classes in rural and urban middle Tennessee schools to 676 children who had to enter a year later because of the birth date cutoff for pre-k eligibility.

The second study also found that children enrolled in state-funded pre-k classes scored significantly higher on emergent literacy and math assessments than the children who had not yet attended pre-k once the age difference was accounted for.

The strongest differences were again in the areas of literacy and language skills, with more modest gains in math skills.

Both studies will continue collecting data for the next four years. The second study will continue collecting data in waves across the state until every region is represented.

“These studies were possible only because of a strong partnership with the Division of School Readiness and Early Learning in the Tennessee Department of Education and the commitment of school districts across the state to learning about the effects of pre-k,” Lipsey said.

The studies are led by Lipsey, research professor of human and organizational development and Peabody Research Institute director, and Dale Farran, professor of education and psychology. Carol Bilbrey, research associate at the Peabody Research Institute, directed data collection.

The research is funded by the U.S. Department of Education’s Institute of Education Sciences.

The researchers will report on these and other findings March 4 at the annual meeting of the Society for Research on Educational Effectiveness in Washington, D.C.

Categories
Press Releases

Vandy NCPI Study Questions Merit of Teacher Bonus Pay

Press Release from Vanderbilt University’s Peabody College National Center on Performance Incentives, Sept. 21, 2010:

NASHVILLE, Tenn.—Rewarding teachers with bonus pay, in the absence of any other support programs, does not raise student test scores, according to a new study issued today by the National Center on Performance Incentives at Vanderbilt University’s Peabody College of education and human development in partnership with the RAND Corporation.

This and other findings from a three-year experiment – the first scientific study of performance pay ever conducted in the United States – were released at a conference on evaluating and rewarding educator effectiveness hosted by the National Center on Performance Incentives at Vanderbilt.

Paying teachers bonuses based on their performance has been a controversial issue nationwide since the 1950s, but until now the concept has never been scientifically researched.

“We tested the most basic and foundational question related to performance incentives — Does bonus pay alone improve student outcomes? – and we found that it does not,” Matthew Springer, executive director of the National Center on Performance Incentives, said. “These findings should raise the level of the debate to test more nuanced solutions, many of which are being implemented now across the country, to reform teacher compensation and improve student achievement.”

The Project on Incentives in Teaching, called the POINT Experiment, took place over the 2007 – 2009 school years with participation by mathematics teachers in grades 5 through 8 in Metropolitan Nashville Public Schools. Nearly 300 teachers, approximately 70 percent of all middle-school math teachers in Nashville’s public schools, volunteered to participate. The complete study, including setup and analysis, began in 2005 and ended in 2010.

POINT tested no other types of incentives or systems of support for the teachers, such as professional development or guidance on instructional practices – many of which have evolved over the five years since POINT began.

“We designed POINT in this manner not because we believed that an incentive system of this type is the most effective way to improve teaching performance, but because the idea of rewarding teachers on the basis of students’ test scores has gained such currency,” Springer said. “We sought a clean test of the basic proposition: If teachers know they will be rewarded for an increase in their students’ test scores, will test scores go up? We found that the answer to that question is no. That by no means implies that some other incentive plan would not be successful.”

Here’s how the POINT experiment worked:

  • Following a year of detailed project design by a multi-disciplinary team from Vanderbilt and RAND, all middle-school math teachers in Nashville were invited to volunteer for the experiment. Approximately 70 percent of all middle-school math teachers in Nashville’s public schools stepped forward to participate.
  • Approximately half of the nearly 300 volunteers were randomly assigned to a “treatment” group, in which they were eligible for bonuses of up to $15,000 per year on the basis of their students’ test-score gains on the Tennessee Comprehensive Assessment Program (TCAP).
  • The other half were assigned to a “control” group not eligible for these bonuses. Teachers were evaluated based on an historical performance benchmark for MNPS teachers, not on competition with one another. All teachers in the treatment group had the chance to earn bonuses. (The names of participating teachers – and which group they were in – have been kept confidential by the research team.)
  • The annual bonus amounts were $5,000, $10,000 or $15,000. Over the course of the experiment, POINT paid out more than $1.27 million in bonuses. Overall, 33.6 percent of the original group received bonuses, with the average bonus being approximately $10,000.

Teacher attrition occurred during the experiment. About half of the 296 teachers who initially volunteered remained through the end of the third year. The teachers who left the study either left the school system, moved to other grades or stopped teaching mathematics. Only one participating teacher specifically asked to be removed from the experiment.

While there was no overall effect on student achievement across the entire treatment group, the researchers found a significant benefit for fifth graders in Year 2 and Year 3 of the experiment: fifth graders taught by teachers who earned bonuses did show gains in test scores. However, the effect did not carry over to sixth grade when students were tested the following year. Springer said this finding raises questions about what is different about fifth grade and what factors –student development, curriculum, teaching and classroom structure – may have played a role.

He also noted that implementation of POINT went smoothly, with no complaints from teachers about the calculation of bonuses, the payment of awards, bonuses they did or did not receive or the fairness of the process. This in itself is a significant finding, Springer said, because historically, teacher associations have opposed performance or merit pay plans, particularly if the pay plan awarded teachers solely on their individual value-added score.

Springer attributed this smooth implementation of the POINT experiment to a broad partnership involving the Metropolitan Nashville School Board and Metropolitan Nashville Public Schools administrators, the Mayor’s Office, the Metropolitan Nashville Education Association, the Nashville Alliance for Public Education, the Tennessee Education Association and the Tennessee Department of Education. The POINT experiment team received guidance and support from these organizations, as well as the participating teachers, throughout the project.

“We believe there is an important lesson here: teachers are more likely to cooperate with a performance pay plan if its purpose is to determine whether the policy is a sound idea, than with plans being forced on them in the absence of such evidence and in the face of their skepticism and misgivings,” Springer said.

The full report is available at http://www.performanceincentives.org/. Archived video of the announcement will be available at that website Sept. 22.

The National Center on Performance Incentives at Vanderbilt University’s Peabody College of education and human development partnered with the RAND Corporation, a nonprofit public policy research institute based in Santa Monica, Calif., in 2006 to complete the study. The POINT experiment was funded by the U.S. Department of Education’s Institute of Education Sciences. Performance bonuses were funded by a private donor.

Springer is an assistant professor of leadership, policy and organizations at Peabody College. For more information about Peabody College, ranked the No. 1 education school in the nation by U.S. News & World Report for the past two years, visit http://peabody.vanderbilt.edu. For more Vanderbilt news, news.vanderbilt.edu.

To read a copy of the report, please click here.