About 95 percent of public school districts pay teachers according to years of experience and degrees earned—a traditional “step and lane” salary schedule. The other 5 percent have captured a great deal of attention, “spurring rapid growth in the number of research studies” and prompting this meta-analysis of the merit pay literature. Researchers at Vanderbilt pulled data from a few dozen merit pay studies to determine the answers to two primary questions: Do performance-pay programs have an impact on student test scores? And to what extent does program design matter—e.g., individual incentives versus group incentives?
The studies chosen for the meta-analysis went through a rigorous selection process. Analysts reviewed almost 20,000 records via social science databases like ERIC or NBER, ultimately choosing forty-four studies on teacher merit pay in the U.S. and internationally. Almost half were from peer-reviewed publications; all of them met standards for sound research design. Twenty-five percent were randomized control trials; the remainder were quasi-experimental designs. The studies came from a twenty-seven-year period (1989–2016), with most of them occurring after 2005 and with an average treatment effect of four years—in other words, the individual performance pay program under study was on average four years old.
The studies measured the impact of performance pay in the form of gifts, one-time bonuses, and permanent salary increases ranging from $26 to $20,000 in U.S. dollars, with the oddly specific former amount coming from a study in a developing country. Researchers ran a number of additional tests and employed research strategies to ensure their sampling of studies was not biased. For example, they included only the most recent data in studies by the same author(s) to avoid overweighting of data that appeared in multiple similar studies.
The results are good news for merit pay proponents. The average effect of teacher participation in a merit pay program was associated with a .05 standard deviation increase in student test scores—the equivalent of about four-and-a-half weeks of additional learning. Removing the international studies, the estimated effect was still significant at .04 (three weeks of learning). When treated separately, the results for math were higher (.07 standard deviation) than for English language arts (.04), though both were still positive and significant. The researchers don’t hypothesize why merit pay programs in the U.S. had a smaller effect size. One plausible explanation could be that the motivational forces at work under merit pay programs are stronger in countries where teacher pay is lower and where bonuses could represent a much higher percentage of overall pay.
The researchers also found that program design mattered. Teacher pay programs with group incentives (for example, rewards for teams of teachers, or those given at the school level) produced effects that were about twice the size of the average effect in the meta-analysis. Those with on-the-job training components—a requirement for the federal Teacher Incentive Fund—did see learning gains, but ones that weren’t any different from the average effect of all studies in the meta-analysis. In other words, schools might skip teacher training and just invest their marginal dollars in higher pay.
The report authors briefly discuss the theoretical basis for merit pay, noting that it can have a positive impact on student outcomes not just by changing teacher behavior and/or motivation but also because it affects who joins or stays in the profession. Six of the studies included in the meta-analysis found that performance pay had positive effects on teacher recruitment and retention. But this particular analysis can’t really answer the most burning questions it raises, like whether the overall positive effects on student test scores came from reducing teacher turnover, increased teacher effort, or some combination of those (or other) factors. Nor can it help us grapple with basic questions regarding how states should evaluate teachers or define their effectiveness. Finally, the meta-analysis is somewhat lacking in rigor given that the majority of the included studies are quasi-experimental rather than “gold star” randomized research designs. The latter such studies on teacher merit pay have in several instances found no or fleeting effects.
Even so, it’s a worthwhile read and could be especially useful to districts that are backing away from teacher performance pay even as the data shows that it could be good for kids.
SOURCE: Lam D. Pham, Tuan D. Nguyen, and Matthew G. Springer, “Teacher Merit Pay and Student Test Scores: A Meta-Analysis,” Vanderbilt University (April 2017).
