Dan Goldhaber and Michael Hansen, Is it Just a Bad Class? Assessing the Stability of Measured Teacher Performance CEDR Working Paper 2010-3 (Seattle, WA: University of Washington, Center for Education Data & Research, 2010).
One legitimate criticism of value-added measures (VAM) of teacher effectiveness is that they’re too unstable from year to year to be considered reliable gauges. Because of measurement errors and “statistical noise,” the effectiveness of individual teachers seems to bounce around a lot from year to year. But is it possible that some of this fluctuation is due to actual changes in effectiveness, rather than measurement-and-analysis problems? In this uber-technical study, labor economists Dan Goldhaber and Michael Hansen deploy a longitudinal dataset from North Carolina to track students and teachers over ten years. Using a less conventional model to estimate value-added (one that detects changes within teachers over time), the analysts find that there is indeed a fair degree of instability. Teachers really do seem to have good and bad years. Further, the researchers argue that high stability is not necessarily good—it could, in fact, reduce the motivational effect of such policies (i.e., if the same teachers get rewarded every year, why should the unrewarded teachers try harder?). This study is an important contribution to our understanding of VAM. Still, stability is not the whole story—plenty of other issues deserve attention as we seek to improve this measure (like data quality, transparency of methods, quality of the test, etc).
