Jesse Rothstein
Princeton University and National Bureau of Economic Research
February 2010 (anticipated)
There's a very serious and scholarly--and to the lay person, nearly unintelligible--exchange happening between academic economists these days on the topic of Value-Added Models (VAM), which rate teacher performance based on student test score gains, rather than snapshots of achievement. In theory, the idea works as follows: Randomly assign students to classrooms such that their average test scores are comparable, and then at the end of the year, give a higher VAM grade to the teachers whose students' test results rise the most. The problem is that, in reality, students are not (and should not be) randomly assigned to teachers--and statistically compensating for this fact turns out to be enormously tricky. As proof, Rothstein breaks down three real-life examples of VAM and applies them to a much larger student sample (approximately 90,000 pupils) than that for which they are typically used. By doing so, he shows that statistical problems that could be hidden in acceptable margins of error in a small sample size are actually larger--and problematic--trends when applied to many more students. While his peers evaluate and reformulate their models based on these findings, it is both reassuring and frightening that the topic has entered the arcana of high economics: reassuring, because there is honest and rigorous debate happening on behalf of better performance measures for our schools, and frightening, because someday someone's going to have to translate these models into terms that teachers and principals can digest. One thing's for sure: Rothstein's cautious recommendation to both include observational data as well as VAM scores in overall teacher evaluation should be taken seriously. Read it here.