Recognizing the contribution of schools and teachers to their students’ learning is a key element of a performance-based accountability system. Yet determining how to measure such contributions remains unsettled science. Two approaches—student-growth percentile (SGP) and value added (VAM)—have emerged as the most rigorous ways to measure contributions to growth. Even within the value-added category, there are several ways to specify the statistical model. (Ohio, along with a few other states, uses a proprietary model that was developed by William Sanders and is run by statisticians at SAS.) But does the model actually matter, as it pertains to teacher-level ratings? In this paper, researchers compare SGP to VAM, using longitudinal student data from public schools in Washington, D.C.. Interestingly, the authors specify a teacher-level VAM (not the Ohio model) that includes student background characteristics and an SGP model that excludes them.[1] Generally speaking, the researchers found strong correlations between the models (greater than 0.9 for both math and reading). However, the analysts found a number of outlying teachers whose growth estimates were substantially different across the two models. As a result, a minority of teachers (14 percent) would have landed in a different rating category depending on the model. The analysts, however, attribute very little of the differences to the inclusion-exclusion of background variables. So is one model superior? The authors can’t say—there’s no “magic model” to compare them with—but their research demonstrates that different models can alter some teachers’ ratings. Ohio’s state-level policymakers should be aware of these facts, as they try to navigate the choppy waters of teacher-evaluation policy.
SOURCE: Elias Walsh and Eric Isenberg, How Does a Value-Added Model Compare to the Colorado Growth Model? (Washington, DC: Mathematica Policy Research, 2013).
