A new Mathematica study examines whether school-level value-added measures adequately capture principals’ effectiveness. Many districts hold them accountable for their schools’ academic performance; this study probes that assumption by asking an important question: Does school-level value added actually reflect the principal’s contribution, or does it mostly reflect other school-level influences (such as neighborhood safety) that are outside the principal’s control?
The authors use longitudinal data from the Pennsylvania Department of Education to study school and principal effectiveness for grades 4–8 from 2007–08 to 2012–13. They include in the data set principals who have been involved in a leadership transition—meaning that, during the analysis period, they started leading a school they had not led before or were replaced by incoming principals. The authors compare departing principals with successors who assumed their positions during 2009–10 to 2012–13. (Alarmingly, 41 percent of schools serving students between the fourth and eighth grades experienced such leadership changes during the study window.) To disentangle the principal’s contribution to growth from the effect of other school-level factors, they sought to isolate the portion of the principal’s impact that is consistent across time and across different samples of students—i.e., the effects on student achievement that principals persistently demonstrate.
Here’s the bottom line: School-level value added is a poor proxy for showing how principals in particular impact student achievement growth over time. (More specifically, no more than 7 percent of any given difference in value-added between two schools reflects persistent differences in the effectiveness of their current principals.) Of course this does not mean that all principals are alike, since they do vary in their value added. In fact, in this sample, the standard deviation of principal effects is at least 80 percent of the size of the standard deviation of teacher effects as estimated by Eric Hanushek in prior studies. What this study is saying is that very little of this substantial variation can be predicted by school-level value added. The likely explanation is that the latter reflects a combination of influences on student achievement outside the principal’s control—and perhaps on other things that principals do not consistently demonstrate from one year to the next.
So, yes, principals matter. The hard part is nailing down exactly how and why.
SOURCE: Hanley Chiang, Stephen Lipscomb, and Brian Gill, "Is School Value Added Indicative of Principal Quality?," Mathematica (June 2016).
