A new working paper by Calder analyzes whether federally funded school turnarounds in North Carolina have impacted student outcomes.
The study uses achievement, demographic and descriptive data about teachers and principals for K–8 schools in the 2010–2014 school years, as well as teacher survey data from North Carolina’s biannual Teacher Working Conditions Survey. The data set includes eighty-five elementary and middle schools that were subject to the state’s school turnaround program, which was funded by federal Race to the Top funds. Most schools used a “transformation model” of turnaround that required replacing the principal, along with other instructional interventions like increasing learning time (but no teacher terminations).
The analysis uses a regression discontinuity design, wherein assignment to the treatment and control groups are based on a school falling right above or below the cut point for placement into the turnaround program. The idea is that whether a school is just above or just below the cut is essentially random.
The key findings: The program had a mostly negative effect on test scores in math and reading—especially so in math. It decreased average attendance by between 0.4 and 1.2 percentage points in 2012 (the first full year after the program was implemented); resulted in a higher rate of suspensions in 2012; and led to a drop in school-wide passing rates in math and reading—more so for particular subgroups in each subject. There were also consistently large decreases (0.36–0.64 standard deviations) in reading scores for high-achievers. The programs had no effect on teachers’ perceptions of the quality of their schools’ leadership (recall most had new leaders); nor did it impact much their perceptions of the quality of professional development, which teachers got more of (they also had to complete more required paperwork). The program also led to an exodus of high-achieving students—which contributed to the decrease in scores, but was not the sole reason.
Pretty dismal stuff.
It is virtually impossible to identify why all of these negative outcomes occurred. But for starters: Was the new principal any better than the last? Perhaps turning over most of the staff would be useful. Perhaps it takes longer than three years to see impacts. Perhaps disadvantaged kids need wraparound services too. Perhaps a district-wide approach is better than a school approach. Perhaps closure is better still. Like most interventions in education, there is no one formula for success. And the black box is pretty dark inside.
SOURCE: Jennifer A. Heissel and Helen Ladd, "School Turnaround in North Carolina: A Regression Discontinuity Analysis," CALDER (March 2016).
