Proponents of test-based accountability generally believe that robust systems—those that set high bars for achieving success, generate copious and transparent data, and impose substantive awards or consequences based on progress (or lack thereof)—are enough to boost student achievement. Another school of thought posits that more funding to schools does likewise. A recent working paper from the Annenberg Institute finds that these two policy levers each have their own strengths, but when they interact, accountability systems can increase the efficiency by which additional school dollars are spent.
Twenty five states enacted school finance reforms sometime between 1990 and 2011, following the wave of “adequacy” court cases that began in 1989, which were driven by provisions in state constitutions that required legislatures to guarantee a minimum level of free education to all students. The resulting school funding schemes substantially raised state transfers to low-income districts. At the same time, test-based accountability policies were gaining momentum. Although the federal No Child Left Behind (NCLB) law ensured nationwide adoption of accountability, thirty states adopted consequential school accountability systems prior to NCLB’s 2002 enactment.
Researchers leverage variation in the timing of finance reforms relative to states’ adoption of test-based school accountability systems. They estimate and compare effects on student achievement in twenty-five “treatment states,” broken into two categories. One batch of thirteen states had test-based accountability in place at the time of their school finance reform. They define those states as “accountability” states. Another batch of twelve states did not have accountability systems in place and are designated “non-accountability” states. A control group consisted of twenty-three states without school finance reform during the study period.
They use NAEP restricted-use data for student achievement and data from the F-33 school district finance survey (made available by the National Center for Education Statistics) for revenues and expenditures. They also classify districts into income quintiles, focusing the analysis on the highest- and lowest-income districts.
Their key finding is that the effects of school finance reforms on student learning were driven entirely by those states that had test-based accountability in place at the time. For low-income districts in these states, results show that test scores improve around 0.012 of a standard deviation each year following a school finance reform. In contrast, the corresponding estimate for low-income districts in states without an accountability policy is only about one-third of this size. Moreover, after accounting for trends leading up to the finance reforms, the effect for accountability states remains the same in magnitude, but the effect for the non-accountability states is statistically insignificant. Thus they posit that school finance reforms “cause” test score increases in accountability states but not in non-accountability states. But they also test whether other things could explain the differences instead.
For instance, maybe the effect of the finance reforms on low-income district spending is larger in accountability states. Yet they find that the “resource effects” are largely similar. Specifically, low-income districts in non-accountability states—where they don’t find much evidence of student achievement gains—increase spending by around 9 percent on average following the finance reform; the comparable figure is 7 percent on average in accountability states, so not a huge difference.
All that said, footnote ten (!) in the report explains that the analysis does “not have statistical power to reject the null hypothesis that the effects in low-income districts between accountability and non-accountability states are the same. We can reject the one-sided hypothesis that the effect is larger in non-accountability states, however.” This somewhat covert admission surely weakens their bottom-line finding that effects are driven entirely by accountability states. Still, analysts performed enough due diligence to demonstrate some complementarity between test-based accountability systems and school finance reforms.
That means accountability policies can improve the efficiency of additional dollars spent, likely because they create rewards or sanctions for schools based on student performance. Of course, it makes sense that holding schools responsible for performance could make them think harder about how to direct new monies to meet the goals set out in their accountability plans.
Unfortunately, the concepts of consequential accountability and school finance have both taken on new meanings in this pandemic, and the risk of losing both—because of the unprecedented crisis and the economic downturn—is real in the wake of it. For the sake of students, we should hang on to both however we can and look forward to the time when they can complement one another once again.
SOURCE: Christian Buerger, Seung Hyeong Lee, & John D. Singleton, “Test-Based Accountability and the Effectiveness of School Finance Reforms,” retrieved from Annenberg Institute at Brown University (August 2020).
