Ohio students have yet to fully recover from pandemic-era declines in achievement. Although English language arts scores for the average Ohio student have rebounded to pre-pandemic levels, test scores in math remain approximately 10 percent of a standard deviation lower—equivalent to students missing roughly one-third of a school year of learning. This average decline masks the fact that current Ohio middle and high schoolers—those who bore the brunt of the pandemic—are behind by approximately 15 percent of a standard deviation.[1] These declines in math achievement are concerning because students’ future incomes, as well as our state’s economic growth, depend heavily on the type of knowledge and skills captured on mathematics exams. In other words, everyone’s economic well-being suffers when math scores decline.
The estimated financial impact of pandemic learning loss, based on the historical relationship between math scores and societal outcomes, is substantial. A 2022 study found that, between 1990 and 2019, an increase of 1 standard deviation in eighth-grade math scores corresponded to an increase of 8 percent in adult incomes, as well as “improvements in educational attainment and declines in unemployment, teen motherhood, incarcerations and arrests.”[2] This estimate implies that each student in the pandemic cohort that lost 15 percent of a standard deviation in math achievement will lose approximately 1.2 percent in lifetime income (approximately $14,400).[3] That’s a total loss of $720 billion nationwide (approximately 50 million students) and $25 billion in Ohio (approximately 1.7 million students). Some have suggested that these estimates are on the low end, however. For example, Stanford economist Eric Hanushek suggests the economic impact of the math achievement decline is comparable to shutting down the U.S.’s $28 trillion economy for a full year. And if such losses persist for subsequent cohorts (Ohio students currently in grades 3-5 remain 5–10 percent of a standard deviation behind and could fall further behind due to financial mismanagement), the losses will be staggering even if the most conservative estimates are used.[4]
Forecasting financial impacts is a tricky exercise that requires strong assumptions, so one should not read too much into the precise dollar figures above. The important takeaway is that it is reasonable to conclude that the economic cost of math achievement declines will be huge. Indeed, since many of these students have already graduated high school (or soon will), we can treat these economic losses as already having been incurred. The cost of failing to bring subsequent cohorts up to pre-pandemic levels will be even larger.
The good news is that, in addition to avoiding future achievement deficits, we can compensate for losses we have already incurred. To do so, however, we need to set our sights higher than simply helping current students recover to pre-pandemic achievement levels. Math achievement did not start declining in 2020. The decline started after math achievement peaked in 2012, in the wake of the Great Recession, when federal stimulus funds ran out and policymakers moved away from test-based school governance. The causes underlying this post-2012 decline are debated, but no one contests that achievement declines predate Covid and that the two decades leading up to 2012 featured significant increases in math achievement. In other words, we know that we are capable of significantly improving students’ math achievement because we’ve done it before. In Ohio, for example, the increase in NAEP scores from 1990 to 2019 (i.e., including the 2012–2019 decline) totaled 60 percent of a standard deviation—a magnitude four times greater than the pandemic-era declines I describe above.
The pre-2012 increase in math scores took twenty years of concerted, bipartisan effort to achieve, and there are reasons to believe that realizing such gains again would be more challenging in our current political environment. Politicians in general have expressed little concern about math achievement and, even if math achievement were a priority, the bipartisan education reform consensus is gone. Nonetheless, the pre-2012 math gains show that we are capable of making up for pandemic learning loss and then some. Doing so would take many years and significant effort, but it is doable if we remind people of the stakes involved. The economic benefits of once again realizing such gains could vastly increase the economic well-being of Ohioans. Importantly, we now also have a wealth of evidence indicating how we might achieve this feat.
What we know about the impact of more public spending
The evidence is clear that we cannot spend our way out of this mess. Recent studies have found that the $190 billion in federal Covid relief funds that districts received had a minimal impact on test scores. Whether those studies focused on all U.S. States (notably the Harvard/Stanford and CALDER/AIR studies) or Ohio (where federal funds exceeded $6 billion), they indicate that the relief dollars had modest effects. Specifically, for every $1,000 dollar spent per pupil, math achievement increased by 0.8 percent of a standard deviation. This estimate implies that additional spending would need to total approximately $10,000 per student (on top of what has already been spent) to address the remaining achievement deficit of 10 percent of a standard deviation. Of course, only a portion of relief funds were meant to address learning loss, but there is little reason to believe that districts would target additional money more efficiently—particularly given what we know about the average impact of spending increases. Indeed, the estimated impact of federal Covid relief funding is approximately what one would have expected based on a recent meta-analysis of the most rigorous quasi-experimental studies.[5]
The best estimates available on the returns to school spending imply that a full recovery for Ohio would require increasing public-school spending (which currently stands at approximately $15,000 per pupil annually) by at least $3,000 per pupil, in perpetuity, to get back to pre-pandemic achievement levels (when spending was lower than it currently is!).[6] “In perpetuity” because pandemic academic losses may be a long-term phenomenon that will continue to affect future cohorts of students, as opposed to a short-term phenomenon limited to a single cohort of students who can recover with a one-time investment. It is prudent to assume that we are in the midst of a new normal characterized by low achievement and chronic absenteeism, and history suggests the impending expiration of Covid relief funds could very well erase the little progress we’ve made.
Unfortunately, there is no way that simply increasing spending by $3,000 per pupil, in perpetuity, is financially worthwhile if increasing math achievement is the goal.[7] Not only is extra spending an inefficient way to address deficiencies in math achievement, it is also likely politically infeasible and won’t enable us to compensate for economic losses we’ve already incurred due to pandemic-era math achievement declines. At the very least, the goal should be to get back to math achievement levels from 2012, not 2019.
The most promising strategies for promoting math achievement in Ohio
Fortunately, extra spending is not necessary. There are at least three options that can yield superior achievement returns at no additional cost:
- Personnel management reforms. As I detailed previously, reforming district personnel management so that schools have discretion in hiring and compensating teachers could be a very effective option, as research consistently indicates large variation in teacher quality and significant achievement improvements when students spend time with a better-than-average teacher. Providing schools with such managerial discretion—by disallowing collective bargaining over rigid teacher compensation schedules, for example—could empower schools to direct their funds towards their greatest teaching needs.[8] That would be particularly helpful for addressing math achievement deficits, as a tight labor market for math teachers has made filling these positions as difficult as ever.[9] “Step and lane” salary schedules that require equal compensation across teachers with similar experience and degrees, regardless of their subject-matter expertise or effectiveness, make it more difficult to attract and retain effective math teachers—particularly in schools serving low-achieving students from low-income households.
- Strategic school closures. Holding schools accountable for student achievement could bring back the kinds of math achievement gains we saw during the golden age of public schooling. As districts right-size their budgets to deal with declining enrollments and the expiration of relief funds, they need to close underperforming schools so that students are directed toward more effective options. This strategy yielded considerable achievement gains in both district and charter schools during the 2000s. Specifically, Ohio students attending urban district and charter schools that closed experienced achievement gains of 7–9 percent of a standard deviation after moving to a new campus—just about the effect size needed to bring student achievement back to pre-pandemic levels. Importantly, these gains were driven by students who ended up in schools that were higher achieving than their closing schools. Since large urban districts often lack the political will to close schools of any kind—let alone prioritizing the closure of poorly performing schools—Ohio should consider enforcing its school closure law for low-achieving district schools, just as it should enforce the test-based automatic charter school closure law that helped strengthen Ohio’s charter sector in the late 2000s.
- Charter school expansion. Ohio’s brick-and-mortar charter schools yield achievement gains in both math and English language arts (relative to district schools) of 2 percent of a standard deviation every 1–2 years. Roughly, this estimate implies that after 5–10 years of charter-school attendance, student achievement is 10 percent of a standard deviation higher than it would have been had those students attended district schools. In other words, if low-income students currently in third grade were to switch from their current district school to a nearby charter school, their average achievement would be back to pre-pandemic levels by the time they graduate, without any additional spending. To realize these gains, one needs to provide families with easy-to-access information on the quality of nearby schools (not all charter schools are high-quality), make it easy to enroll their kids in those schools, and support efforts that allow existing charters to expand (e.g., by supporting the financing of school facilities) if they have a track record of improving student achievement.
Critically, the above strategies—related to personnel management, school closure, and charter expansion—must be directly tied to student achievement. The era of strong growth in math achievement from 1990 to 2012 was characterized by test-based accountability systems. These systems were far more heavy-handed than the strategies I suggest above (the federal No Child Left Behind was unpopular in part for this reason, in spite of some positive impacts), but their essential feature is that they used test scores to drive school closure, changes to personnel management practices, and school choice. Research has shown that these primarily-state-designed, test-based systems incentivized districts to realize superior student achievement with the state funds they received.
Student achievement deficits in mathematics cost all Ohioans, and they will continue to cost us and our children long into the future. Some argue that these expected economic consequences justify throwing more money at the problem, but rigorous research makes clear that this is a poor strategy. We have strong evidence that other options can address pandemic-era achievement declines at no additional cost and far more effectively. Indeed, if we pursue all three of the above strategies—making sure that they are explicitly tied to student outcomes—we may very well exceed pre-pandemic achievement levels in the years to come.
[1] The modest rebound in math scores since the end of the pandemic may be due to relatively higher test scores among younger cohorts of students, though their scores also remain below pre-pandemic levels by 5–10 percent of a standard deviation.
[2] Elena Doty, Thomas J. Kane, Tyler Patterson, and Douglas O. Staiger. 2022. “What Do Changes in State Test Scores Imply for Later Life Outcomes?” NBER Working Paper No. 30701.
[3] The estimate is based on the authors’ estimates that the present value of K–12 students’ lifetime incomes was $1.2 million in 2021.
[4] The achievement deficits are more modest among younger cohorts (for now). Nevertheless, replicating the above calculation across all future students would obviously yield a very large economic loss.
[5] The meta-analysis indicates an increase in spending of $1,000 per pupil for four years leads to an increase of 3.2 percent of a standard deviation in student achievement. Multiplying four years by the estimate of 0.8 percent of a standard deviation yields a four-year effect size of 3.2 percent. This extrapolation requires some strong assumptions about how spending impacts accumulate, however.
[6] This estimate is based on the previously mentioned meta-analysis and assumes that, after four years of additional spending, gains plateau at 3.2 percent of a standard deviation. Increasing spending by $3,000 per pupil would then lead to achievement gains of 9.6 percent of a standard deviation (3 x 3.2 percent)—just below the approximately 10 percent post-pandemic deficit in Ohio’s (and the U.S.’s) math achievement.
[7] The cost-benefit calculations in the Harvard/Stanford study require heroic assumptions (e.g., that the marginal impact of spending would stay the same with additional funds and that the achievement gains would not disappear when funding ended and new cohorts of students entered school) and fail to consider the availability of less costly options.
[8] Economist Eric Hanushek’s suggestion that districts use the federal relief funds to buy out ineffective teachers and increase the compensation of effective ones who agree to increase their class sizes would have enabled districts to improve their teachers without having to deal with the political challenges of reforming collective bargaining.
[9] A recent study using Tennessee data makes the case that the nature of teacher shortages can be highly localized, but the challenges are nonetheless quite general when it comes to recruiting math teachers. The study indicates that just before the pandemic, 79 percent of districts had too few applicants for math positions, as compared to under 20 percent had too few applicants for physical education, social studies, and elementary positions.
