More than 70 percent of school districts nationwide reported expanding their summer school programming in 2022, mostly using federal ESSER funds, in one of many efforts to mitigate pandemic-era learning loss. A new study published in the American Educational Research Journal provides the first large-scale evaluation of the effectiveness of that programming.
The 14-strong research team estimates the effect of summer school on learning loss recovery by tracking the academic progress of students who attended programs in the summer of 2022 across eight school districts participating in the Center for Education Policy Research’s Road to Recovery project. The districts were mostly large and urban (including Dallas, Tulsa, and one that remains anonymous in the report), with a total enrollment of about 400,000. They served higher percentages of Black and Latino students (56 percent) and students eligible for free and reduced-price lunch (55 percent) than the national average (33 and 45 percent, respectively). Based on the Education Recovery Scorecard, all had seen sharp decline in math and reading test scores between 2019 and 2020, with varying levels of recovery seen through the end of the 2021–22 school year. On average, the remaining learning loss was approximately 0.19 standard deviations (SD) in math and 0.02 SD in reading, both higher than the national average (0.15 and 0.09 SD, respectively).
Their analytic sample comprises nearly 130,000 students expected to enter grades one through eight in the fall of 2022 and who had NWEA MAP Growth scores in reading and math for both the spring and fall 2022 test administrations. Using value-added models that control for students’ demographics and prior achievement, the research team estimated an overall, though small, positive effect of summer school on math test achievement in all eight districts, ranging from 0.03 to 0.04 SD. The longer students attended (hours and days), the higher the positive impact. However, since attendance was estimated at just 13 percent of K-8 students per district, summer school participation was estimated to have remediated a mere 2 percent of the districts’ loss in math. To add to the bad news, only two districts saw statistically significant impacts on reading test achievement, with the others—and the overall impact—statistically indistinguishable from zero. There was very little variation in impact among subgroups of students, as well. Even worse: Some of the districts actually combined summer programming with tutoring—to no better effect than summer school alone in either math or reading.
Interestingly, these uniformly unimpressive outcomes result from eight disparate efforts, with many permutations of scheduling, duration, content, staffing, and student recruitment across districts and, one assumes, even across different schools in the same district. This report goes into a little detail on matters of implementation in the specific study districts, but one member of the research team has previously conducted a much deeper analysis of Covid recovery programing in four anonymous Road to Recovery districts. Challenges cited there include reaching the neediest students, maintaining student attendance, finding adequate and effective staff, and a lack of central office support for out-of-school-time programming. These speedbumps, similar to those noted in this report, would seem to be universal across districts, schools, and intervention types, as a new and “sobering” analysis of tutoring impacts has also shown.
It seems clear now that recovery from Covid-era learning loss was never going to happen quickly—despite money[1], good intention, and effort—and 2022 was far too soon to see it. At least some states and schools appear to be making headway in more recent years, even if it is too late for the millions of impacted students who have already finished school in the interim.
SOURCE: Ian Callen et al., “Summer School as a Learning Loss Recovery Strategy After COVID-19: Evidence from Summer 2022,” American Educational Research Journal (July 2025).
[1] There is no discussion of the cost of providing summer school in the paper. Analysis of ROI—which some research shows can be high for summer programming—would add some nuance to these generally lackluster findings.
