In February 2020, California settled a case which alleged that the state was violating the right to an education by sending kids to schools that didn’t teach them how to read. Among the terms of the settlement, the state agreed to create the Early Literacy Support Block Grant (ELSBG)—a targeted initiative to improve reading scores in the lowest-performing elementary schools in the Golden State—to the tune of roughly $50 million. Did this resolution achieve its important aims? A recent study by Stanford’s Tom Dee and Sarah Novicoff looks into the achievement impacts of the first two years of implementation to find out.
To conduct their analysis, Dee and Novicoff identified a group of 5,256 unique elementary schools (district and charter) with reading achievement data for grade three in each of seven tested school years between 2014–2015 and 2022–2023 (sans 2020 and 2021 due to the pandemic). The state identified ELSBG-eligible schools by averaging the percent of grade three students across the 2017–18 and 2018–19 academic years who scored at the lowest of the four levels on the state’s English language arts assessment and weighting the average by the number of test-takers in each of the years. The ELSBG program was intended to improve reading in the lowest performing seventy-five elementary schools in the state. Originally, all seventy-five eligible schools agreed to participate, although the treated sample ultimately ended up totaling sixty-six. The rest of the 5,000-some schools comprised the comparison group (more on that shortly).
Based on individual school needs, ELSBG schools were awarded three-year grants that averaged roughly $642,000 per school. Besides the funding, schools received external support from the County Office of Education, lots of professional development (separate from the grant funding), and spending flexibility (with a requirement to supplement not supplant existing funds), along with a requirement to submit quarterly reports on progress toward goals. They had to develop a three-year literacy plan proposing how they would improve outcomes, with allowable expenditures on instructional coaches, diagnostic tools, instructional materials, tutoring, parental outreach, and mental-health resources to name a few. The state sponsored thirty-six sessions on the science of reading before the literacy plan was due, plus an “Online Elementary Reading Academy” during implementation. Early analysis found that staff compensation (i.e., salaries and benefits) represented 69 percent of the budgeted expenditures for the ELSBG funds in the first year.
Analysts compare the changes in outcomes over time between students involved in the school-wide program and not. However, the treatment schools were already trending downward relative to the comparison schools before the grant began, so the analysts take multiple steps to match treatment and comparison schools, including weighting the data in such a way that aligns pre-treatment trends across both groups.
The key finding is that the grant had positive and statistically significant effects on grade three ELA test scores. Specifically, it increased those scores by 0.14 standard deviations among third graders served by the targeted schools over the first two years of the grant. Similarly, it increased the share of students performing at Level 2 or higher by 20 percent (i.e., a 6-percentage-point gain relative to a pre-treatment baseline of 30.5 percent). And it led to smaller, cross-subject gains in grade three math achievement (i.e., 0.11 standard deviations) but, not surprisingly, no spillover effects among grade five students in math or ELA, who were outside the program’s focus. (Recall that they didn’t have data for school year 2020 or 2021, so the same third graders couldn’t be tracked through fifth grade.)
These are modest but positive impacts, and the analysts note that, compared to interventions that seek to reduce class size, the average one-year cost of $1,144 per pupil for ELSBG was on par for the outcomes achieved—and incidentally, way more promising than the disastrous math reform in San Francisco that Novicoff and Dee also evaluated.
Will a third year of ELSBG data show continued improvement? What about after the funding runs out (which occurred in June 2024)? Will the benefits fade? Or perhaps the wider implementation of the science of reading nationwide—which shares commonalities with ELSBG—will help the third graders of California finally reach a new level of literacy achievement and actually maintain it. Time and data will tell.
SOURCE: Sarah Novicoff and Thomas S. Dee, “The Achievement Effects of Scaling Early Literacy Reforms,” Annenberg Institute at Brown University (December 2023).
