Last month, pundits debated the validity of Mississippi’s remarkable gains in student achievement. In the January 2026 issue of Significance, statisticians invoke David Hume and Immanuel Kant while questioning the idea that Mississippi produced improvements for its children, instead arguing that selection bias and tricky math—driven by excluding struggling students from testing cohorts—is behind rising scores. They suggest paltry spending on reading interventions in Mississippi compared to national per pupil spending is particularly suspect. The authors narrow in on third-grade retention policies as the source of artificially inflated achievement. Do retention policies make cohorts appear stronger without real gains, or do they actually drive improvement?
A new working paper by economist Sakib Mahmud breaks down how third-grade reading retention policies impact student outcomes—and not just in Mississippi. He doesn’t set out to settle the debate on retention-cohort samples, and the analysis was conducted before the recent round of statistical disagreements. Still, the findings suggest improvements reflect real learning rather than testing low-performing kids a year later.
Using a staggered-adoption difference-in-differences design, Mahmud analyzes retention policies in Arizona, North Carolina, and Ohio beginning in 2013–14 and in Mississippi starting in 2014–15. Each state implemented literacy reforms during this time period, including statewide third-grade retention policies. He relies on 2010 to 2019 data from SEDA (Stanford Education Data Archive), which creates a uniform scale of state and district scores aligned with NAEP to assess academic trends.
The first comparison looks at student outcomes in retention-mandate states compared to places without statewide policies during the study period. The retention states show improvement in fourth grade reading scores compared to states without retention policies (0.07 to 0.10 standard deviations, a range that’s both statistically and practically significant). Further, duration appears to matter, with post-treatment estimates indicating sustained improvement over time. He also adjusts for Mississippi’s policy timing—the state announced the Literacy-Based Promotion Act in 2013, giving schools extra time before implementing the retention mandate in 2014–15. It appears the preparation period led to even larger gains (0.159 standard deviations).
Isolating the state retention mandate by comparing to states with similar literacy reforms where retention policies are determined at the local level allows Mahmud to examine whether the retention policies are responsible for student gains. Here, the effect almost disappears in Arizona, North Carolina, and Ohio (0.001 standard deviations for this group). Only Mississippi shows a positive and statistically significant improvement due to its state-mandated retention policy specifically, but the effect size drops dramatically to about 0.04 standard deviations in mean language arts scores. In other words, retention played a role, but it wasn’t the primary driver of the state’s improvement.
Retention is never the first choice when a young student struggles to develop critical skills. In Mississippi, it was a piece of a much larger slate of interventions that included progress monitoring, teacher training, and individualized support via literacy coaches. Mahmud indicates that it is this comprehensive approach, rather than the standalone impact of the retention policy, that drives improvements—findings that support insights from the professionals who developed and implemented Mississippi’s reforms. Don’t call it a miracle, they insist, but rather the result of sustained commitment to evidence-backed strategies. As Mahmud writes, success in Mississippi is “primarily due to the intensity of its implementation” rather than the retention policy. Improvements began before the 2014–15 retention policy, when state legislators demanded improvement and backed up that demand with a $15 million annual allocation to support teacher training and other interventions. This amount may seem small to statisticians working outside K–12 education, but it was a substantial commitment in one of the country’s poorest states and appears to be one that led to measurable gains.
There’s no singular solution for improving early literacy. In this study, state retention mandates in Arizona, North Carolina, and Ohio did not appear to increase the effectiveness of their comprehensive reform efforts. Mississippi is different, but not because the state is manipulating student data to create a mirage of improvement. Rather, the evidence suggests that Mississippi’s state-level leadership and significant financial investment—before the retention policy began—drove its improvements in reading. Sustained commitment to evidence-based literacy instruction, coupled with this support, matters far more for student success than state-mandated third grade retention.
SOURCE: Sakib Mahmud, “Does State-Mandated Third-Grade Reading Retention Policy Improve Achievement? Evidence from a Staggered-Adoption Difference-in-Differences Design,” EdWorkingPaper: 25-1352. Annenberg Institute at Brown University (December 2025).
