Late last year, researchers Sarah Cohodes and Susha Roy partnered with the MIT Department of Economics to release a paper summarizing the results of lottery-based charter studies. The topline conclusion is straightforward and promising:
Existing evidence shows that charter schools can improve academic achievement and longer-term outcomes like four-year college enrollment, particularly among lower-performing students, non-white students, low-income students, and students with disabilities.
That was the inspiration for an easy column if I’ve ever seen one. Unequivocally, charter schools benefit students, and anyone who disagrees needs to just read the evidence already. Effects vary by school, of course, but overall results are positive. One large coffee with my word processor and we’ll all be on our merry way by close of business.
The reviewed studies paint charter schools in glowing colors. Charters have a “large positive impact on standardized test scores in both math and ELA,” and also non-standardized measures of academic achievement such as end-of-unit exams. Charter school students are less likely to partake in “extremely risky behaviors,” such as sex without contraception, or end up incarcerated. And finally, attendance at such schools boasts enrollment in four-year colleges and civic participation.
What’s not to love? If only social science research were so simple.
Hesitations arise in the nature of the research itself. Selection bias riddles gross comparisons between charter and traditional public schools. Are students who opt to enroll in charter schools—and so complete all necessary paperwork and jump through every loophole—from more stable families? Do charter schools enroll more affluent or poor students? Broad comparisons may only measure baseline differences in students, not effectiveness of the schools.
To account for this causal confusion, one approach—here under review in MIT’s white paper—is to rely on a natural experiment that arises in the system. When oversubscribed, charters are normally required by law to enroll students through a lottery, which makes for a randomized-controlled trial by another name. How do students compare who enrolled but did or did not receive acceptance via lottery? Like medical trials, such built-in control and experimental groups allow researchers to tie outcomes to the treatment itself—in this case, enrollment in a charter school.
However, there’s a bit of a trap hidden in this research: A lottery requires oversubscribed charter schools, those that have more students applying than there’s room for. As a result, lottery studies examine the most in-demand schools, accounting for only 26 percent of charter schools. How much does it really tell us to measure the effectiveness of what may be just the most effective quarter of the charter sector? That the most popular (and, so we would hope, most effective) schools are in fact better at raising student achievement? Such research tells us very little about the performance of the charter sector overall.
When examined alongside other study designs, however, something more like a proper consensus arises. Perhaps the largest, most comprehensive investigations into the charter sector, the CREDO studies from Stanford (the most recent of which released in 2023), compare students at charter and traditional public schools with similar prior achievement—another way to ferret out selection bias or causal effects—and find that charter schools have a positive if modest impact on student achievement.
The topline conclusions of both the CREDO and lottery studies regarding the charter sector as a whole mask a more impressive result: urban charters, in particular, wildly outperform similar traditional public schools. A 2015 study from CREDO found that attendance at an urban charter provides students the equivalent of twenty-eight extra days of learning in English and forty extra days of learning in math.
The most solid conclusion to be drawn from the Cohodes-Roy paper: Good schools really do matter. And I don’t mean to be flippant. There’s an influential strand of educational thought that contends that schools really don’t matter that much. Other factors such as family background or general intelligence are so strong that they overwhelm any impact that a school could have. I’ve been guilty of such fatalism myself.
But these lottery studies—even if they don’t vindicate the charter sector as a whole—do demonstrate that quality schools can alter a child’s trajectory, improving their odds of escaping poverty, staying out of prison, going to college, and leading a successful life. Such institutions are perhaps the single most promising, scalable policy to shrink the achievement gap between low-income and affluent students.
There’s a lot we still don’t know about the charter sector. What’s more, simply placing the label “charter” on a school by no means guarantees success; the sector has many lackluster schools. Even so, as it stands, continued expansion of the sector overall seems to be a largely all-win, no-lose scenario. And Cohodes and Roy have provided an impressive research review in support of that proposition.
