This study examines the impacts of the most recent iteration of the D.C. Opportunity Scholarship Program, which was first established by Congress in 2003 but was allowed to expire in 2009 before being reauthorized again in 2011.
Overall, the study finds that participating in the program for one year reduced achievement in math and reading by 7.3 and 4.9 percentile points, respectively. However, because the study only includes data from the first year of program participation, these numbers should be interpreted with caution.
Interestingly, for the 68 percent of students in grades K–5, participating in the program reduced achievement in math and reading by 14.7 and 9.3 percentile points, respectively; whereas, for the 32 percent of participants in grades six through twelve, it was associated with statistically insignificant increases of 7.6 and 4.9 percentile points. Across all grades, the program had a little apparent impact on the 71 percent of students applying from low-performing “schools in need of improvement” (SINI), who were given priority in the scholarship lottery. However, for the 29 percent of students enrolled in a non-SINI school, it reduced math and reading achievement by 18.3 and 14.6 percentile points, respectively. Finally, on a slightly more positive note, the study finds that the program had a statistically significant positive impact on parents’ perceptions of school safety. However, it finds no significant effect on parental satisfaction.
The study’s broadly negative findings stand in sharp contrast to the positive results of an earlier study of the first version of the program, which found that the program boosted graduation rates by 21 percentage points after four years, and had a marginally significantly positive impact on reading scores, but no statistically significant impact on math scores. Since both studies used an experimental design that compares winners and losers of the scholarship lottery and included an identical number of private schools (fifty-two) that accounted for a similar fraction of all D.C. private schools (roughly 55 percent), the differences in their results probably aren’t attributable to differences in the sample, the research design, or program participation.
That leaves two plausible explanations: First, the negative results of the latest study may reflect the “transition costs” associated with switching to a new school, which numerous studies have demonstrated can be erased by the second or third year after the switch, assuming the new school is higher performing than the old one. Second, it’s possible that the performance of D.C.’s public schools—including both its district schools and its many charter schools—may have improved between the first study and the second. Indeed, an analysis of NAEP scores in twelve major urban school districts found that Washington, D.C., showed the most improvement between 2005 and 2013, after controlling for student demographics.
Coming as it does on the heels of similar studies of voucher programs in Indiana, Louisiana, and Ohio—all of which found negative effects for program participants—the D.C. study has added more fuel to the already fiery debate over private school choice. However, like the Indiana and Louisiana studies, this one is highly preliminary, so it probably makes sense to reserve judgment until more years of data are available.
SOURCE: Mark Dynarski et al., “Evaluation of the D.C. Opportunity Scholarship Program Impacts After One Year,” Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance (April 2017).
