David Figlio, a researcher at Northwestern University, recently released his seventh-annual evaluation of Florida’s Tax Credit Scholarship Program. The study uses scholarship students’ results on national assessments, like the Stanford Achievement or Iowa Test of Basic Skills, to examine whether they are making year-to-year gains. (Elsewhere in this issue, I review the study in greater detail.) The Sunshine State’s program, which enrolls nearly 60,000 students, is akin to Ohio’s EdChoice and Cleveland scholarship (a.k.a., “voucher”) programs.
One of the study’s findings was particularly striking: Private schools in Florida, especially Catholic ones, appear to have a relatively larger impact on scholarship students’ reading scores than math. Across all schools, Figlio found that voucher students made a 0.1 percentile gain in reading but posted a loss of -0.7 percentiles in math. The overall math-reading difference may or may not be trivial—there is no test of statistical significance across the subject areas. But larger differences in reading-to-math gains appear when gains are disaggregated, for example, by religious affiliation:[1] Consider the large annual gain in reading for voucher students attending a Catholic school (1.98 percentiles) versus the slight loss in math (-0.25). True, the larger reading gains don’t hold across all school types—non-religious schools seem to make a fairly big difference in math—but it does seem like many of Florida’s private schools are having greater success boosting reading scores.[2]
Table 1: Average reading and math gains of Florida scholarship students by schools’ religious affiliation, 2011-12 to 2012-13
[[{"fid":"113541","view_mode":"default","fields":{"format":"default"},"type":"media","attributes":{"height":"171","width":"482","class":"media-element file-default"},"link_text":null}]]
Source: David Figlio, Evaluation of the Florida Tax Credit Scholarship Program, pg. 23.
This is especially noteworthy, because many of today’s popular reforms—such as high performing charter schools and Teach For America—are almost always more successful in boosting math than reading scores. If Catholic schools are a counter-example, that’s important to know—and to know why. At the least, we might want to understand how they are teaching reading and consider replicating it in district and charter schools.
The story seems to be similar in the Buckeye State, where data on Ohio’s voucher students show more success in reading and less in math. In the Ohio Department of Education’s recently released data for 2012-13, we notice that reading proficiency rates for African-American voucher students, many of whom attend a Catholic school, far outpace their math proficiency rates—sometimes by over thirty percentage points as table 2 shows.[3] For instance, 72 percent of voucher students in Columbus reached reading proficiency, while just 44 percent did so in math. As a reference point, reading proficiency for Ohio African-Americans in public schools statewide was, on average, thirteen points higher than math across grades three through ten in 2012-13.
This analysis is of course different than what I presented on Florida for several reasons. Most importantly, the table below reports proficiency, a one-year snapshot of achievement and a limited basis for understanding the quality of private schools. It does not indicate the growth students have made over time (i.e., “gains,” which could be understood as the impact of a school on achievement). Ohio’s data also uses state assessment results, not national ones (and gains relative to national percentile ranks); and obviously, the data are not disaggregated by religious affiliation. Caveats aside, Ohio’s voucher test data do hint that private schools—and perhaps Catholic ones—might be more successful in reading than math.
Table 2: Proficiency rates for Ohio African-American voucher students by their district of residence, 2012-13
[[{"fid":"113542","view_mode":"default","fields":{"format":"default"},"type":"media","attributes":{"height":"266","width":"547","class":"media-element file-default"},"link_text":null}]]
Source: Ohio Department of Education
Overall, so far as I can tell, private schools that enroll voucher students tend to fare better in reading relative to math. But higher reading proficiency may be an illusion too: We can’t ascertain whether the difference is attributable to the school per se or to outside factors. (It’s long been known that reading is more closely related to students’ family and home background than is math.) So, in the end, we eagerly await a rigorous analysis of Ohio’s voucher data (at a student-level as in Florida) that can help us better grasp the impact of private schools on voucher students’ achievement over time.
[1] The study breaks out math and reading gains in a four different ways: (1) by the percentage of scholarship students educated in the school; (2) the length of the school year; (3) student-to-teacher ratio; (4) religious affiliation. The math-reading differences seemed most interesting from a religious-affiliation perspective.
[2] It’s worth noting that Paul Peterson and Elena Llaudet estimated higher reading gains (using NAEP) for private schools, especially so for Catholic ones, relative to math (see pg. vi). The study, however, was not restricted to just voucher students. Also of note is Jonathan Plucker, et al’s finding that Cleveland voucher students who remained in the program for seven years had significantly higher achievement than their public-school counterparts in language, science, and social studies. The study did not uncover differences in math (see pgs. 166-168).
[3] The data are reported by race, gender, and grade and subject of the exam. There are no aggregate statistics for voucher students by their district of residence, so I choose to compare math-reading proficiency within just African-American students.
