Alan B. Krueger and Pei Zhu
Princeton University
April 2003
This paper closely examines - and reanalyzes - the data from the private voucher experiment in New York City, which were originally analyzed in a 2002 study by David Mayer, Paul Peterson, David Myers, Christina Tuttle and William Howell. The original work generated headlines based on the finding that black students using vouchers made significant academic gains. Here, Krueger and Zhu contest these claims and state that "the safest conclusion is probably that the provision of vouchers did not lower the scores of African American students." The controversy is rooted partly in the differing statistical methods of the two groups. For example, Krueger and Zhu argue that Peterson et al. chose an unnecessarily complicated design (which led to minor errors) and improperly excluded scores of those students for whom baseline data (initial test scores) were not available. Non-statisticians may have difficulty following the intricacies of these arguments, but a response to the Krueger and Zhu paper by Mayer and Myers (available at http://www.mathematica-mpr.com/PDFs/anotherlook.pdf) seems to accept the legitimacy of the methods suggested by Krueger and Zhu. Another matter of dispute is the definition of "black." The original research determined the race of the student by that of the mother alone; Krueger and Zhu broaden this to define as black any student for whom either parent is black. The new definition expands the black population and has the effect of reducing the test-score gains in this group to the point that they're so small as to be statistically insignificant. This certainly is interesting, but in the end it's unfortunate that this study has generated headlines as exaggerated as those generated by the original study (for example, see Education Week's "Study: No Academic Gains From Vouchers for Black Students"). Such headlines obscure the fact that it's far too soon to generalize about the impact of vouchers on test scores, given that all we now have are a few isolated trials of this potentially important reform. And, as Mayer and Myers point out, the original study had "cautioned readers about placing too much emphasis on the average impact for African American students." The original report is available online at http://www.mathematica-mpr.com/PDFs/nycfull.pdf, and Krueger and Zhu's piece is available at http://www.ers.princeton.edu/workingpapers/1_9.pdf. Peterson also plans to issue a response to Krueger and Zhu.
