Skip to main content

Mobile Navigation

  • National
    • Policy
      • High Expectations
      • Quality Choices
      • Personalized Pathways
    • Research
    • Commentary
      • Gadfly Newsletter
      • Gadfly Podcast
      • Flypaper Blog
      • Events
    • Covid-19
    • Scholars Program
  • Ohio
    • Policy
      • Priorities
      • Media & Testimony
    • Research
    • Commentary
      • Ohio Education Gadfly Biweekly
      • Ohio Gadfly Daily
  • Charter Authorizing
    • Application
    • Sponsored Schools
    • Resources
    • Our Work in Dayton
  • About
    • Mission
    • Board
    • Staff
    • Career
Home
Home
Advancing Educational Excellence

Main Navigation

  • National
  • Ohio
  • Charter Authorizing
  • About

National Menu

  • Topics
    • Accountability & Testing
    • Career & Technical Education
    • Charter Schools
    • Curriculum & Instruction
    • ESSA
    • Evidence-Based Learning
    • Facilities
    • Governance
    • High Achievers
    • Personalized Learning
    • Private School Choice
    • School Finance
    • Standards
    • Teachers & School Leaders
  • Research
  • Commentary
    • Gadfly Newsletter
    • Flypaper Blog
    • Gadfly Podcast
    • Events
  • COVID-19
  • Scholars Program
Flypaper

A flawed study of Indiana’s voucher program

Jeremy Smith
9.23.2021
Getty Images/Image Source

A recent study looks at the impact of Indiana’s Choice Scholarship Program (ICSP) on school proficiency rates in math and reading at traditional public schools. Its finding that public school proficiency rates on state tests decreased in the long term after ICSP was introduced seems to suggest that the program has a negative long-term impact on public school students. But its reliance on school-level proficiency rates as opposed to student-level data means we can draw few meaningful conclusions about ICSP’s real impact.

The scholarship program, which serves low- and (more recently) middle-income families, is one of the most expansive school voucher programs in the U.S. today. Nearly eighty percent of Indiana students are eligible for the vouchers, although fewer than 4 percent currently make use of them. Because the money follows the student in Indiana, public schools have an incentive to improve their operations to compete with ICSP.

The study has the trappings of a rigorous regression analysis, including controls for student demographics and a large sample size (1268 schools in 283 districts), leading to some impressively-precise-sounding results. It found that a 1-percentage-point increase in a school district’s participation rate in ICSP resulted in a 0.922-percentage-point decrease in a public school’s math proficiency rate and a 0.840-percentage-point decrease in that school’s reading proficiency rate.

At first, one might conclude that the program harms student outcomes, but this is a statistical illusion. To understand ICSP’s impact on public school students, student-level data are needed. School-level data on proficiency rates are not enough. Suppose, for example, that students using the program in a school district are almost all achieving proficiency on state tests. If a 1-percentage-point increase in ICSP participation from such students results in a less than 1-percentage-point decrease in proficiency rates among the other students, then ICSP is actually improving student outcomes.

Even if some of the students using the program are not proficient, the results could be misleading. Imagine a population of one hundred students. Suppose that, before ICSP existed, all of these students would have attended the same public school, and that fifty of those students would have achieved proficiency at that school. That’s a proficiency rate of 50 percent. With the ICSP in place, however, twenty of these students decide to use it to attend other schools, sixteen of whom would have achieved proficiency at the public school. At the same time, the public school makes some improvements so that four students who would have not have been proficient under the old system now achieve proficiency. Suppose also that all students who would have achieved proficiency at the public school without the scholarship program also achieve proficiency at the public school with ICSP. Overall, the effect on students was positive, since four more are now proficient. But our public school now has thirty-eight students who are proficient, and forty-eight students who are not proficient, a proficiency rate of 44 percent!

In other words, this study does not reveal much about the ICSP’s competitive effects on public schools because proficiency rates are a flawed measure of school performance. That’s unfortunate because studies of the long-term impact from school voucher policies are important.

SOURCE: Yusuf Canbolat, “The Long-Term Effect of Competition on Public School Achievement: Evidence from the Indiana Choice Scholarship Program,” Education Policy Analysis Archives (July 2021).

Policy Priority:
High Expectations
Topics:
Evidence-Based Learning
Governance
Private School Choice

Jeremy Smith is a research assistant with the Thomas B. Fordham Institute. A Virginia native, Jeremy holds a bachelor’s degree in physics from Hamilton College and a master’s degree in physics teaching from the University of Virginia. He is currently working toward a master’s degree in education policy at American University. Prior to joining Fordham, Jeremy worked as a high school teacher, university instructor, and…

View Full Bio

Sign Up to Receive Fordham Updates

We'll send you quality research, commentary, analysis, and news on the education issues you care about.
Thank you for signing up!
Please check your email to confirm the subscription.

Related Content

view
Gadfly Bites
School Funding

Gadfly Bites 5/16/22—Wiggle room indeed

Jeff Murray 5.16.2022
OhioOhio Gadfly Daily
view
Quality Choices

When it comes to the Charter Schools Program, little details can have big impacts

Christy Wolfe 5.13.2022
NationalFlypaper
view
Ohio charter news logo
School Choice

Ohio Charter News Weekly – 5.13.22

Chad L. Aldis, Jeff Murray 5.13.2022
OhioOhio Gadfly Daily
Fordham Logo

© 2020 The Thomas B. Fordham Institute
Privacy Policy
Usage Agreement

National

1016 16th St NW, 8th Floor 
Washington, DC 20036

202.223.5452

[email protected]

  • <
Ohio

P.O. Box 82291
Columbus, OH 43202

614.223.1580

[email protected]

Sponsorship

130 West Second Street, Suite 410
Dayton, Ohio 45402

937.227.3368

[email protected]