The Education Gadfly Weekly

Shortly after Ohio lawmakers enacted a new voucher program in 2005, the state budget office wrote in its fiscal analysis, “The Educational Choice Scholarships are not only intended to offer another route for student success, but also to impel the administration and teaching staff of a failing school building to improve upon their students’ academic performance.” Today, the EdChoice Scholarship Program provides publicly funded vouchers to more than eighteen thousand Buckeye students who were previously assigned to some of the state’s lowest-performing schools, located primarily in low-income urban communities. Yet remarkably little else is known about the program.

Which children are using EdChoice when given the opportunity? Is the initiative faithfully working as its founders intended? Are participating students blossoming academically in their private schools of choice? Does the increased competition associated with EdChoice lead to improvements in the public schools that these kids left?

Fordham’s new study utilizes longitudinal student data from 2003–04 to 2012–13 to answer these and other important questions. 

Three key findings:

• Student selection: The students participating in EdChoice are overwhelmingly low-income and minority children. But relative to pupils who are eligible for vouchers but choose not to use them, the participants in EdChoice are somewhat higher-achieving and less economically disadvantaged.
• Competitive effects: EdChoice modestly improved the achievement of the public-school students who were eligible for a voucher but did not use it. The competition associated with the introduction of EdChoice appears to have spurred these public-school improvements.
• Participant effects: The students who used vouchers to attend private schools fared worse on state exams compared to their closely matched peers remaining in public schools. Only voucher students assigned to relatively high-performing EdChoice eligible public schools could be credibly studied.

Dr. David Figlio, Orrington Lunt Professor of Education and Social Policy and of Economics at Northwestern University, led the research.

Shortly after Ohio lawmakers enacted a new voucher program in 2005, the state budget office wrote in its fiscal analysis, “The Educational Choice Scholarships are not only intended to offer another route for student success, but also to impel the administration and teaching staff of a failing school building to improve upon their students’ academic performance.” As economist Milton Friedman had theorized decades earlier, Ohio legislators believed that increased choice and competition would boost education outcomes across the board. “Competition” in the words of Stanford’s Caroline Hoxby, “would be the proverbial rising tide that lifts all boats.”

Today, the EdChoice program provides publicly funded vouchers (or “scholarships”) to more than eighteen thousand Buckeye students, youngsters previously assigned to some of the state’s lowest-performing schools, located primarily in low-income urban communities.[1] That much is known. Yet remarkably little else is known about the program. Which children are using EdChoice when given the opportunity? Is the initiative faithfully working as its founders intended? Are participating students blossoming academically in their private schools of choice? Does the increased competition associated with EdChoice lead to improvements in the public schools that these kids left?

The present study utilizes longitudinal student data from 2003–04 to 2012–13 to answer these important questions. Specifically, the analysis utilizes the results from state tests—which all EdChoice students are required to take—to examine the vouchers’ effects on two groups of pupils. First, the study inspects the scores of public school students who were eligible for vouchers—but did not take one—in order to gauge the competitive effects of EdChoice (i.e., its impact on traditional public school students and their schools). Second, it examines the academic impact of EdChoice on those students who actually use the vouchers to attend private schools.

This is the first study of EdChoice that uses individual student-level data, allowing for a rigorous evaluation of the program’s effectiveness. (Earlier analyses by Matthew Carr and Greg Forster used school-level data to explore its competitive impact.) To lead the research, we tapped Dr. David Figlio of Northwestern University, a distinguished economist who has carried out examinations of Florida’s tax credit scholarship program. He has also written extensively on school accountability, teacher quality, and competition. Given his experience, Dr. Figlio is exceptionally qualified to lead a careful, independent evaluation of Ohio’s EdChoice program.

In this report, he sets forth three main findings:

• While the students who participate in EdChoice—the pupils who actually use a voucher to attend private schools—are primarily low-income and minority children, they are relatively less disadvantaged than other voucher-eligible students. Figlio reports that more than three in four participants are economically disadvantaged, and three in five are black or Hispanic. Viewed in relation to Ohio’s public school population as a whole, students in EdChoice are highly disadvantaged—not surprising, given eligibility rules that require participants to have attended a low-achieving public school. But relative to students who are eligible for vouchers but choose not to use them, the participants in EdChoice are somewhat higher-achieving and somewhat less economically disadvantaged. This finding may be, in part, an artifact of the program’s basic design: It allows private schools to retain control over admissions, and a child must gain admission into a private school before he or she can apply for a voucher. This multi-step process might be more easily navigated by relatively more advantaged families; their children might also be more likely to meet the private schools’ admissions requirements.
   
• EdChoice improved the achievement of the public school students who were eligible for the voucher but did not use it. When examining the test results of pupils attending public schools just above and below the eligibility threshold, the analysis finds that achievement in math and reading rose modestly as a result of voucher competition. (The analysis leverages the state’s voucher eligibility rules to isolate voucher competition from other potential competitive effects, such as charter schools.) In other words, the voucher program has worked as intended when it comes to competitive effects. Importantly, this finding helps to address the concern that such programs may hurt students who remain in their public schools, either as a result of funds lost by those schools or the exodus of higher-performing peers. Quite the opposite has occurred in the case of EdChoice: Achievement improved when the voucher program was introduced and public schools faced stiffer competition (and the risk of losing their own students).
   
• The students who use vouchers to attend private schools have fared worse academically compared to their closely matched peers attending public schools. The study finds negative effects that are greater in math than in English language arts. Such impacts also appear to persist over time, suggesting that the results are not driven simply by the setbacks that typically accompany any change of school.

Let us acknowledge that we did not expect—or, frankly, wish—to see these negative effects for voucher participants; but it’s important to report honestly on what the analysis showed and at least speculate on what may be causing these results. One factor might be related to the limits of credible evaluation: while the rigor of the methodology ensured “apples-to-apples” comparisons of student achievement, Dr. Figlio was limited to studying students who attended (or had left) public schools that were just above or below the state’s cutoff for “low-performing.” By definition, this group did not include the very lowest-performing schools in the state. It’s possible that students who used a voucher to leave one of the latter schools might have improved their achievement; we simply cannot know from this study. The negative effects could also be related to different testing environments—higher stakes for public than private schools—or to curricular differences between what is taught in private schools and the content that’s assessed on state tests. Finally, although this analysis does not enable us to identify individual schools as high- or low-performing, it may be the case that some of the private schools accepting EdChoice students are themselves not performing as well as they should.

***

Taken as a whole, the results reported here for Ohio’s EdChoice program—one of the nation’s largest voucher programs—are a mixed bag. The program benefitted, albeit modestly, thousands of public-school students; yet among the somewhat small number of participants studied here, the results are negative. The study mirrors important trends that can be seen in other voucher research. The modest, positive competitive effect on public school achievement replicates findings from jurisdictions like Florida, Louisiana, and Milwaukee, findings that also offered evidence that voucher competition improved public school outcomes. These are, of course, encouraging for advocates of competition and choice. Yet this study also extends a recent (and, to us, unwelcome) trend that finds negative effects for voucher participants in large statewide programs. While earlier evaluations of privately and publicly funded scholarship programs—usually administered at the city level—found neutral-to-positive impacts on participants, newer studies of Louisiana’s and Indiana’s statewide programs have uncovered negative results, particularly in math.

There’s been much discussion about what might be behind these participant results. Is too much regulation discouraging high-quality private schools from joining the program? Are state exams failing to capture important private school contributions to student success? Do large, statewide programs lack the tools and resources to ensure quality at scale? Or are private schools simply struggling to raise achievement—especially in math—in relation to their public school counterparts? Some or all of these (or other) factors may be at work, but no one really knows for certain. More research on the effects of statewide voucher programs is obviously warranted.

Even though we don’t have all the answers, we believe that thoughtful policy makers can draw from the extant research as well as on-the-ground experience to give these programs the best chance of succeeding for more students, whether attending public or private schools. The pertinent lessons seem to us applicable both in states considering new private school choice programs and in states (like Ohio) that are seeking to improve an existing program.

First, we need to foster a healthy, competitive environment in K–12 education. A competitive jolt can awaken sleepy, lazy, or slipshod schools to clean up their act and attend more closely to the academic needs of their students. On the policy side, this means that lawmakers should continue to encourage a rich supply of school options, including not just private schools (in their many flavors, including religious and non-sectarian) but also public charter, STEM, and career and technical schools. At the same time, families can do their part by demanding more quality school choices. Competition and choice—two sides of the same coin—can incentivize all schools to work harder at meeting the needs of their pupils.

Second, policy makers should resist calls to pile more input-based regulations upon voucher-accepting private schools. Ohio’s private schools already face heavier regulation than those in many states. For example, they must adhere to state operating standards and hire state-licensed or certified teachers. Most of this was true before EdChoice came along (which makes less likely the “overregulation” explanation for disappointing participant results, at least in Ohio). Policy makers should tread lightly when adding to schools’ regulatory burdens: After all, freedom from regulation is precisely what makes private schools different and—for many—worth attending in the first place.

Third, as this study suggests, private schools likely vary when it comes to quality, and the public needs maximum transparency about this. Accordingly, state leaders should help families better understand the quality of their options by providing easy-to-compare information on the performance of voucher-accepting private schools. While Ohio already reports voucher students’ proficiency rates at the school level (subject to FERPA limitations), we know that those results are likely to be conflated with non-schooling factors like family income. They are also hard to track down. To be fair to private schools that educate disadvantaged voucher pupils, we suggest the adoption of a value-added measure—a school quality indicator that is more poverty-neutral than conventional academic proficiency rates. States (including Ohio) should make sure that these academic outcomes for voucher-accepting private schools are easily accessible to parents, perhaps in a report card-like format akin to those adopted for public schools. In Ohio, this would not add any additional testing or regulatory requirements on private schools.

Fourth, policy makers should craft simple, parent-friendly program rules. From the perspective of families, EdChoice is fairly complex, which may have influenced who participates in it. Eligibility hinges on public schools’ annual ratings from the state—which can change from year to year—and the state has no obligation to notify parents of their children’s eligibility. This means that families must bestir themselves to visit the state’s website or seek eligibility information through other channels. To ensure awareness, states should require direct notification of eligibility from the state department of education or a competent nonprofit agency. (This should also happen when eligibility is based on income.) Making matters more complicated, current EdChoice application rules require eligible students first to gain admission to a private school; then the school applies to the state for a voucher. It would be far simpler for parents if they could apply directly to the state for a voucher and then shop for the right private school. This process would not only empower parents but also give policy makers a much clearer picture of the demand for vouchers.

The present report breaks important new ground, but it is by no means the final word on EdChoice. We still have much to learn, including whether vouchers impact non-testing outcomes such as post-secondary success. We also need a deeper understanding about the quality of individual private schools. But the information set forth in the pages that follow is critically important as thoughtful policy makers consider the design and implementation of voucher programs, both in Ohio and across the nation. Programs that aim to better the lives of children must face scrutiny from independent, credible evaluators. Even when its findings are unexpected and painful, rigorous, disinterested evaluation remains the best way to prod improvements and make progress toward the program’s goals. In the case of EdChoice, the program appears to have met one of the two objectives conceived by its founders: Competition has spurred some public school improvement. The challenge ahead is to forge a stronger EdChoice program, one that can lead to widespread academic improvements for children who take their scholarships to the state’s private schools.

No Child Left Behind (NCLB) required states to identify and intervene in persistently low-performing schools. Some states opted for more aggressive intervention with the creation of recovery school districts, including the Achievement School District in Tennessee, the Recovery School District in Louisiana, and the Education Achievement Authority in Michigan. Here in the Buckeye State, we don’t have a statewide recovery district—but we do have “academic distress commissions” (ADCs).

ADCs were added to Ohio state law in 2005 as a way for the state to intervene in districts that consistently fail to meet academic standards. Only two districts (Youngstown and Lorain) have ever been placed under ADC control, while a third (Cleveland) avoided the designation because of its implementation of the Cleveland Plan. In the summer of 2015, however, ADCs blasted onto the front pages of Ohio newspapers thanks to House Bill 70. The bill—widely known as the “Youngstown Plan” [1]—sharpened the powers and duties of ADCs in Ohio and was signed into law by Governor Kasich in July. (See here for an overview of the bill’s biggest changes to ADCs.) 

In December 2015, the long-awaited reauthorization of NCLB became a reality with the Every Student Succeeds Act (ESSA). The biggest change was the devolution of some authority from the federal government to states—including much greater discretion over how to identify persistently failing schools and what to do about them.

A close read of the new law suggests that it won’t conflict with Ohio’s current ADC structure. ESSA requires the identification of individual schools, while an ADC identifies an entire low-performing district. As long as the Buckeye State follows ESSA’s requirements, it shouldn’t trouble the feds that Ohio also intervenes in struggling districts—particularly since ESSA permits “additional statewide categories of schools” that are identified “at the discretion of the state.”

But as policy makers work to develop Ohio’s state accountability plan under ESSA, it’s worth asking whether the current system of ADCs can provide some helpful lessons and a head start on designing the school-based identification and intervention guidelines required under the new federal law. Here’s a look at a few of the key elements:

School identification

Ohio statute requires that any district receiving an overall grade of F for three consecutive years be placed under the control of an ADC. This is a clear measure that is based on Ohio’s accountability report card data, which the state has been using for some time. ESSA, meanwhile, requires that struggling schools be given a designation of “comprehensive support” (any Title I school that’s in the bottom 5 percent statewide or fails to graduate 67 percent of its students) and “targeted support” (any Title I school with a subgroup that is labeled “consistently underperforming” by the state). Although the comprehensive support measures are clearly defined, Ohio is well positioned to use its robust school report card measures—already utilized in the context of ADCs—to develop the required “consistently underperforming” definition for targeted support. Ohio’s accountability system will have to undergo some relatively minor changes under ESSA, but these changes won’t require the state to stop using report cards and letter grades.

Locally made plans

ADCs are required to get local input in the form of a CEO-convened community stakeholder group,[2] which is tasked with “developing expectations for academic improvement” and “building relationships with organizations in the community that can provide services to students.” Under ESSA, both comprehensive support and targeted support schools will also be subject to improvement plans. What these plans must contain matters, but how they’re developed is just as important—and schools in both categories are required to seek stakeholder input.

To be fair, ADCs and local control haven't exactly gotten along so far. The rapid proposal and passage of House Bill 70 was controversial and drew condemnation from the very stakeholders that ESSA champions—school leaders, teachers, and parents. This should be a warning to Ohio as it devises its ESSA-required intervention strategies: If policy makers want the system to work, they would be wise to earnestly seek local input on how to craft plans for comprehensive support and targeted support schools. 

Creating school choice

Like NCLB before it, ESSA permits districts to allow students who are enrolled in persistently failing schools the option of transferring to another public school served by the district. Ohio’s ADC legislation has similar provisions that the state could consider mirroring for ESSA purposes. First, any student enrolled in an ADC-designated district is eligible to participate in the EdChoice Scholarship Program, Ohio’s largest voucher program. Second, the ADC is responsible for expanding “high-quality school choice options in the district” and can do so by creating a high-quality school accelerator—an organization that is not operated by the district and is responsible for attracting and recruiting high-quality sponsors and schools. The accelerator model could further empower families at comprehensive support and targeted support schools while interventions are underway. 

Exit Criteria

ESSA requires a school identified for intervention to meet a set of high expectations in order to “exit” identification and its associated interventions. The law permits states to craft their own exit criteria as long as schools that fail to meet that criteria within a certain number of years are subjected to “more rigorous state-determined action.” USDOE’s proposed regulations contain some additional stipulations, including that schools must meet exit criteria within four years.

Ohio has already developed similar exit criteria for its ADCs. First, a district must earn an overall grade of C on the state report card. Once that benchmark is met, the district begins a transition period. If it maintains an overall grade higher than F for two consecutive years after the first C, it ceases to be under ADC control. If, however, the district receives an F during the transition period, it reverts back to its ADC designation.

ADC statute also requires rigorous action long before USDOE’s fourth year. Starting in the first year of the existence of an ADC, if a district doesn’t earn an overall C grade, it is subject to an increasingly severe ladder of interventions—which includes reconstituting any school, altering or suspending collective bargaining agreements, and appointing a new board of education. In theory, Ohio could turn some of these criteria and consequences into its exit standards for ESSA-identified schools; however, the backlash against the Youngstown plan could make that move difficult.

***

ESSA’s focus on school identification and an ADC’s attention to district performance could lead some to assume that the two laws aren’t compatible. But they’re far from being mutually exclusive. In fact, the dual focus on schools and districts could work in Ohio’s favor, since ESSA requires states to “provide technical assistance” and support to each district that serves “a significant number” of comprehensive support and targeted support schools. It stands to reason that any district with a significant number of these schools is already or soon to be a candidate for ADC designation, so it makes sense that the state would oversee interventions at the district level while the district oversees interventions at the school level.

Furthermore, Ohio’s experience in developing and working with ADCs gives it a distinct advantage in considering how to design school-level interventions. Ohio’s ADC legislation has plenty of similarities to ESSA, and these similarities should contribute to the state’s development of a strong identification and intervention plan for struggling schools.

Like Mom and apple pie, everyone loves and believes in a well-rounded education. Ensuring that every child gets one, however, has proven to be a challenge of Herculean magnitude—despite compelling evidence that it’s precisely what disadvantaged students most desperately need to close persistent achievement gaps and compete academically with their more fortunate peers. Enter the Every Student Succeeds Act. As this report from Scott D. Jones and Emily Workman of the Education Commission of the States (ECS) notes, while concerns about providing children a well-rounded education “have not received the same degree of attention as hot-button issues like equitable funding and accountability indicators, it could be considered a foundational element of the new federal law.”

Foundational, perhaps. But is it enforceable? Education Secretary John King has lately been using the bully pulpit to promote the virtues of a well-rounded education. “States now have the opportunity to broaden their definition of educational excellence, to include providing students strong learning experiences in science, social studies, world languages, and the arts,” King is quoted as saying by the ECS authors. “That’s a huge and welcome change.”

Yes and no. In truth, states have always had the “opportunity” to broaden their definition of educational excellence. The question is why, in the main, they haven’t done so. On the one hand, a case can be made that federal education policy has discouraged states from providing a well-rounded education by tacitly promoting a too-narrow view of reading (I have argued elsewhere that this is precisely what happened under NCLB). Accountability policies that demand fast and measurable gains in reading functionally privilege a skills-and-strategies approach to reading instruction. This discourages the kind of steady investments in knowledge and vocabulary that build mature reading comprehension, which is a slow-growing plant. Merely encouraging a well-rounded education is insufficient. If states don’t use their “opportunity” under ESSA to actively and aggressively incentivize the delivery of a well-rounded education, the phrase will remain a mere platitude.

Jones and Workman offer examples of how ESSA might improve policy and practice. For starters we have the law’s expanded definition of a well-rounded education, which now includes writing, engineering, music, technology, and career and technical education. There’s Title I, which requires that all districts provide a “well-rounded program of instruction that meets the needs of all students,” and Title II, which allows funds to be used to help teachers “integrate comprehensive literacy instruction into a well-rounded education.” And there are “flexible block grants” through which ESSA “creates some accountability around incentives for providing a well-rounded education…particularly for minority groups, including women, English language learners, students with disabilities, and low-income students.” By not limiting states to specific areas in which to apply for funding, local education agencies “are free to emphasize any of the multiple subjects listed in ESSA, select their own, or integrate across subjects,” the authors note: “The possibilities are endless in how states can utilize this [block grant] program to make meaningful investments in their students.”

“With ESSA, districts are asked to conduct a comprehensive needs assessment to identify the needs of their unique populations and to make investments to address those needs,” the authors note. Let me offer districts a leg up on that needs assessment: Every child needs a well-rounded education. And you don’t start that after children learn how to read. You build readers by providing it from the very first days of school. The next Massachusetts will be the state that best understands this truth, adopts curriculum that delivers it, trains teachers to implement it, and uses their newfound flexibility to ensure that kids benefit from it.

Or we can just keep talking about it.

SOURCE: Emily Workman and Scott D. Jones, “ESSA’s Well-Rounded Education,” Education Commission of the States (June 2016).

I remember the exact moment I became a charter school supporter. It was 2006, and I was a few days away from completing my first year of teaching in Camden, New Jersey. The mother of one of my students wanted to speak with me after school. I’ll never forget what she asked me: She wanted to know if she should send her daughter to a nearby charter school for first grade or keep her in our district school. Specifically, she asked, “What would you do if you were me—if this were your child?”

If someone had asked me then my opinion on charter schools, or choice generally, I wouldn’t have had one. But I did have a strong opinion about wanting her child (small for her age, with a tough exterior that could be mistaken for anger if you didn’t know her well) to thrive. The charter up the street was the only one I’d ever heard of, even though the city suffered from a desperate shortage of schools where reading and math proficiency scores weren’t in the single digits. I knew a bit about that particular school. It was safe and orderly, placed high expectations on students, offered an extended school day and school year, and provided opportunities that our school didn’t. So I said yes, unequivocally. Standing there with a young mom around my age—a single mom, living in a neighborhood notorious for poverty and crime but unwilling to let that define her daughter’s story—there really wasn’t much to deliberate about.

One of the primary problems we face in education policy making is our inability, or unwillingness, to see through the eyes of moms, dads, and students in search of better options. We’re reluctant to let go of tired talking points and simply ask, “What would I want for my own child?”

Every student deserves a school like the Dayton Early College Academy (DECA), a charter high school helping students defy the odds in one of Ohio’s lowest-performing districts. Three out of four of its students come from economically disadvantaged families, but 100 percent enroll in college. These students include Khadidja, who is attending West Virginia University this fall, and Shyanne, who appears in the video below and elucidates how DECA’s positive culture and high expectations have inspired her pathway to success.

Shyanne’s Story (Dayton Early College Academy) from Good Charters, Good Choices on Vimeo.

Whatever your opinions about school choice or charter schools up to this point, we urge you to watch this video and listen to what Shyanne has to say. Better yet, go visit a high-quality charter school in your community and talk to teachers, parents, and students directly. Good charter schools like DECA are good choices. They play a critical role in getting more low-income students to and through college, challenge the notion that socioeconomics is destiny, and empower parents who want their kids to have the best shot in life. That’s something all of us have in common. 

Eighteen months ago, Ohio proved it was finally serious about cleaning up its charter sector, with Governor Kasich and the Ohio General Assembly placing sponsors (a.k.a. authorizers) at the center of a massive charter law overhaul. The effort aimed to hold Ohio’s sixty-plus authorizers more accountable—a strategy based on incentives to spur behavioral change among the gatekeepers of charter school quality. Poorly performing sponsors would be penalized, putting a stop to the fly-by-night, ill-vetted schools that gave a huge black eye to the sector and harmed students. Under House Bill 2, high-performing sponsors would be rewarded, which would encourage authorizing best practices and improve the likelihood of greater quality control during all phases of a charter’s life cycle (start-up, renewal, closure).

While the conceptual framework for these sponsor-centric reforms is now toddler-aged, the actual reforms are still in their infancy. (House Bill 2 went into effect in February of this year, and the earlier enacted but only recently implemented sponsor evaluation is just now getting off the ground.) Even so, just five months in, HB 2 and the comprehensive sponsor evaluation system are having an impact. Eleven schools were not renewed by their sponsors, presumably for poor performance, and twenty more are slated to close. Not only are academically low-performing schools prohibited under law from hopping to a new sponsor,[i] but it also appears that Ohio’s sponsors are behaving more cautiously in general—making it less likely that they will take on even mediocre performers (not just those for whom such prohibitions apply). Even more important, if the data in Graph 1 is any indication, sponsors seem to be applying caution to new school applicants as well—meaning that it’s far less likely that half-baked schools will open in the first place. Given Ohio’s recent track record of mid-year charter closures, this is a major victory, though a dramatically slowed opening rate sustained over time may indicate looming issues for the sector.

For the 2016–17 school year, the Ohio Department of Education lists just ten “potential” new charter schools, an all-time low in the number of charters opened in a given year. At least one is a replication of an existing top-notch school (Breakthrough’s Village Preparatory model). As Graph 1 depicts, this is significantly fewer compared to past years.

Graph 1: Charters opened each year, 1999–2016

[[{"fid":"116558","view_mode":"default","fields":{"format":"default"},"type":"media","link_text":null,"attributes":{"height":"349","width":"1021","style":"width: 600px; height: 205px;","class":"media-element file-default"}}]]

Source: Data come from the enrollment history listed in the Ohio Department of Education’s annual community schools report  (2015).[ii]

In the days leading up to HB 2’s passage last October, we urged Ohio to expect more from charter school sponsors during the new school application process: “Closing poor performers when they have no legs left to stand on is only a small part of effective oversight. A far better alternative would be to prevent them from opening in the first place….If Ohio charter schools are going to improve, better decision making about new school applicants will be critical.”

Ohio’s charter reforms appear to have influenced sponsor behavior in opening new schools. There are likely other reasons contributing to the overall slowdown—saturation in the academically challenged communities to which start-ups are restricted, for example, and a lack of start-up capital—but new, higher sponsor expectations and a rigorous sponsor evaluation system undoubtedly have played a sizeable role.

Sponsors await the first round of performance ratings this fall, with one-third of their overall scores tied to the performance of the schools in their portfolios. Fordham remains concerned that the academic component leans too heavily on performance indicators strongly correlated to demographics rather than student growth, making it difficult if not impossible for even good sponsors to earn high marks in this category. At minimum, however, the rigor of the evaluation is spurring long-overdue changes to the new school vetting process and making all sponsors think long and hard before handing out charter contracts.

For the moment, slowing new charter growth is a good thing. The state has just started down the path of serious charter school reform, and the sector, which was too lax for too long, is in need of balance. Like any pendulum, however, it’s possible that Ohio will move itself too far in the other direction—if, for instance, the opening rate continues to stall or even stops entirely. If even the state’s best sponsors can’t open new schools, the sponsor accountability framework will need revision so that families and students are not deprived of high-quality choices. Stagnant growth could also occur if Ohio charters continue to be starved of vital start-up funds. It is telling—and worrisome—that just one of Ohio’s proposed new charter schools is a replication of an existing high-quality network. It’s also troublesome that Ohio’s $71 million in federal Charter School Program funds—a program responsible for helping the state’s best charter networks start and expand—remains on hold.

Now that the state has accomplished the tough task of revising its charter law, Ohio needs to consider ways to make it easier for the best charters to expand and serve more students. Leaders of Ohio’s best charter schools point to inequitable funding, lack of facilities, and human capital challenges as serious impediments to growth. Lawmakers should explore ways to fast-track the replication of Ohio’s best charters while keeping an eye more broadly on preserving the autonomy of the sector, rewarding innovation and risk-taking, and resisting efforts to over-regulate. If we are serious about lifting outcomes for Ohio’s underserved children and preventing the sector from too closely mimicking traditional public schools, these issues demand our attention. For the time being, it’s worth celebrating early milestones indicating that Ohio is on its way to serious improvement.

On June 22, the Dropout Prevention and Recovery Study Committee met for its first of three meetings this summer. The committee is composed of two Ohio lawmakers (Representative Andrew Brenner and Senator Peggy Lehner) and several community leaders. It was created under a provision in House Bill 2 (Ohio’s charter reform bill) and is tasked with defining school quality and examining competency-based funding for dropout-recovery schools by August 1.  

Conducting a rigorous review of state policies on the state’s ninety-four dropout-recovery charter schools is exactly the right thing to do—not only as a legal requirement, but also because these schools now educate roughly sixteen thousand adolescents. The discussion around academic quality is of particular importance. These schools have proven difficult to judge because of the students they serve: young adults who have dropped out or are at risk of doing so. By definition, these kids have experienced academic failure already. So what is fair to expect of their second-chance schools?

Let’s review the status of state accountability for dropout-recovery schools and take a closer look at the results from the 2014–15 report cards. In 2012–13, Ohio began to provide data on the success of its dropout-recovery schools on an alternative school report card—a rating system that differs from that of traditional public schools. Dropout-recovery schools, for example, do not receive ratings for the conventional Performance Index or Value Added measures; neither are they assigned ratings on an A–F scale like other Ohio public schools. In 2014–15, the state began rating these schools as “Exceeds Standards,” “Meets Standards,” or “Does Not Meet Standards.”

The overall grades are calculated by following a two-step process outlined in Tables 1 and 2. First, points are assigned to schools based on the four individual report card components: graduation rate (a composite of the four-, five-, six-, seven-, and eight-year adjusted cohort rates); the twelfth-grade assessment passage rates on all of the Ohio Graduation Tests or the end-of-course assessments once they are phased in; gap closing (a.k.a. Annual Measurable Objectives); and student progress. Each school receives a rating of Exceeds, Meets, or Does Not Meet for each component; points are awarded based on that designation.

As you can see from Table 1, the graduation rate and progress components are weighted somewhat more heavily than other two (30 percent versus 20 percent). Once all component points are added and divided by possible points earned, Table 2 can be referenced to determine which rating a school receives. When it came to overall school ratings for 2014–15, 43 percent of dropout-recovery schools received a Does Not Meet rating, 49 percent received a Meets rating, and 8 percent received an Exceeds rating (a total of ninety-three schools were part of this accountability system).  

Table 1. Points assigned to DOPR schools based on component ratings

Table 2. Overall ratings assigned, based on the summation of points earned per category

The graduation rate, assessment passage, and gap-closing measures generally mirror traditional school report cards (though with some modifications). But one of the more interesting aspects of the dropout-recovery report cards is their progress measure (i.e., student growth over time). To measure student growth, dropout-recovery schools use NWEA’s Measures of Academic Progress (MAP) test rather than calculating gains on state exams. (State law specifies the use of a “nationally norm-referenced” assessment to measure progress in dropout-recovery schools. [1]) To gauge progress, dropout-recovery students must have taken both the fall and spring administrations of the MAP test. The differences in achievement from fall to spring are compared to a norm-referenced group (provided by the vendor) to generate an indicator of a student’s academic progress. Consistent with Ohio’s value-added measure, progress is evaluated based on whether a student maintains his relative position from one test administration to the next. In general, a student scoring at, say, the fiftieth percentile in one testing period would be expected to remain at that percentile in the next one. Achievement above or below expected growth classifies a student as exceeding or failing to meet expectations.[2]

While the norm-referenced approach makes sense in the dropout-recovery context, the results from the first year of implementation appear oddly distributed to us. Consider Chart 1, which displays the number of dropout-recovery schools and their progress ratings. While fifty-nine schools did not meet the progress standards, just one—one!—school exceeded the growth standards. (Another thirty-three schools met the growth standards.) It should be noted that students failing to take both administrations of the MAP test are not counted in a school’s overall progress measure, so the disappointing results probably cannot be explained by either mobile or chronically absent students who didn’t take one of the MAP exams.

Chart 1: Progress ratings for Ohio’s dropout-recovery charter schools, 2014–15

Unlike the chart above, traditional district schools’ value-added scores demonstrate a wide range of scores and ratings (see here, for instance). However, because kids in dropout-recovery schools have experienced previous academic difficulties, we might expect their progress to lag behind that of their peers. Still, it seems strange to observe virtually no schools exceeding the growth expectations. It is worth investigating whether these results would be observed using a different value-added measure (such as the one used by districts, which compares a student’s performance to his previous test scores instead of a national norm). State policy makers should also confirm that the comparison group used by NWEA is appropriate for Ohio’s dropout-recovery students.

Further, confirming that a large enough sample size was used to generate these results is imperative. But it’s hard to identify the cause of this odd distribution. It could be related to methodology; or perhaps the average student in most dropout-recovery schools really is failing to achieve adequate progress. This question—along with a host of others surrounding dropout-recovery and prevention school attendance, funding, and “success”—should be addressed by the Dropout Prevention and Recovery Study Committee. To be sure, this is just one year of data, and a first using this type of growth measure. But policy makers should definitely look into these issues and keep a close eye on the 2015–16 progress results to see if the pattern recurs.

The dropout-recovery and prevention report cards provide essential accountability for Ohio’s alternative schools and offer much-needed information to parents, educators, and the public. The nature of dropout-recovery programs as second-chance schools begs the question of whether their success and progress should be measured with a different stick, but adjustments in report card composition and its measures account for the unique population they serve. Now we must confirm that the measures are useful and accurate as we seek to hold schools accountable for the job they were created to do: educating and graduating students who have fallen behind. Here’s hoping the Dropout Prevention and Recovery Study Committee prioritizes the accurate identification of dropout-recovery schools that are making a positive impact, confirming that the accountability system is working as it should.