As Ohio’s annual report cards are released this week, Fordham is gearing up to dive into the data and explore what it means about K-12 public education in the Buckeye State.
We won’t be alone; reporters, bloggers, and education advocates will all offer their own hot takes, many of which will examine charter school performance data. If the past is any indication, some headlines and stories will be patently unfair.
Sadly, much of this will be intentional. Charter foes have historically used the report card release as an opportunity to denigrate the sector, lumping even the very best schools together with perennial low performers and those seeking to evade accountability.
In other instances, apples-to-bananas comparisons may be inadvertent. Reporters are expected to consume a massive amount of information and quickly produce insightful stories about educational performance trends or accomplishments of schools in their region. Still other reporters may be new to the education beat, or may rely on the analyses published by said charter critics without having the time or experience to subject them to scrutiny.
Let’s discuss what a fair comparison looks like and what to watch out for.
Attempting to make apples-to-apples comparisons
Fordham has typically evaluated charter school performance against Ohio’s Big 8 districts (Akron, Canton, Cincinnati, Cleveland, Columbus, Dayton, Toledo, and Youngstown). This is because the bulk of Ohio’s charters are located in those cities, serving similar, and similarly disadvantaged, students. Rather than compare charters to district schools across the state—including those in middle-class and wealthy communities—we look at Ohio’s urban charters versus urban district schools, a more apples-to-apples comparison. Charter schools serving high rates of socioeconomically disadvantaged students and students of color should be compared with traditional public schools serving roughly the same demographics.[1]
Now, some folks argue that because Ohio charters can and in many instances do draw students from anywhere in the state, we should compare them to each and every district from which students flow. In their minds, a Columbus charter school with two students from Worthington Public Schools (a high-wealth community just north of the city) should also be compared to Worthington, despite the fact that Columbus serves a student population that is very economically disadvantaged, while Worthington’s is not. Their reasoning is that because Worthington has “lost” money to the Columbus charter—an argument that in and of itself is very problematic—it’s fair to compare the two.
This is preposterous. Given what we know about the long-standing statistical correlation between socioeconomic status and student achievement on proficiency exams, it should be no surprise that wealthier districts and schools, as well as statewide averages that include these schools’ scores, out-perform charters serving primarily poor students. Such statistics should not be used as a battering ram against the charter sector.
Furthermore, schools that score C or D on Ohio’s Performance Index may at first glance appear to be low performing, but if their value-added results (student growth) are an A or a B, this indicates that students are making steady gains. Keep that in mind when evaluating high-poverty schools in either the charter or district sector.
Ohio has a significant number of specialty charter schools
As if finding the right comparison for a traditional brick-and-mortar charter school wasn’t hard enough, it’s also important to know that Ohio’s charter sector includes a variety of “specialty” schools. For example, there are dozens of schools that serve high percentages of students with disabilities such as autism, or those whose special education rates veer towards 80 or 90 percent. These schools are currently exempt from Ohio’s automatic closure law, and great care should be taken when comparing student-level performance in these schools with that of other schools with vastly different numbers of students with special needs. Similarly, there are over 80 dropout prevention and recovery charter schools, which have their own accountability system and performance benchmarks. This alternative accountability system is a good place to start when looking at performance in these schools, and comparisons can be made between dropout recovery schools operated as charter schools and those run by school districts.
Finally, you’ve probably heard that Ohio has a significant number of e-schools. The student population at these schools tends to be whiter and less socioeconomically disadvantaged than Ohio’s brick-and-mortar charters, in the aggregate at least. Students also come from all over the state. The comparison group would tend to be more akin to the statewide average; however, many students at e-schools enroll for just a brief period of time, again making fair comparisons challenging.
Overall, Ohio’s specialty charter schools—special needs focused, dropout recovery, or online—serve tens of thousands of students in well over one hundred schools. So while it’s essential to analyze whether all schools are serving students well, we’d do well to take the extra time to find the appropriate comparison group.
District averages obscure what’s happening at a school or neighborhood level
Any analysis that compares individual schools to entire school districts is also problematic. Comparing a charter school with a high percentage of students in poverty to a mixed-income district—whose lower-poverty schools pull up the overall performance average—will probably yield a scenario wherein a district scores higher. Columbus City Schools, which has a number of schools serving higher-income families and therefore scoring higher on proficiency measures, should not be compared to a charter or district school in the poorest neighborhood in Columbus serving a 100 percent disadvantaged population. While comparing individual schools against district or state averages is fine as a benchmarking exercise, we should focus performance comparisons at a school level.
***
Ohio’s annual report card release provides a vital opportunity to see how our schools and districts are serving Ohio students. The data it provides enable us to examine trends over time, target support and intervention to struggling schools, and recognize and learn from outstanding schools. Yet when it comes to gauging the performance of Ohio’s 360+ charter schools, comparisons to their district counterparts should be careful and intellectually honest. Better yet, education advocates on all sides might be better served in celebrating, supporting, and learning from any Ohio school—of any type—that is delivering on the promise of providing high-quality educational opportunities to students who are historically denied them.
[1] Even this is imperfect. A more robust and accurate analysis would examine student-level data, like the CREDO studies out of Stanford University.
