GOP presidential candidate Donald Trump recently visited Cleveland Arts and Social Sciences Academy, a charter school educating predominately minority and low-income children. I write not to comment on Mr. Trump’s candidacy, his thoughts on education policy, or even Ohio’s charter schools. Rather, this is my takeaway from the whole brouhaha—and be forewarned, it’s a wonky one: Ohio needs to return to a multi-year value-added measure.
Here’s why. Charter critics, media, and even a respected education reform group were quick to label Cleveland Arts and Social Sciences Academy a “failure.” They relied on the school’s 2014–15 school report cards, which indeed showed low A–F grades. One glaring rating was the school’s F on Ohio’s value-added measure—not good at face value, because the measure is generally uncorrelated with student demographics and is therefore a metric that high-poverty schools can and do succeed on. (Value added gauges growth over time, regardless of students’ prior achievement.)
Keep in mind, however, that Ohio is presently basing value-added rating on one year of data—and those ratings can swing quite dramatically from year to year. Consider, for example, that Toledo Public Schools received an A rating on value added in 2013-14. But in 2014-15, the district received an F. Did the district’s performance suddenly collapse? Probably not. The same phenomenon has happened to other schools and districts in Ohio; in fact, we once called this the “yo-yo effect.”
This means that when it comes to value-added ratings, we need to take stock of multiple years of data. One year doesn’t tell the entire story. When we consider the longer track record of Cleveland Arts and Social Sciences Academy in the table below, we actually see that it has performed pretty well on the state’s value-added measure, save for 2014–15 and 2010–11.
[[{"fid":"117134","view_mode":"default","fields":{"format":"default"},"type":"media","attributes":{"style":"width: 700px; height: 109px;","class":"media-element file-default"},"link_text":null}]]
Notes: Prior to 2012–13, Ohio rated schools on a three-tier system for value-added: Above, Met, or Below. Starting in 2012–13, the state moved to an A–F rating system. In 2013–14, the state rated schools on value-added based on a three-year rolling average (if the school had three years of data). With the transition in state exams, it discontinued the multi-year approach and rated schools on one-year of data in 2014–15. Ohio will again rate schools on one year in 2015–16 due to another transition in testing.
As I’ve argued before, because value-added scores—and their ratings—tend to fluctuate from year to year, we need to smooth out the volatility to gain a clearer understanding of school performance. It’s similar to how economists make “seasonal adjustments” to annual employment data due to, e.g., spikes in retail jobs around the holidays. The state should do the same with value added so as not to mislead the public into believing that a school is a massive failure (or resounding success) based on just one year of data.
At the end of the day, Cleveland Arts and Social Sciences Academy may indeed have experienced a “bad year” in 2014–15. To be sure, that rating should not be excused, and there is some reason for concern. But the preponderance of value-added data indicate that the school has been modestly successful over the longer haul, and certainly not an organizational failure in my view. So let’s call this a lesson learned: When it comes to value-added ratings, base them on multiple years of data.
