Almost three years ago, Fordham and the Northwest Evaluation Association published a landmark study, The Proficiency Illusion, which found that state “proficiency cut scores” varied tremendously, not just from state to state but also within states. Cut scores for elementary school kids were lower than for middle school kids; cut scores for math were higher than for reading; and “cut scores” tended to drift downward over time.
This created an incredibly patchy and misleading portrait of student achievement across the land. In the report’s foreword, Checker Finn and I wrote:
What does this mean for educational policy and practice? What does it mean for standards-based reform in general and NCLB in particular? It means big trouble—and those who care about strengthening U.S. K-12 education should be furious. There’s all this testing—too much, surely—yet the testing enterprise is unbelievably slipshod. It’s not just that results vary, but that they vary almost randomly, erratically, from place to place and grade to grade and year to year in ways that have little or nothing to do with true differences in pupil achievement. America is awash in achievement “data,” yet the truth about our educational performance is far from transparent and trustworthy. It may be smoke and mirrors. Gains (and slippages) may be illusory. Comparisons may be misleading. Apparent problems may be nonexistent or, at least, misstated. The testing infrastructure on which so many school reform efforts rest, and in which so much confidence has been vested, is unreliable—at best.
That study didn’t examine the Empire State, but now we know that the “proficiency illusion” was in full bloom there, too. What’s notable is that Commissioner David Steiner is acknowledging the problem and taking action to fix it. Reportedly, he told the state Board of Regents last week that “The word ‘proficient’ should tell you something, and right now that is not the case on our state tests.”
So he and his board adopted the new Common Core standards (and were rewarded when New York became an RTT finalist) and will presumably shift over one day to new assessments aligned with those standards. Meanwhile, they raised cut scores on the state’s current tests in order to ensure that a student deemed “proficient” is actually on track to pass the Regents’ exams in high school—and to succeed thereafter. Though commonsensical, this move was also courageous, for it immediately illustrated just how far many kids and schools and districts have to go to reach that real-world standard.
As everyone knows, if you suddenly raise the bar, lots more kids will fail to clear it. So demonstrated the results released yesterday, which showed big declines in the number of young New Yorkers passing the state test. This has implications on many fronts, but perhaps most visibly in Gotham. As I told the New York Times, these revelations complicate the Bloomberg administration’s message about the success of their reforms. Complicate, but do not contradict. The mayor and Chancellor Klein don’t control the state test and deserve no blame for it having become such an unreliable indicator of achievement. Still, the sky-high proficiency rates that they’ve celebrated in recent years have turned out to be, well, illusory in their city, as well as across the Empire State.
That doesn’t mean the Big Apple schools aren’t making progress. Gotham Schools quoted Regents chancellor Merryl Tisch thusly: “If you haven’t noticed that the city school system is improving, then you’re walking around with blinders.” She’s right. District leaders can legitimately state that, regardless of where the proficiency bar is set, the city’s kids are making respectable gains. Its average “scale scores” in math and reading have risen significantly over the past five years, both on average and for individual ethnic groups. What’s more, the latest results from the National Assessment of Educational Progress show significant improvement from 2007 to 2009 in fourth-grade reading.
As in most states, New York experienced a phony achievement bubble. Now there’s been a correction, and we can start fresh from a more honest and accurate position. This is good news, and another illustration of how badly the country needs standards and assessments it can believe in.
This piece first appeared on Fordham’s blog, Flypaper. Subscribe to Flypaper’s RSS feed here.
