For advocates of evidence-based urban education policy, a recent New York Times profile of New York City Schools’ Chancellor Carmen Fariña should offer serious cause for concern. That Fariña has worked to dismantle several of the promising Bloomberg-era education reforms is not the main offending issue. (The former is unfortunate, but hardly unexpected from the current administration.) As Robert Pondiscio has previously pointed out in this space, far more worrisome is Fariña’s apparent view of the proper role of research in education policy—one seemingly rooted in the bad old days when high-quality empirical research was dismissed or ignored.
Chancellor Fariña plainly nurtures none of the previous administration’s fondness for data, preferring a more “holistic” approach. Nor, for that matter, does she even require test scores to know which schools are performing well. The chancellor, perhaps with Spidey-sense, knows a good school when she sees it.
To be fair, I’m open to the claim that perhaps some of the Bloomberg reforms were too technocratic. And no one could have reasonably expected Chancellor Fariña to be an empirical data junkie. But her recent statements reveal a remarkable disdain for science’s role in formulating education policy. The following New York Times passage is particularly startling:
Asked recently, for example, about a multiyear study showing that students who attended the small high schools started by the Bloomberg administration were more likely to graduate and attend college than their peers at larger schools, [Chancellor Fariña] dismissed its conclusions. “It’s one view of things,” she said. “There are many views about everything.”
As the chancellor well knows, the reporter was referring to a 2014 study by Howard Bloom and Rebecca Uterman of MDRC (a nonprofit research group) that was published in the Journal of Policy Analysis and Management—a distinguished, peer-reviewed public policy and administration journal.
Taking advantage of certain students’ random assignment to small NYC schools, Bloom and Uterman adopted a randomized field trial (RFT), widely considered the “gold standard” of social science research methodologies, to measure educational outcomes based on school size (small or regular).
RFTs, it is true, have their own limitations (beyond the scope of this post to describe). Nevertheless, Bloom and Unterman’s findings raised eyebrows: Students randomly assigned to small high schools were, on average, 9.5 percentage points (!) more likely to graduate from high school than had they attended larger, traditional public high schools. Their result, robust to a variety of statistical specifications, is, quite literally, a life-changing outcome for the students involved. Indeed, few other modern education reforms have rivaled such empirically observed impact.
To the best of my knowledge, there has been no serious critique of Bloom and Unterman’s methods or findings (at least as they relate to the impact of the policy on students attending small schools). Nor are there competing studies of similar methodical rigor purporting to find contradictory results. When it comes to the impact of small schools on students, the study is on the cutting edge of knowledge.
For what it’s worth, I am no small schools evangelical. Though supportive of the trial as a valid experiment, I had no particular expectation of the outcome. Further, plenty more remains to be learned about small schools; no single analysis can or should settle a major policy issue. Yet to treat a randomized field trial, with large, robust findings, conducted by highly respected researchers writing in a prestigious journal, as merely “one view” among “many” suggests an alarming disregard for empirical social science—or at least impartial research at odds with Chancellor Fariña’s worldview.
Alas, this is not the first time the chancellor has played fast and loose with data. Take, for instance, her recent comments suggesting that the city’s charters push out low-achieving students to improve test scores. As Pondiscio and others have pointed out, despite numerous requests from charter school supporters, she has yet to supply any direct evidence supporting her bold accusation, even though the necessary data is stored in her office. To date, the chancellor’s spokesperson has only been able to produce a finding by New York City’s Independent Budget Office (IBO) that students with disabilities are more likely to exit charters than traditional public schools—despite the fact that IBO itself had previously repudiated it. (Indeed, when IBO recently released a follow-up analysis on special ed, its finding directly contradicted the chancellor.) Given such indifference to hard facts, one could scarcely expect the chancellor’s office to acknowledge the reality that students with disabilities and those learning English are—as demonstrated by my research (here and here), as well as that of the IBO—significantly less likely to exit New York City charters than traditional public schools.
With such ideologically driven leadership at the helm of America’s largest public school system, what’s in store for U.S. education reform efforts?
Not too long ago, high-quality empirical research on the impact of education policies was out of vogue. As recently as 2001, sociologist Thomas Cook lamented that education research was in the midst of an extended period of domination by a consensus that held that empirical research based on rigorous experiments was too simplistic to understand the complex nature of schools. Instead, researchers studied individual schools in isolated, bubble-like fashion, with the convenient consequence that overarching policy frameworks could never be applied.
Happily, empirical education research has since grown by leaps and bounds. At the time of Cook’s article, only one of 116 U.S. Department of Education-funded studies was a randomized field trial. Today, as any young assistant professor will tell you, it’s extraordinarily difficult to secure major research funding without employing random assignment or other strategies that can plausibly lead to causal inference. By notable contrast, Chancellor Fariña’s belief in her own judgment over evidence remains rooted in a dated, science-phobia mindset.
High-quality empirical research is, admittedly, not the exclusive domain of the modern education reform camp. When they occur, divergent findings should spark a serious, empirically grounded conversation on the proper direction of education policy. I’m eager for that conversation.
Empirical researchers should also, perhaps, attach greater importance to (serious) qualitative studies and other descriptive research. Yet this much is certain: High-quality empirical research offers invaluable objective information on the quality of American schools and, thus, on the impact of public policies. Pursuing the alternative—a reversion to the days when school chancellors adopted policies based on their gut instincts and “feel for the situation”—would be a grave mistake.
Marcus A. Winters is a Manhattan Institute senior fellow and an assistant professor at the University of Colorado—Colorado Springs. Follow him @MarcusAWinters.
