Trust, but verify
The era of judging New York City Schools on academics is over. Robert Pondiscio
The era of judging New York City Schools on academics is over. Robert Pondiscio
With little fanfare, the New York City Department of Education (DOE) last month released a draft of its new “School Quality Snapshot”—Chancellor Carmen Fariña’s bid to evaluate each of Gotham’s more than 1,800 schools based on “multiple measures.” The DOE’s website invites public comment on the new reports until May 8. Here’s mine:
I confess I wasn’t the biggest fan of the single-letter-grade school report cards of the Bloomberg-Klein era. But as a signaling device to schools and teachers about what mattered to the higher-ups at the DOE’s Tweed Courthouse headquarters, they were clear and unambiguous: Raise test scores, but most importantly raise everyone’s test scores. With 85 percent of a school’s grade based on test scores—and 60 percent of the total based on test score growth—the report cards, for good or for ill, left little room for doubt that testing was king. Valorizing growth was an earnest attempt to measure schools’ contributions to student learning, not merely demographics or zip code.
Fariña, whose contempt for data I’ve remarked upon previously, values “trust.” You may worry if your child can’t read or do math. So does Chancellor Fariña, sort of. But she deeply cares if “teachers trust the principal at his or her word,” whether “teachers trust each other,” if students say teachers treat them with respect, and if parents say that a school’s staff “works hard to build a trusting relationship with parents.” The signal feature of the new report cards is the warm, enveloping embrace of the circle of trust.

Other elements of the draft Snapshot are equally warm and fuzzy. Sure, “rigorous instruction” is on top, but “student achievement,” like the proverbial pony, is buried beneath a pile of collaborative teachers, supportive environments, and strong family-community ties. In short, this document feels driven more by philosophy than data, relying on qualitative measures of uncertain value while strongly de-emphasizing student achievement in general and student growth specifically.
When she announced her intention to overhaul the school report cards last year, Fariña promised “the first balanced picture of a school’s quality,” one that “reflects our promise to stop judging students and schools based on a single, summative grade.” To be fair, it was widely acknowledged even in the pre-Fariña DOE that the reductive nature of the school report cards was having unintended ill effects. A 2013 white paper noted that the existing progress reports’ overreliance on state tests meant that “some educators have felt pressure to engage in test prep, narrowing the curriculum,” and that “finding the right balance of multiple measures is critical.” Even top-performing schools were anxious about their letter grades.
However, there are a couple of serious problems with this offering. Like a driver overcorrecting and losing control, the new reports go from focusing almost exclusively on student achievement to making it one of seven areas of reporting. And where 85 percent of a school’s value was based on test scores in the past, no weighting is given to any area of evaluation in the new reports. The biggest and potentially most troubling change is the removal of a coherent growth methodology by which schools are measured. The risk in de-emphasizing growth is that it might push schools and teachers back to focusing on how many kids pass the tests, which incentivizes focusing on “bubble kids” at or near the passing line. If a school’s going to be successful, everyone needs to improve.
If test data isn’t driving the new Snapshot, what is? Most of the information comes from the NYC School Survey administered annually to parents, teachers, and students, or else from a school’s “quality review”—ostensibly an extensive school visit in which an experienced educator observes classrooms, interviews school leaders, and evaluates how well the enterprise supports student achievement. Curriculum, teaching, assessments, school culture, and teacher collaboration are among the areas scrutinized during each school’s review.
In theory, it’s a robust and useful lens through which to view a school. At present, though, not every New York City school is reviewed every year (it’s been three years since some schools have been reviewed). Worse, there are reports that what used to be a rigorous three-day inspection is now being performed in as little as half a day—little more than a drive-by. (A DOE spokesperson has not responded to a request for comment on the number or duration of reviews performed or scheduled this year). Meanwhile, charter schools are not subject to quality review at all, which will render the Snapshots virtually meaningless (many growth-focused charters, it should be noted, benefitted mightily from the single-letter-grade report cards).
A former DOE official lamented to me privately that nearly everyone with data expertise was driven out of Tweed when Fariña took over. Thus, perhaps the greatest concern is not even the new Snapshots’ favoring qualitative measures over student achievement, but Tweed’s capacity to gather meaningful, high-quality data even on the things it prizes.
In sum, balancing achievement data with a quality review makes good sense; it was the direction in which the DOE was headed even before Fariña . But devaluing test data—combined with infrequent or cursory reviews—risks making what should be a dynamic and useful report to parents and teachers either static or a mere exercise in confirmation bias. At heart, New York City’s School Quality Snapshot feels like a political text. It is not hard to foresee it casting a warm glow on the kinds of schools Fariña likes, whether or not student achievement is robust.
Having worked in a DOE school for several years, I can attest that schools focus on gestures from Tweed as if they were smoke signals from the Sistine Chapel when a new pope is chosen. Thus, the most important function these reports serve is to indicate what matters to schools and teachers. The draft School Quality Snapshot says clearly and unambiguously that the days of measuring a school by academic performance in New York City are over.
In early April, I wrote that school choice is the highest form of fairness because it rewards positive behavior and aligns the interests of parents, children, and schools. Some disagree, arguing that school choice disadvantages the non-choosers. It is admirable to want to protect the most vulnerable students—the children of parents who do not or cannot engage effectively. But we must not do this at the expense of families who are engaged and do make good decisions for their kids.
As we parents often remind our children, two wrongs don’t make a right.
By encouraging parents to make choices, we also send an important message to students about our values. For the kindergartners at my school, a choice might be as simple as how to share the wooden blocks with a new friend. This simple and safe experience helps them practice the larger and more consequential decisions they will face.
Across the income spectrum, the parents I’ve met are concerned with our failure in both schools and civil society to inculcate critical values in our children. Research affirms the importance of persistence, delaying gratification, and other “gritty” non-academic values. If we ignore parent behaviors in the name of fairness, how can we expect children to appreciate those values?
A recent episode of This American Life underscored this dilemma. The reporter followed students from University Heights High School in the Bronx, a school whose poverty is inversely proportional to its proximity to Fieldston, one of New York’s famously wealthy private schools. The two schools operated an exchange program that allowed the largely underprivileged students from University Heights to glimpse the incredible opportunities available to those at Fieldston.
Speaking of her freshman year, a University Heights student named Melanie recalled the first few days. “This has got to be a joke…They don’t offer any AP classes….We didn't have a library, and I love books….We’re really in a poor part of the Bronx where we’re not being considered.”
Despite starting at a deep disadvantage, Melanie quickly emerged as one of the kids who would make it to college. Nominated for a prestigious (and highly competitive) Posse scholarship, her teachers and peers could all see her at Middlebury. But when she was dropped in the final round, Melanie lost her way completely.
Tracked down a decade later at the grocery store where she is now employed, Melanie acknowledged that she does the work she’d always hoped to escape: “wearing the uniform, servicing these people.” She meant the ones who'd gone to Fieldston.
But she knew that she was the one to blame: “I just grew angry at myself for making that choice of saying, ‘Well, I’m going to accept this,’ instead of fighting against it.” Melanie saw that losing out on the scholarship didn’t have to be the end of her dream. By failing to respond to the many teachers, both at University Heights and Fieldson, who reached out to her and wanted to support her, she let that happen.
The same reporter spent time with Jonathan Gonzalez and Raquel Hardy, classmates of Melanie’s who’d won scholarships to two esteemed colleges. Their paths were not much easier. Hardy told the reporter, “In the beginning, I struggled too. My first year, I got C-pluses and B-minuses. It was devastating to me because I was an A-plus student in high school, and we both were like, ‘This is a lot. This is crazy. I wasn't expecting this.’”
Her boyfriend Jonathan could not afford to buy textbooks and, embarrassed by his poverty, started skipping classes and hiding in his dorm room. Raquel, who faced the same struggle at her own school but knew to ask for help, told Jonathan to go to the university library.
I run a Brooklyn charter school, and I discussed this story with our Director of Operations, Ferrugia Sonthonax. “Oh,” Ferrugia told me, “I applied for a Posse too.“ Like Melanie, she did not win. But like Raquel and Jonathan, she knew to keep trying.
For Ferrugia, college was only a question of where, not if. Like generations of immigrant children before her, she attended the City University of New York and continued down the path her Haitian-born parents had always expected for her.
Critics of choice point to the Melanies and ask, “What if someone had tried harder to help her?” Speaking for myself, I wonder, “What if we helped to foster more Raquels, Jonathans, and Ferrugias?
The trolley problem is a thought experiment that asks if it is acceptable to harm one person to save five from a runaway train. Throw the switch and one person will be crushed; don’t, and five others will. The ways we respond to this ethical dilemma underscore our humanity.
Life is all about choices. Before critics denounce parents who look for alternatives to schools that routinely fail to prepare their students for meaningful participation in civil society, let’s hope they acknowledge the moral choice we are making for them.
Matthew Levey is the executive director of the International Charter School, a Brooklyn elementary school opening this fall. His three children attend New York City public schools.
Jack Jennings was the most influential education policy staffer on the Democratic side of Congress—probably on both sides—for the past half century. He served on the House Education Committee team for some twenty-seven years, then founded and led a well-regarded quasi-think tank called the Center on Education Policy, which continues to issue useful studies.
His new book, timed to coincide with the fiftieth anniversary of the Elementary and Secondary Education Act, is forceful, opinionated, informative, and sometimes quite wrong. (A simple example: He several times attaches my own stint in the Education Department to the wrong president. More importantly, he misstates Richard Nixon’s K–12 proposals and incorrectly describes their handling by Congress.) As Andy Rotherham says on the back cover, “If you agree with everything in this book, you probably didn’t read it closely.”
But there’s much useful history and perceptive analysis here, as well as some pie-in-the-sky recommendations for the future. Particularly interesting to me was how Jennings traced the onset of federal involvement with results-based accountability to the 1988 Title I amendments shaped by Committee Chairman Augustus Hawkins. Those revisions, he writes, “marked a change in attitude among congressional leaders, characterized by increased demands on educators to show academic results as a consequence of receiving federal aid. These leaders no longer believed…that the problem was simply a lack of money. Rather, the belief was growing that schools receiving federal funds should demonstrate that their students were making greater academic progress.”
This is an important addition to the usual account, which leaps from A Nation at Risk in 1983 to the Charlottesville summit in 1990, then to Goals 2000 and IASA four years later.
Jennings acknowledges that these ever-greater demands for results and tighter federal strings were products not just of worrisome education mediocrity, but also decades of cumulative evidence that simply handing out money via programs like Title I wasn’t accomplishing much. He deserves credit for not glossing over such failings, as well as credit for some keen insights into what does and doesn’t work in the federal education policy sphere.
But he does not, in my view, deserve credit for the suite of federal policies he proposes for the future, which comprises a radical increase in spending (to be financed from—where else?—cutting defense and homeland security), a major tightening of the screws on states to force them to make all the changes that Jennings believes would benefit kids, and, finally, either a Supreme Court decision or constitutional amendment to turn educational equity (and funding) into a federal obligation. Opinions will differ as to whether any of this has merit in the abstract. But I can’t picture any scenario under which it would actually happen for as far into the future as I can peer.
SOURCE: Jack Jennings, Presidents, Congress, and the Public Schools: The Politics of Education Reform (Cambridge: Harvard Education Press, 2015).
ACT’s new report is based on a survey it administered to graduating high school seniors who took its college entrance exam, a cohort that now comprises 57 percent of the nation’s graduates. The report analyzes data on the self-reported career interests of nearly 1.85 million students, compared to those who took the ACT in the previous four years; it focuses particularly on those who expressed an interest in education as a profession. This includes survey respondents who planned to major in administration/student services, general teacher education, the teaching of special populations (e.g., early childhood, special education), and the teaching of specific subject areas like math or a foreign language.
The researchers found that between 2010 and 2015, the total number of graduates who planned to work in education decreased more than 16 percent—even though the number of ACT test takers rose 18 percent. Similarly, the percentage of all test takers planning to walk that career path decreased from 7 percent in 2010 to 5 percent in 2016. These students also achieve lower ACT scores than the national average in math, science, and reading—something that was also true in 2010. And the cohort is less diverse than some might prefer: 72 percent of those indicating interest in education were white, which was 16 percentage points higher than the proportion of all test takers.
All in all, it’s a grim snapshot. The individuals who will teach our kids in the future should outperform the national average, not the other way around. When will we heed the call to reform the teaching profession?
SOURCE: “The Condition of Future Educators 2014,” ACT, Inc. (April 2015).
A new report from the Institute of Education Sciences presents new data from a national survey of teachers, which is part of a longitudinal study of public school teachers who began teaching sometime between the school years 2007–2008 and 2011–2012. Of the many findings, six stand out.
So what should we make of all this? Well, the study confirms several things that we already know: Many beginning teachers leave the school they are working at either because they move to a new school or because they leave the profession entirely, and most teachers who leave the profession do so voluntarily (because they’re dissatisfied).
If we want to retain more of these teachers, the study suggests that there are a couple ways to go about it. We could pay them more, or we could assign them mentors in their first year of teaching (which seems like common sense given that more than half of the teachers who leave the profession do so after their first year).
Finally, one statistic that stood out to me was that 40 percent of fifth-year teachers who changed schools did so involuntarily. I think it’s worth asking how many of these folks are good teachers who were laid off and how many are bad teachers who are just being shuffled from one school to another. Unfortunately, this report doesn’t tell us that.
SOURCE: Lucinda Gray, Soheyla Taie, and Isaiah O’Rear, “Public School Teacher Attrition and Mobility in the First Five Years: Results From the First Through Fifth Waves of the 2007–08 Beginning Teacher Longitudinal Study,” U.S. Department of Education, National Center for Education Statistics (April 2015).