Diane Ravitch's June 7 Gadfly article took the New York City Department of Education to task for hyping the most recent reading scores for students in grades 3-8. "The scores," she wrote, "were mainly flat or declining." And the much-ballyhooed rise in eighth-grade reading "downplayed the curious fact that eighth grade scores were up across the state." Her interpretation drew a critical review from David Cantor, the NYC Department of Education's Press Secretary. His critique and Ravitch's response follow.
Cantor: Don't trivialize Bloomberg's accomplishments
Regarding Diane Ravitch's blithely unprincipled dismissal of New York City's 2007 elementary and middle school English Language Arts scores ("Reading scores in New York City: Achievement Stalled"):
Two years ago, shortly before the announcement of fourth- and eighth-grade reading results, Ms. Ravitch pointed to a possible rise in scoring because the state had liberalized its 2005 test exemption policy for English language learners (ELLs). "This change will remove an unknown but significant number of low-scoring students from the testing pool and boost scores," she wrote. When fourth-grade scores were subsequently revealed to have risen by an impressive ten points, Ms. Ravitch continued to point to the exemption of larger numbers of ELLs to help explain the increase.
Earlier this year the policy shifted again, but in the other direction. Responding to a federal directive, the state required all ELLs with at least one year in school to take the ELA test. Previously, these students were exempt until after at least their third year. The change in the 2007 test-taking population was dramatic: whereas the 2005 policy shift had affected about 2,100 fourth graders, under the new rule nearly 8,000 more ELLs in fourth grade were tested in 2007. Put another way, the share of all fourth-grade test-takers who were ELLs rose from 5% in 2006 to 16% in 2007. And the share of test-takers in all grades who were ELLs more than doubled, from 6% in 2006 to 13% in 2007.
Ms. Ravitch conveniently ignores these developments and their influence on the 2007 scores. At pains not long ago to emphasize the score-enhancing possibility of another, considerably less substantial ELL exemption policy, she suggests the recent scores are best described without recognizing the existence of the new policy--one whose effect, it is universally agreed, was to depress the scores.
The results are absurd. Ms. Ravitch presents a grade-by-grade comparison between 2006 and 2007 scores that doesn't mention the fact that the number of ELLs tested in third grade in 2007 rose by 570%, for instance. She compares the performance of what she calls "the same cohort of students" as they advanced in grade from 2006-2007, but the students who took the test this year differ substantially from those who took it last year, confounding her analysis.
Ms. Ravitch can hardly be unaware of the landscape. The State Commissioner of Education highlighted the new exemption policy when he announced test results in May. The issue has received wide play in both mainstream and educational media, as has the fact that city students proficient in English continued to make gains across the board. "Let's look at New York City, again an apples to apples comparison--grade three held onto the high performance from last year and in every other grade [performance] went up," said State Commissioner Richard Mills. (Moreover, ELLs, despite the very much larger number who were tested and the short time many have spent in this country, performed at their highest level yet.)
Ms. Ravitch's aim appears to be trivializing the accomplishments of the Bloomberg administration. She would judge the Mayor on the basis of fourth-grade reading gains, calling fourth grade "the true testing ground of mayoral control." But she refuses to give the Mayor credit for the first year he actually controlled the system (with its large gains). And she makes no effort to measure the city's achievement under mayoral control by comparing it to that of other large cities in the state and to the state as a whole--the only way to control for changes in the difficulty of the test.
Here are the easily verifiable facts: since the Mayor took control of the schools, whether you begin counting in 2002 or 2003, New York City reading gains have substantially outpaced other cities and the rest of the state in both fourth and eighth grades, the two grades for which we can make comparisons over time. Grades 3-8 in aggregate (based on a scale that aligns past city scores with the state) have risen 11.5 points since 2002, and 9.8 points since 2003 (plus another five points if ELLs are held constant to control for policy changes). It's worth adding that under the Bloomberg administration, math gains for grades 3-8 have risen 27.8 points since 2002 and 23.2 points since 2003.
Ravitch: Mayor's accomplishments owe more to press strategy than education strategy
Issues in education today, especially those involving test scores and graduation rates, need more discussion, more public review, more candid exchanges among scholars and public officials. Yet Mr. Cantor's letter shows how the Press Office of the New York City Department of Education reacts to anyone who dares to question "the accomplishments of the Bloomberg administration."
By now, independent observers of the Bloomberg administration's education program--that is, those who do not work for the administration, whose universities do not have multi-million dollar contracts with the administration, whose organizations do not receive grants from the Bloomberg Foundation, or who are not themselves on the Mayor's private payroll---realize that many of its "accomplishments" are more to be credited to a press relations strategy than an educational strategy.
Consider the "historic" score gains of 2005, which by coincidence was the year that the Mayor ran for re-election. For whatever reason, the New York State Education Department informed school districts in January 2005 that it would not be necessary to test students with limited-English proficiency who had resided in this country for less than five years. I and others (most notably, New York City Public Advocate Betsy Gotbaum) questioned whether and to what extent this new policy affected the reported gains; after all, one reliable way to raise test scores is to exclude historically low-performing students. At that time, the New York City Department of Education insisted that very few such students were excluded, and that the exclusion of so few had no bearing on the results. Subsequently, the federal Department of Education directed the New York State Department of Education to include these students in future tests so as to comply with federal law.
As it happened, in 2005 New York City schools saw a gain in fourth-grade reading scores, but Syracuse, Yonkers, and Rochester saw even bigger gains. This led some observers to wonder if the test was easier, as the gains were so large as to be suspect (10 points in New York City, 15 points in Rochester). It was not me, but the New York Times that drew attention to the peculiar statewide pattern.
Since then, every announcement of state test scores is accompanied by a jubilant press release from the NYC Department of Education, boasting that the latest scores are the best ever, most historic, most dramatic, etc. I do not doubt that there have been gains, though it is impossible to decipher their significance without independent scrutiny by researchers who do not work for the Department of Education. Most people would not consider the Department of Education's Press Office a credible source of information and objective analysis about the condition and progress of education in New York City over these past five years.
Since Mayor Bloomberg took control of the public school system, its budget has increased by $1 billion every year (not including any of the anticipated revenue from the Campaign for Fiscal Equity lawsuit). Some portion of that additional spending has underwritten intensive test-preparation activities in every classroom that participates in the state testing program. Earlier this year, the Department of Education awarded a $90 million contract to McGraw-Hill to administer five additional interim assessments each year in reading and in mathematics.
Apparently, education is now defined as getting ready to take the test and then taking the test and then starting all over again to prepare for the next test. I am willing to bet that neither Chancellor Joel Klein nor Mayor Michael Bloomberg would have inflicted such an "education" on their own children.
I have twice found the New York City Department of Education making claims that would never pass the scrutiny of independent analysts:
- Earlier this year, I discovered that the DOE was wrongly claiming credit for the big test score gains of 2003, which preceded the implementation of their reforms. In the news stories published in 2003 when the scores were first published, it is clear that the administration recognized they had nothing to do with those scores. Indeed, at the moment that the children were taking the tests in January 2003, Mayor Bloomberg and Chancellor Klein were announcing their plans and selecting their appointees. Their program was launched in September 2003. Yet, all of the recent press materials from the DOE claim credit for scores that were recorded before their reforms were put in place.
- A few weeks ago, when the state math scores were released, the DOE press office claimed (as Mr. Cantor's letter repeats) that math scores for grades 3-8 have risen "27.8 points since 2002 and 23.2 points since 2003." How did they arrive at these numbers? The state began testing students in grades 3-8 only in 2006, to comply with NCLB. So the DOE blithely combined the results of city tests administered in previous years, but now discontinued, with the state tests of 2006 and 2007. Is there any reputable psychometrician prepared to say that this action--merging the results of these two entirely disparate series of tests--is responsible and valid? If so, none has stepped forward to certify the validity of these numbers. And by the way, the total gain for grades 3-8 on the state tests of English language arts from 2006-2007 was .001 percent--one-tenth of one percentage point. This unimpressive result after years of mayoral control suggests why the administration is so eager to add the now-obsolete city test scores to their own present-day small gains.
So, let's look at the data, without Mr. Cantor's spin.
In fourth-grade English language arts, which the state has tested from 1999-2007, scores have increased from 52.5% meeting standards in 2003 (when the Bloomberg program began) to 56% meeting standards in 2007. There has been a gain of 3.5 percentage points since the Mayor's reforms were installed. In the years before mayoral control began, under Chancellors Rudy Crew and Harold O. Levy, the reading scores in this grade rose from 32.7% to 52.5%. From 1999 to 2003, under Crew and Levy, there was a gain of 19.8 percentage points, as compared to a gain of 3.5 points under mayoral control.
In fourth-grade mathematics, which the state has tested from 1999-2007, scores increased from 66.7% in 2003, when the Mayor's program began, to 74.1% in 2007, for a gain of 7.4 percentile points over the years of mayoral control. Not bad, but not nearly as impressive as the gains that occurred prior to mayoral control. From 1999 to 2003, math scores rose from 49.6% to 66.7%. Under Crew and Levy, there was a gain of 17.1 percentage points, as compared to a gain of 7.4 percentage points under mayoral control.
In eighth-grade English language arts, which the state has tested from 1999-2007, scores increased from 32.6% meeting standards in 2003 to 41.8% in 2007, a gain of 9.2 percentage points. Before mayoral control, scores were flat or falling, declining from 35.3% in 1999 to 32.6% in 2003. Only in this grade and in this subject were the gains higher after mayoral control than before mayoral control. However, eighth-grade scores in reading soared in 2007 by similar or greater margins in districts across the state, raising questions about whether the state test was easier than in previous years.
In eighth-grade mathematics, which the state has tested from 1999-2007, scores increased from 34.4% meeting standards in 2003 to 45.6% in 2007, a gain of 11.2 percentage points. Before mayoral control, under Crew and Levy, scores increased from 22.8% in 1999 to 34.4% in 2003, a gain of 11.6 percentage points. Thus, the gains for this grade and subject are virtually tied before and after mayoral control.
One wonders why a majority of students in eighth grade continue NOT to meet state standards, if "social promotion" was truly ended, as the administration says. How is it possible that so many eighth-grade students fail to meet state standards in reading or mathematics if the DOE actually eliminated social promotion in third grade, fifth grade, and seventh grade? Why do we not see 80-90% of students in this grade meeting the standards after holding back the low performers in previous grades?
No wonder the Bloomberg administration has upped the population of the Education Department's press and communications office to 29 people, according to Sol Stern ("Grading Mayoral Control," City Journal, Summer 2007). Before mayoral control, the press office under Crew and Levy had only four people. Having good press, it turns out, is even more valuable than having good education.
If only the same amount of funding were applied to the establishment of a professional research department, and if only that department were allowed to serve as a nonpartisan, independent agency rather than as a booster of the Mayor's and the Chancellor's "accomplishments," then the rest of us might learn some useful lessons about what is really happening in the schools of New York City.