In the effort to reform American education, big-city school systems are where the action is. But remarkably, until now nobody could answer with a modicum of reliability a rock-bottom question: How are students faring academically in Los Angeles relative to those in Atlanta? There just wasn't enough information to make those kinds of city-to-city comparisons.
It is thus a very welcome development that the National Assessment of Educational Progress (NAEP) has embarked on a "trial urban district assessment" that looks at fourth- and eighth-grade student performance in ten cities. In fact, only nine of the cities provide useful comparative data, since the tenth - Charlotte-Mecklenburg - is demographically distinct.
What does this "trial" assessment - let's hope it continues - tell us? At heart, the lesson is: schools matter. Students who appear demographically similar learn more in some educational settings than in others.
For example, compare Houston and Washington, D.C. The comparison is politically fraught: Secretary of Education Rod Paige's record as superintendent of that city is under attack, as part of a general effort to discredit the No Child Left Behind reforms that are partly based on "the Houston miracle." Congress is contemplating vouchers for D.C. students. The new NAEP data provide ammunition for supporters of both voucher experiments and the testing and accountability provisions in the federal legislation. Houston does consistently well, while NAEP scores in the nation's capital are dismal.
On the 4th grade math test, an abysmal 64 percent of students in D.C. public schools lacked what NAEP defines as the minimal "basic" command of the subject expected of all students. But in Houston, only 30 percent of fourth-graders fell below the basic level--a big contrast. By 8th grade, the percentage with a less-than-basic command of math rose to 48 percent, still far better than D.C.'s 71 percent. Similar, though somewhat less dramatic, differences between the two cities showed up on 4th and 8th grade reading tests.
Of course, in all nine cities, the "Below Basic" figures tell only part of the story. Intercity differences may simply reflect variations in demographic makeup. The schools in D.C. have a much higher proportion of black students than those of Houston. Los Angeles, which ranks near D.C. at the bottom, has many more Hispanics than New York City, whose scores tend to cluster near the top, like Houston's.
Because of this, you need to break down the data by race and class. It might be, after all, that the higher concentration of Hispanics in Houston is obscuring African American scores that are as poor as those in D.C., where black performance cannot be concealed by averaging.
In fact, that's not the case. On 4th grade math tests, for instance, blacks in Houston came in first (average score 221), while those in D.C. placed last (average of 202). That 19-point gap is huge. On the 2003 national NAEP math assessment, black 4th-graders scored an average of 27 points lower than whites. Given the magnitude of the racial gap in academic achievement across the nation, the difference between average white and black scores is naturally likely to be greater than the difference between black student scores in two cities. But the difference in NAEP scores between black students fortunate enough to attend the Houston schools, and those unfortunate enough to be enrolled in the D.C. system, was a surprising two-thirds as large as the national black-white gap.
Figure 1. Math scores of black 4th- and 8th- graders by eligibility for federal lunch program, 2003
D.C.'s Hispanic 4th-graders were also at the bottom, while Houston's were again at the top, with a gap of 21 points between them. The national gap between whites and Hispanics in 2003 was also exactly 21 points - suggesting, again, the magnitude of the disparity between the two cities.
Figure 2. Math scores of Hispanic 4th- and 8th- graders by eligibility for federal lunch program, 2003
Black 8th-graders in D.C. again came in last on the math assessment - 19 points behind top-ranked Houston. Hispanics also did poorly in the nation's capital, but those in Los Angeles did even worse, scoring 21 points below Houston.
Reading scores varied a bit less from city to city, perhaps because math skills depend more on school quality. But the differences were still striking. Black 4th-graders in D.C., for instance, scored 17 points behind their peers in Houston and New York. In 8th grade, African Americans in Los Angeles did even worse than in D.C. Likewise, Los Angeles Hispanics were in last place, 21 points below those in top-ranked Chicago.
Figure 3. Reading scores of black 4th- and 8th- graders by eligibility for federal lunch program, 2003
Perhaps these inter-city differences in NAEP scores reflect social class disparities that are not apparent in looking at race or ethnicity alone. By the measures of family income and education, African Americans in low-scoring D.C. and Los Angeles could be much more disadvantaged than those in Houston or New York, with their higher levels of academic skill. That is, looking at the scores of black students in nine cities may obscure important socioeconomic differences; perhaps African American family income is typically much lower in D.C. than in Houston.
Figure 4. Reading scores of Hispanic 4th- and 8th- graders by eligibility for federal lunch program, 2003
In comparing such disparities in poverty rates, we run into a problem of imperfect information. NAEP identifies the children who participate in the federal lunch program, but eligibility for lunch subsidies is a crude proxy for income. Eligible children come from families across a wide economic spectrum, with incomes ranging from below $10,000 to more than $30,000 a year. Some black children falling into NAEP's low-income category, in other words, suffer deeper poverty than others. Likewise, in controlling for parental education, the data fail to capture subtle differences that may have an impact on school performance. Some high school dropouts have stayed in school longer than others, for instance. In addition, NAEP unfortunately gathers no information on family structure, and there could be intercity differences in the incidence of two-parent families that are related to achievement levels.
The data are thus imperfect; nevertheless, they're good enough to allow us to draw some conclusions. Most non-Asian minority students qualify for the federal lunch subsidy, and thus controlling for income might be expected to provide little additional information. Indeed, that turned out to be the case. Black 4th-graders in L.A., for example, scored 14 points below those in Houston, but that difference only rose to 16 points when more affluent kids were removed from the calculation. Thus, with respect to black and Hispanic youngsters, family poverty does not alter the basic story: Houston and New York ahead, Los Angeles and D.C. behind.
What about parental education? Perhaps the superior Houston results simply reflect a higher percentage of students with better-educated parents. Again, not the case. Taking parental education into account, the spread remained between Houston and New York, on the one hand, and D.C. and L.A. on the other. Looking at Hispanic 8th-graders whose parents lack a high school diploma, the difference in math scores between Houston and Los Angeles is 18 points. Hispanic 8th-graders who had a parent with some college education, but no diploma, scored 29 points higher in New York than in Los Angeles, a gap 9 points wider than the overall difference in Hispanic performance in the two cities.
What to conclude from these numbers? The new NAEP data reveal consistent inter-city differences when we look just at black and Hispanic scores. African American and Latino kids are faring much better in Houston and New York than in D.C. and Los Angeles. (Atlanta scores also hover near the bottom, but those in the four other cities in the study generally fall somewhere in the middle.) The racial gap in academic achievement has not disappeared - whites and Asians still outperform blacks and Hispanics nationwide - but the urban comparisons add a new dimension to our understanding of that gap.
Per pupil spending and class size do not explain these large inter-city differences. The district that tops the nation in most of the comparisons is Houston, yet District of Columbia schools spend 75 percent more per pupil. Further, the pupil-teacher ratio is about a third higher in D.C. San Diego is indistinguishable from Los Angeles in both expenditures per pupil and pupil-teacher ratios, yet it outperforms its Southern California neighbor, by an average of more than 8 points, in every comparison that can be made. Its students also almost always do better than those in D.C. and Atlanta as well, despite those cities' greater spending and lower student-teacher ratios. New York and Boston, to be sure, spend a lot and do comparatively well. But New York usually does better than Boston, even though the student-teacher ratio is more than 40 percent higher in the Big Apple.
Recent attacks on Houston's education reform record make that city's impressive accomplishments particularly noteworthy. It's not possible, in the available space here, to review the errors made by Houston's critics; suffice to say, they do not stand up to scrutiny. And that's good news for America's black and Hispanic students: testing and accountability, done well, can make a difference.
Abigail Thernstrom is a member of the Massachusetts State Board of Education and a senior fellow at the Manhattan Institute. Stephan Thernstrom is a professor of history at Harvard University. They are coauthors of the recently published No Excuses: Closing the Racial Gap in Learning (Simon & Schuster).
