Tom Loveless, Brown Center on Education Policy, Brookings Institution
September 2002
Tom Loveless, director of the Brookings Institution's Brown Center, has taken to issuing annual reports on American education, each examining several topics in interesting and provocative ways. This year is no exception. The new Brookings report takes up three issues. The one that got scant press attention-doubtless because it revealed no alarming problem-is whether high schools that are "sports powerhouses" are weaker academically than schools less adept at athletics. The answer is no, no "zero sum" game is at work, and there's "no evidence that schools suffer academically when they excel at athletics." In fact, the two forms of success may even be "mutually reinforcing." The other two issues in this report are getting big-time media notice, however. One deals with arithmetic. Loveless asserts that, despite some evidence that U.S. students and schools have made modest gains in math achievement in recent years, they have "stagnated or even declined" when it comes to computation in general and arithmetic in particular. His great concern is that America is turning its back on arithmetic, not paying nearly enough attention to it in an era when schools and school systems (and state standards) are more attentive to the "problem-solving skills" emphasized by the National Council of Teachers of Mathematics. (The NCTM, it's fair to say, is not Loveless's favorite group.) His particular beef in this report is that the National Assessment of Educational Progress (NAEP) is overly enamored of NCTM-style math and doesn't even report a separate sub-score on computation or arithmetic whereby that particular form of achievement could be accurately tracked. He charges that the "main NAEP" exam pays too little attention to computation. But he broke some individual test items out of the NAEP "trend exam" (a different test, used for longitudinal tracking) that he believes are primarily about arithmetic, and on those items he finds declining prowess, particularly for older (17 year old students). NAEP staff reply that Loveless did not select all the pertinent test questions and that, if he had included those dealing with percentages, he would have found a stronger level of performance. They also say that other branches of math wouldn't be showing even (modest) achievement gains if the kids couldn't do basic arithmetic. (Loveless replies that the "main NAEP" exam lets them use calculators.) We can't resolve this one but we agree with Loveless that arithmetic and computation skills are fundamental to success in math and that it would be good for NAEP to track them in a clear and consistent way-along with the rest of this key subject. The Brown Center's biggest headlines this year were attached to the report's third section, which says that charter-school students aren't doing as well on state tests as their peers in regular public schools. This issue is fraught with data complexity and analytic difficulty, and in fact Loveless's conclusions are tentative and circumscribed. The press, however, didn't pay much attention to the qualifiers and limitations, and, as a result, charter-school protagonists are mightily irked at Loveless and Brookings. Some of the irk is warranted: Brookings's own news release had an alarmist headline and its publicists doubtless craved exactly the kind of attention that the report has gotten. If one actually reads it, however, one learns essentially what we have learned from sundry other studies of academic achievement in charter schools-most of which have not been around long enough for any definitive conclusions to be reached: in some states, their test scores are lower than those of public schools serving demographically similar youngsters. That's not true everywhere, however. Indeed, in just four of the ten states examined by Brookings were the differences statistically significant. In one of the other six (Colorado), charter students actually did better. Loveless acknowledges how little any of this really proves and how varied the explanations may be: "One possible explanation is that charter schools are not doing a very good job. But an equally plausible explanation is that charters attract large numbers of students who are struggling academically in public schools before ever setting foot on a charter school campus. The charters, in fact, may be doing an excellent job, bringing these low achievers up to a level that, although still below average, is not as low as when the students attended public schools." Too bad this part didn't make it into the press coverage. We know from plenty of charter-school studies that many of their pupils were indeed far behind the educational 8-ball to start with. (We also know that many charter schools are new and still getting their acts together.) If there was ever a case for "value-added" analysis rather than simple comparisons of average test scores, it's here. But an inconclusive report along those lines wouldn't garner much attention and thus, in the world of contemporary think tanks, would scarcely be worth issuing. You can obtain a copy of the Loveless/Brown report at http://www.brookings.edu/dybdocroot/gs/brown/brown_hp.htm.
