I've just finished reviewing the latest Brown report from Tom Loveless at Brookings for this week's Gadfly. And it's a good one. It's a three-parter and I suggest you read the entire trio, but I was most interested in his PISA (Program of International Student Assessment) study, which is the real headliner. That's because NGA and other national groups are looking to the test as the holy grail of international standards and assessment*. Loveless's study, in a nutshell, presents a stinging indictment of PISA's fascination with political ideology and student attitudes???neither of which belongs in a test of science (he doesn't go so far as to say that, but I am???.).
PISA, you see, asks lots of questions about self-efficacy in science, as opposed to science content itself. Added up, these questions give us measures of students' ???self-concept in science,??? ???enjoyment of science,??? ???interest in scientific topics,??? and ???future motivation to learn science,??? to name just a few. PISA finds a positive correlation between these things and student achievement, but Loveless finds otherwise--the more confident kids are in their science abilities, the lower that nation's scores.
But one question: Who cares either way? This isn't a test of confidence. It's supposed to be about what kids actually know about math and science content, not what they think they know about math and science, or what they are motivated to want to learn! Loveless says it this way: ???The danger is acute in PISA because the attitudinal questions elicit students' beliefs about issues, not their knowledge of issues???[and] history teaches us that beliefs untethered from knowledge can make for some rather unscientific decision making.??? Here's hoping that we slow down and take a serious look at what this test is measuring before we deem it worthy of an international benchmark.
*The NGA has asked us to clarify that it doesn't view PISA as the ???holy grail of international standards and assessments.??? PISA was mentioned in its recent benchmarking report as one of several international exams that states might look toward when developing more rigorous, and more common, standards and internationally benchmarked assessments. (Corrected on February 27, 2009)
