The new federal education law, the No Child Left Behind Act, demands in many places that programs funded by federal dollars be supported by "scientifically based research," but among practitioners, and even some researchers, there is great uncertainty about what this means. To clarify what scientifically based research is and to explain why it is so crucial, the U.S. Department of Education hosted a seminar on the topic in February 2002. The transcript of that seminar, as well as papers by the presenters, are now available on the department's web site. In the opening presentation, Valerie Reyna of the Office of Educational Research and Improvement describes the alternatives to scientific research and lays out the logic behind randomized field trials-which she describes as the best kind of evidence-and other kinds of research (including quasi-experimental or correlational studies) in plain language. In another presentation, Steve Raudenbush of the University of Michigan describes how the medical profession was gradually converted to the view of basing its practice on scientific research, explores when random assignment studies are desirable and when they are not, and explains how we can judge the scientific quality of studies that do not use random assignment, with the key factor being whether investigators have effectively evaluated competing explanations for what is found. A transcript of and papers from the seminar on scientifically based research hosted by the U.S. Department of Education can be found at http://www/ed.gov/nclb/research.
An article in last week's Economist describes an international effort to summarize the evidence for the effectiveness of various social interventions, including education initiatives. The Campbell Collaboration, which held its second annual conference in Philadelphia last month, brings together social scientists, statisticians, and policymakers to identify all relevant experimental studies on a topic, such as the effectiveness of whole-language reading instruction, and review them systematically, choosing the best studies using clearly defined criteria and combining the results in a statistically valid way. So far, the group has identified 11,000 experimental studies (i.e. studies that involve random assignment) in all of the social sciences, and reviews have been completed on a range of topics. For more see "Try it and see," in the February 28 issue of the Economist. More information about the Campbell Collaboration can be found at http://campbell.gse.upenn.edu.