As any education researcher will tell you, conducting a "scientific" study of educational programs or practices is difficult at best, primarily because so many factors contribute to pupil achievement, including students' previous knowledge, teacher quality, the degree of parental and community support, etc. But a recent article by education blogger Joanne Jacobs questions whether these underlying difficulties are the true cause of the dearth of scientifically based research in the field, or whether the problem is more fundamental: most education researchers don't really know how to conduct rigorous scientific studies. Already this year, the Chronicle of Higher Education (click here) reported that "fewer than 10 percent of American Education Research Association [AERA] members are knowledgeable about randomized trials. And even fewer have actually worked on a randomized trial." Worse still, rather than learn the skills necessary to conduct such studies (which are now required to earn the "scientifically based" moniker), education researchers demonstrate "a deep well of hostility to cold, hard, number-heavy science." And, as Thomas Cook has shown, the intellectual culture of colleges of education has little use for randomized experiments and kindred research. As a result, economists (like Caroline Hoxby, mentioned above), statisticians, and psychologists are increasingly being called upon to conduct the scientifically based studies that traditional education researchers are unwilling to take on.
"Rigor-free research," by Joanne Jacobs, TechCentralStation.com, September 27, 2004
"Sciencephobia," by Thomas D. Cook, Education Next, Fall 2001