A. Abdulkadiroglu, J. Angrist, S. Cohodes, S. Dynarski, J. Fullerton, T. Kane, P. Pathak
The Boston Foundation
January 2009
This report evaluates Massachusetts's several types of school options: traditional public schools, charter schools, and pilot schools. (For those who may be unfamiliar with the last of these, pilot schools were born as the Boston Public Schools' (BPS) and Boston Teachers' Union's (BTU) 1995 response to charter schools; described by the authors as a "middle ground," they have curricular, staffing, and scheduling autonomy but retain a collectively bargained pay scale and seniority rights.) The report combines two parallel studies. First, analysts used Massachusetts state test data to track individual student progress over time and by school. This piece included all schools in the Bay State and controlled for observable characteristics (e.g. race, English language proficiency etc.) but not for such intangibles as student motivation. Second, analysts evaluated school lotteries to compare students who did and did not receive spots in charter and pilot schools; this portion sought to control for both observable and unobservable student characteristics but only evaluated schools and years where there were significantly more applicants than open spots and where lottery records were complete. The findings are telling: the authors found "large positive effects" on student achievement for charter schools, but none for pilot schools, when both kinds of choice-schools were compared with traditional public schools. In fact, pilot school students may actually lose ground in some areas compared to BPS students, although these findings were not statistically strong. The authors note that, taken in isolation, each portion of the study's design has challenges: data may be "potentially affected by selection bias"--e.g. charter and pilot schools might skim the best students--and not all pilot schools participate in lotteries. Nonetheless, this holistic research design is worth maintaining--and reusing--as Massachusetts and other states continue to expand school choice options. You can find the full study and data analysis here.