National Association of State Boards of EducationOctober 2005
Henry I. Braun, Educational Testing Service September 2005
Together, these booklets offer useful insights into value-added testing models - how they work, their strengths and weaknesses, and issues to consider before implementation. NASBE delves into the models' practical tradeoffs, but leaves largely untouched the question of how they actually work. ETS fills this void and offers descriptions of the statistics and processes used to determine which schools and teachers are "adding value" to their students' scores each year. Understanding value-added models is no easy task. NASBE notes that Tennessee's much-discussed program "involves the solving of thousands of equations iteratively," and uses software which "enables a massive multivariate, longitudinal analysis using all achievement data for each student no matter how sparse or complete." This complexity may be value-added analyses' greatest challenge: Will the public accept high stakes accountability based on metrics they must simply trust to be reliable? Perhaps an equal challenge is that the nature of these systems - which can account for "'external' factors such as socioeconomic status, race, parents' educational levels, or innate ability" in assessing a school - might hurt NCLB's no-excuses mentality. As a measurement tool, it may be appropriate to account for students' backgrounds, but as policy it could send a dangerous signal. (William Sanders, father of the Tennessee model, happens to argue against using student characteristics, because while they correlate with current scores, in his view "the correlations... with gains are essentially zero.") States must ask themselves whether models that account for demographics, or those that relate performance to absolute standards (which "growth models" must do under NCLB), better meet their particular needs. Anyone using a value-added system should reflect on both of those policy issues and decide how best to handle them, especially if they want to win approval from Education Secretary Spellings to use a "growth model" as part of their NCLB accountability systems (see here). It's a complex topic, and these booklets are certainly helpful. You can find them online here and here.
