National Association of State Boards of EducationOctober 2005 Henry I. Braun, Educational Testing Service September 2005
National Association of State Boards of EducationOctober 2005
Henry I. Braun, Educational Testing Service September 2005
Together, these booklets offer useful insights into value-added testing models - how they work, their strengths and weaknesses, and issues to consider before implementation. NASBE delves into the models' practical tradeoffs, but leaves largely untouched the question of how they actually work. ETS fills this void and offers descriptions of the statistics and processes used to determine which schools and teachers are "adding value" to their students' scores each year. Understanding value-added models is no easy task. NASBE notes that Tennessee's much-discussed program "involves the solving of thousands of equations iteratively," and uses software which "enables a massive multivariate, longitudinal analysis using all achievement data for each student no matter how sparse or complete." This complexity may be value-added analyses' greatest challenge: Will the public accept high stakes accountability based on metrics they must simply trust to be reliable? Perhaps an equal challenge is that the nature of these systems - which can account for "'external' factors such as socioeconomic status, race, parents' educational levels, or innate ability" in assessing a school - might hurt NCLB's no-excuses mentality. As a measurement tool, it may be appropriate to account for students' backgrounds, but as policy it could send a dangerous signal. (William Sanders, father of the Tennessee model, happens to argue against using student characteristics, because while they correlate with current scores, in his view "the correlations... with gains are essentially zero.") States must ask themselves whether models that account for demographics, or those that relate performance to absolute standards (which "growth models" must do under NCLB), better meet their particular needs. Anyone using a value-added system should reflect on both of those policy issues and decide how best to handle them, especially if they want to win approval from Education Secretary Spellings to use a "growth model" as part of their NCLB accountability systems (see here). It's a complex topic, and these booklets are certainly helpful. You can find them online here and here.
United States Government Accountability Office
November 2005
The District of Columbia has a larger percentage of its students - over 20 percent - in charter schools than any state (although in urban districts, it trails Dayton for highest percentage of charter students). Their schools are approved and overseen by two authorizers - the D.C. Board of Education (BOE), which also runs the city's traditional public schools, and the D.C. Public Charter School Board (PCSB). How well are these two authorizers doing their jobs, and in what areas can they improve? That's what this report, required by the 2005 D.C. Appropriations Act, sets out to answer. This informative study neatly examines the authorizers' resources (money and staff) and how they are used, oversight practices and response to school closures, and use of services available to them. To accomplish this, GAO researchers analyzed the authorizers' budgets, pored over documents and monitoring reports related to school closures, and conducted interviews. The results? Both authorizers are doing a fairly good job, but, especially for the BOE, there's lots of room for improvement. (BOE has been criticized by a recent Progressive Policy Institute report as well.) For one, the BOE gives the same level of oversight to all its schools. It would do far better to focus its energy and limited resources on struggling charters where students aren't achieving in the classroom or where internal management isn't up to par. Also, the BOE did not always review monitoring data "in time to react effectively and in a timely manner" to potential problems brewing in its charters. But most instructive is the report's finding that neither the BOE nor the PCSB has a process in place to guide schools, students, and families through a charter school closing. The study laments the possibility that, without such a process, student records may be lost, student transfers become needlessly complicated, and the authorizers do not learn from or build upon the experiences of past closures. The report is informative, and it provides valuable information, not only for D.C. charters, but for charter authorizers in other cities, too. The report is available here.
Sensitive readers: Avert your eyes. The Lexington Herald-Leader reports that 15-year-old Davie Miles of Kentucky's Nicholas County High School urinated in his gymnasium's ice machine. Some 30 students and staff reportedly consumed ice from the dispenser before the contamination was reported and the machine was quarantined. Panic ensued shortly thereafter. Only when county schools police chief Ben Buckler contacted the state's Health Department, and was told that the incident (although "gross and morally wrong") didn't represent a health risk, were parents somewhat calmed. But why did Davie do it? A new form of territorial gang marking, perhaps? Not quite. Bullying emerged as the true culprit. Miles was reportedly so displeased with his school, so distraught by the daily teasing he endured there, that he decided public urination into an ice machine was his only remedy. "I'd do anything to get out of that school," he said. As for the ice machine, Buckler said if the school decides to use it again, it will undergo a thorough cleaning.
"Nicholas County student urinates in ice machine," by Cassondra Kirby, Lexington Herald-Leader, November 18, 2005
"Student links urinating incident to bullying," by Cassondra Kirby, Lexington Herald-Leader, November 22, 2005
Paul Hill's National Charter School Research Project at the University of Washington recently issued a wide-ranging, nicely balanced appraisal of the charter-school movement circa 2005 - emphasizing how loosely coupled it is, how different from state to state and school to school, indeed how it's not fairly termed a national movement at all. (See here.)
One chapter is especially provocative and timely: Andy Rotherham on "the pros and cons of charter school closures."
Everyone knows that a key element of the "charter bargain" is that non-performing schools are supposed to vanish via sponsors shutting them or not renewing their contracts. Yet in the real world this doesn't happen very often. Across all the states that reported school-closure data for 2004-2005, Rotherham says, there were just 65 such events, involving only 2 percent of the schools in those states. Fifteen states reported no closures.
Yes, there are occasional high-profile events such as the abrupt shut-down (on grounds of financial mismanagement) of the multi-campus California Charter Academy in summer 2004. In fact, that's where the topic begins to get interesting and complicated. Did that school's sponsors behave conscientiously? Did they even have the right incentives to behave conscientiously? One doesn't have to dig very deep into the CCA saga, Rotherham explains, to discover that the authorizers, far from being attentive watchdogs, were complicit in something akin to charter fraud. They were making money from the authorizer fees they were collecting.
I don't know much about how it works in California, but in Ohio, where the Fordham Foundation is an authorizer, the main source of revenue to sponsors (other than whatever money of their own they can afford to apply to this work) is fees paid by the schools and calibrated to the schools' enrollment. In effect, authorizers have a financial incentive to sponsor lots of schools, to sponsor bigger schools, and not to close them down (or reduce their size), no matter what.
This is perverse and wrong, and it flies in the face of responsible authorizing. And it's exacerbated by the political difficulty of closing (or even threatening to close) weak charters. "That is because," Rotherham writes, "charter schools, even poorly-performing ones, are proving wildly popular with parents.... The old adage that closing a school is like moving a graveyard is as true of charter schools as it is of traditional neighborhood schools."
We certainly see this in Ohio, where desperate parents cling to their charter school even when it fails to deliver on its academic promises, if only because it's safe, friendly, and welcoming. The Dayton Daily News's perceptive education writer, Scott Elliott, recently wrote that "what appeals to parents" are "access, discipline, character building, back-to-basics." In state after state, efforts to close or non-renew a charter school run into fierce resistance from parents and community residents (and their elected officials) as well as from school operators (and their attorneys).
Must we abandon this crucial element of the "charter bargain"? Rotherham embraces the NACSA principles of quality authorizing and sets forth the two obvious choices for authorizers: either be fussy in advance about the schools they authorize or relentless (the heck with politics) about closing those that malfunction.
Yet state funding schemes (as in Ohio) push the other way. Being too fussy in advance also means avoiding risks that might be worth taking, such as when someone without much of a track record still has the spark that could yield a terrific school. And being relentless about closures is simply na??ve, both about political realities in American cities where desperate parents face bad alternatives, and about the readiness of school operators - and, especially, their prospering lawyers - to fight all such moves in court. Or (in places with multiple authorizers - another key tenet of charter-school theology) the operator of a failing school will simply seek a less persnickety sponsor.
"In practice, therefore," Rotherham correctly notes, "high-quality authorizers find themselves between these two extremes."
What to do? Today the charter movement is feeling its way. There aren't many good precedents in K-12 education. Nobody except the marketplace closes private schools. And district-operated schools almost never close (save when enrollment plummets) and certainly not due to quality issues (though NCLB pressure to "restructure" them may change that).
If the authorizers don't figure this out, the charter movement is in trouble. Ohio House speaker Jon Husted, a longtime champion of charter schools, spelled this out for attendees at an important statewide conference on charter quality on November 18. State officials, he said bluntly, will continue imposing regulation after regulation on both schools and sponsors until and unless the "movement" finds a way to assure quality and performance.
My own view in late 2005 (subject to change as we get more experience in the authorizing role) can be summarized in this way:
We gotta get this right.
Even if you don't believe that principals should have unfettered choices when it comes to hiring new teachers (see here), surely few people would defend a system whereby principals have no choice over the individuals who teach in their schools. Yet, according to a blockbuster report by The New Teacher Project, fully 40 percent of vacancies (in five urban districts studied) were filled by veteran teachers over whom the principal had little or no say in hiring. Principals adjust to this perverse set-up, both by hiding their vacancies from the central office and by encouraging poor performers to exit for other schools (where administrators will have no choice but to take them). So who is defending this system? Reg Weaver of the NEA, of course, who tells the L.A. Times: "I think it's another smoke screen to blame union rules for our society's lack of commitment to children.... This diversion gets us away from the responsibility we have collectively to make sure that no children are left behind." When we know that teachers are among the most important factors in raising student achievement, one can hardly call this problem a "diversion."
"Report Says Teacher Union Contracts Are Holding Schools Back," by Duke Helfand, Los Angeles Times, November 17, 2005
Margaret Spellings's November 18th announcement of an NCLB "growth model" pilot project capped a busy year of new-found flexibility for the Secretary and her policy minions. Since her confirmation, the Department of Education has made a number of adjustments to the No Child Left Behind Act. They can be grouped into three categories: those that seek to correct fundamental flaws with the law, those that attempt to use the carrot of flexibility to coerce states into stronger cooperation, and those that represent pure political payback.
April's decision to exempt a greater proportion of special ed students from regular testing, as well as the new "growth-model pilot," surely fit into the first category. Nobody thinks kids with significant cognitive disabilities should be expected to reach the same standards as everybody else; it's hard if not impossible to find anyone in the Administration or on Capitol Hill who claims credit for the idea of including special education students as an NCLB subgroup in the first place. Once it was embedded in the law, however, special ed advocates seized on its potential, making it impossible to remove. Spellings - and Secretary Paige before her - have been carefully walking federal policy back from this brink since the Act's ink was dry.
The need for some sort of growth or value-added achievement gauge within the NCLB accountability apparatus has also been clear since day one. Virtually everyone - from the American Federation of Teachers to the National Association of Public Charter Schools - agrees that it's unfair to penalize schools that help their students make incredible progress (like two years of gains in just 10 months) but still don't reach the "proficient" bar because the kids were so far behind when they started. That's why many charter schools (and traditional middle schools) find themselves on the "needs improvement" list. At the same time, proponents of the law, like the Education Trust and the Citizens Commission on Civil Rights (CCCR), rightfully worry that a simple growth model that merely demands a year of gain for every year in school won't push the nation to close our shameful achievement gaps or get all kids to proficient before the next millennium. Spellings threaded this needle by stating that she will only approve state accountability systems that aim to reach universal proficiency by 2014 - which in practice means that low-performing schools and low-performing students will have to show dramatic gains between now and then. (We'll leave for another day the vexed question of whether universal proficiency nine years hence is itself a realizable goal. Nobody doubts that it's a worthy one.)
This autumn's decisions to extend the "highly qualified teachers" (HQT) deadline and to allow four Virginia districts to offer students tutoring before offering them public school choice fit into the second category. In both of these policy domains, the department faced a dilemma: states and districts were implementing the letter but not the spirit of the law. States created definitions of "highly qualified teachers" that were a joke, yet arguably legal, while districts alerted parents to their choice options by sending jargon-filled letters but nothing else. The federal government doesn't handle such gray areas well. Spellings's solution was to offer new flexibility - an extra year to meet the HQT requirements, the right to flip-flop choice and supplemental services - in return for better performance, such as sending effective teachers to needy schools and aggressively reaching out to parents.
Unfortunately, all of these reasonable approaches are clouded by decisions that can only be described as political. Her approval of Florida's plan to exclude the performance of subgroups - such as African-Americans - from many schools' adequate yearly progress determination (see here) is the most egregious state-specific example that's come to light. Her change of heart on failing districts being eligible to be tutoring providers can best be viewed as a reward to the Council of Great City Schools in recognition of its pro-NCLB stance. And this summer's handling of state requests for additional flexibility utterly lacked transparency, creating a sense of distrust and favoritism.
This might explain why advocates for NCLB, including Ed Trust and CCCR, cast a skeptical gaze upon each announcement of flexibility. Spellings & Co. have an opportunity to demonstrate over the next several months that they understand that the process of policymaking - especially its transparency and openness - can be as important as the policies themselves, and that getting both right is what truly makes for good politics.
Four years ago, when Congress enacted NCLB, it showed uncommon wisdom in requiring all states to participate in the National Assessment of Educational Progress (NAEP). The reasoning: the national test would serve as a benchmark of the states' own reports of progress, and the resulting sunshine would potentially buffer state leaders from the inevitable political pressure to lower their own standards. Gadfly noted in October, following the release of the 2005 NAEP results in reading and math, that there was good reason to believe state exams showing huge achievement gains were in fact suspect. (See here and here.) The New York Times also took note, and it published a trenchant piece illustrating the cavernous gap between states' definitions of "proficiency" and NAEP's. In Tennessee, for example, 87 percent of the state's 8th-grade students received "proficient" scores on the state exam, but just 21 percent of eight graders who took the NAEP math exam reached the same level. That trend is also reflected, sadly, in the just-released Trial Urban District Assessment, a special project of NAEP (see here and here). Just as with the national scores released last month, urban district scores are basically flat. For example, the percentage of fourth-grade New York City students at the proficient level jumped seven points from 2003 to 2005 on the state assessment, but didn't budge on the NAEP. The value of a single standard and single assessment instrument is clear, and we have Congress to thank for showing us the way.
"Students Ace State Tests, but Earn D's From U.S.," by Sam Dillon, New York Times, November 28, 2005
It's been an eventful two weeks for education in the nation's courtrooms. A lawsuit organized by the National Education Association - that argued that NCLB improperly forced school districts to spend their own funds to implement the law - was dismissed by a federal judge in Michigan. The judge ruled that if "Congress had intended to prohibit unfunded mandates," it would have explicitly "stated that the federal government will reimburse the states for all costs they incur in complying with" the law. The NEA will appeal the ruling. The New York Times editorial board celebrated the decision, saying the suit was not about funding, but was rather a "transparent attempt by the National Education Association, the nation's largest teachers' union, to sabotage the No Child Left Behind education act." The paper asked, "Why does it [the NEA] put so much emphasis on the teachers? What about the children whose lives are cast into permanent shadow when they have to attend dismal, nonperforming schools?" Good question. The Wall Street Journal focused on another recent judicial event wherein the Texas Supreme Court struck down a statewide property tax used to fund public schools and rejected the argument that Texas wasn't spending enough on education. The court declared, in what the editorial heralded as a momentous judicial first, that "more money does not guarantee better schools or more educated students," and it shifted the debate away from dollars and toward student achievement. The Journal's take on the case's repercussions: "Better send the paramedics to check for heart failure at National Education Association headquarters." The NEA gets hammered from the left and the right - ouch.
"A Victory for Education," New York Times, November 29, 2005
"Texas School Lesson," Wall Street Journal, November 29, 2005 (subscription required)
"Schools lose lawsuit, fear money crisis," by David Ashenfelter, Detroit Free Press, November 24, 2005
"NEA to appeal 'unfunded mandate' claim ruling," by Stephen Sawchuk, Education Daily, November 29, 2005 (subscription required)
National Association of State Boards of EducationOctober 2005
Henry I. Braun, Educational Testing Service September 2005
Together, these booklets offer useful insights into value-added testing models - how they work, their strengths and weaknesses, and issues to consider before implementation. NASBE delves into the models' practical tradeoffs, but leaves largely untouched the question of how they actually work. ETS fills this void and offers descriptions of the statistics and processes used to determine which schools and teachers are "adding value" to their students' scores each year. Understanding value-added models is no easy task. NASBE notes that Tennessee's much-discussed program "involves the solving of thousands of equations iteratively," and uses software which "enables a massive multivariate, longitudinal analysis using all achievement data for each student no matter how sparse or complete." This complexity may be value-added analyses' greatest challenge: Will the public accept high stakes accountability based on metrics they must simply trust to be reliable? Perhaps an equal challenge is that the nature of these systems - which can account for "'external' factors such as socioeconomic status, race, parents' educational levels, or innate ability" in assessing a school - might hurt NCLB's no-excuses mentality. As a measurement tool, it may be appropriate to account for students' backgrounds, but as policy it could send a dangerous signal. (William Sanders, father of the Tennessee model, happens to argue against using student characteristics, because while they correlate with current scores, in his view "the correlations... with gains are essentially zero.") States must ask themselves whether models that account for demographics, or those that relate performance to absolute standards (which "growth models" must do under NCLB), better meet their particular needs. Anyone using a value-added system should reflect on both of those policy issues and decide how best to handle them, especially if they want to win approval from Education Secretary Spellings to use a "growth model" as part of their NCLB accountability systems (see here). It's a complex topic, and these booklets are certainly helpful. You can find them online here and here.
United States Government Accountability Office
November 2005
The District of Columbia has a larger percentage of its students - over 20 percent - in charter schools than any state (although in urban districts, it trails Dayton for highest percentage of charter students). Their schools are approved and overseen by two authorizers - the D.C. Board of Education (BOE), which also runs the city's traditional public schools, and the D.C. Public Charter School Board (PCSB). How well are these two authorizers doing their jobs, and in what areas can they improve? That's what this report, required by the 2005 D.C. Appropriations Act, sets out to answer. This informative study neatly examines the authorizers' resources (money and staff) and how they are used, oversight practices and response to school closures, and use of services available to them. To accomplish this, GAO researchers analyzed the authorizers' budgets, pored over documents and monitoring reports related to school closures, and conducted interviews. The results? Both authorizers are doing a fairly good job, but, especially for the BOE, there's lots of room for improvement. (BOE has been criticized by a recent Progressive Policy Institute report as well.) For one, the BOE gives the same level of oversight to all its schools. It would do far better to focus its energy and limited resources on struggling charters where students aren't achieving in the classroom or where internal management isn't up to par. Also, the BOE did not always review monitoring data "in time to react effectively and in a timely manner" to potential problems brewing in its charters. But most instructive is the report's finding that neither the BOE nor the PCSB has a process in place to guide schools, students, and families through a charter school closing. The study laments the possibility that, without such a process, student records may be lost, student transfers become needlessly complicated, and the authorizers do not learn from or build upon the experiences of past closures. The report is informative, and it provides valuable information, not only for D.C. charters, but for charter authorizers in other cities, too. The report is available here.
