An Analysis of Some Unintended and Negative Consequences of High-Stakes Testing
Audrey Amrein and David BerlinerArizona State UniversityDecember 2002
Audrey Amrein and David BerlinerArizona State UniversityDecember 2002
Audrey Amrein and David Berliner
Arizona State University
December 2002
In last week's Gadfly, Jay Greene reported on the failings of a much-publicized study of high-stakes testing by Arizona State University's Audrey Amrein and David Berliner. [http://www.edexcellence.net/gadfly/issue.cfm?issue=6#412] It turns out this dynamic duo has done TWO related studies. The second one, published electronically by the Education Policy Studies Laboratory (and also paid for by teacher unions), reviews data from 16 states with high-stakes graduation exams, and, as you might expect, purports to find all sorts of dire things happening in them: more high school dropouts, more teacher dropouts, etc. This is a cooked-to-order "study" that isn't really worth your time but if you're a glutton for punishment - or just trying to monitor the efforts of the anti-testing crowd and their fellow-traveling Panglosses - you can find it online at http://www.greatlakescenter.org/pub/H-S%20Analysis%20final.pdf.
Sherman Dorn
Education Policy Analysis Archive
January 2003
In this 30-page study, "published" by Arizona State's "Education Policy Analysis Archive," Sherman Dorn of the University of South Florida's College of Education tries to use 20th Century history to predict the effects of high-stakes tests that must be passed to earn high-school diplomas. You may not be surprised to learn that the author thinks they will cause graduation rates to decline, dropouts to rise, and confusion to persist over the "social meaning of diplomas." Aaargh. See for yourself, if you must, at http://epaa.asu.edu/epaa/v11n1/.
Boston University School of Education
November 2002
Boston University's School of Education is reviving and remaking its 125-year-old education journal under the editorship of Richard Silberman. It seeks to play a role in education policy debates akin to the roles played by Foreign Policy, The Public Interest and The American Scholar in their respective fields - somewhere between an academic journal and a popular magazine. This debut issue is 117 pages long, not at all glamorous to look at, but has some pretty good stuff in it: a meditation on teaching by Thomas Cottle, a thoughtful commencement address by former education dean Edwin J. Delattre, a discussion of the tension between teacher creativity and accountability, an explanation of why Department of Defense schools do relatively well academically and what local systems might learn from them, a couple of perceptive book reviews, and more. There doesn't appear to be a website but you can request a subscription by writing Journal of Education, Boston University School of Education, 605 Commonwealth Ave., Boston, MA 02215.
Education Week, January 2003
This bulky annual data compilation cum policy analysis from Education Week is getting better. If you can find a shelf large enough for its 182 oversized (and un-foldable) pages, you'll want to retain and refer back to it from time to time. As everybody knows, this year's issue focuses on teachers, in particular on the "teacher gap" (the concentration of less able, less prepared and less experienced teachers in schools serving poor kids vs. the relatively stronger teaching workforce in middle class schools) and on state policies that may boost the supply of "highly qualified" teachers, which are supposed to be the only kind left in U.S. classrooms three years hence. (Besides state-level data, this year the editors surveyed 30 large school districts.) It's somewhat circumscribed in its policy imagination, given that it reports on what exists in American public education today rather than thinking anew about what might be done differently. You will not, therefore, find much discussion of (say) giving school principals sweeping authority over the employment and compensation of their teachers - nor will you find much effort to learn from innovations in charter and private schools. Despite those limitations, this report tallies an impressive array of efforts to solve a tangle of teacher-related problems. Mostly, though, it illustrates how far most states still have to go to get within striking distance of real solutions, and how intractable are a host of local problems (e.g. the sluggish, bureaucratic maze of teacher hiring and placement in urban school systems) that are almost immune to state-level policy manipulation.
Though the press hasn't said much on the topic, this edition of Quality Counts also updates sundry indicators of state policy, practice and performance that bear on achievement, standards and accountability, school climate, resources, etc. For the most part, it deploys the best available data to be found on these topics, including some from less conventional sources than before. Here, too, there's progress to be glimpsed but vast distances yet to be traversed. For example, this year just 19 states have in place the kinds of testing regime that No Child Left Behind will require of every state. Though nearly all states issue "school report cards," fewer than half "disaggregate" their achievement results according to the various demographic categories mandated in NCLB. Just 24 states publish the pass rates of their teacher-training programs (though all must report those rates to the federal government) and, across the 35 states that claim to identify low-performing teacher preparation programs, a grand total of just 59 such programs have as yet been fingered for this dubious distinction. And so on and on. The publication, in other words, is considerably superior to the performance that it reports. You probably already have a copy but, in case not, you can get one by surfing to http://www.edweek.org/sreports/qc03/
Tom Corcoran and Jolley Bruce Christman
Consortium for Policy Research in Education
November 2002
In this report, Corcoran and Christman examine the impact of the Annenberg Challenge in Philadelphia - and the Children Achieving reform it funded - nearly eight years after it first began. This enormous reform effort began in 1995, when Walter Annenberg awarded $50 million to Philadelphia (which was matched with $100 million in funds raised elsewhere) and then-superintendent David Hornbeck created an ambitious plan to turn around Philadelphia's abysmal public schools. The report reads like a case study in the problems associated with changing a vast and bureaucratic system. The reform began with promise - and the right ideals - as Philadelphia implemented standards, testing, and accountability and then decentralized decision-making to give schools flexibility in meeting these standards. Test scores improved modestly, but the visions of great change never materialized. Ultimately the reform efforts unraveled from both the bottom - where schools were ill-prepared for additional responsibility and teachers were given little time to adapt to the new standards - and the top - where politics and battles over funding disillusioned the community and ultimately led to Hornbeck's resignation. Perhaps the most universal lessons are that implementation is paramount, massive school reform is incredibly unpredictable, and continually cultivating support from teachers and the community is imperative. This report provides a conclusion to the unfinished stories told in previous reports, including a chapter from Fordham's own Can Philanthropy Fix our Schools: Appraising Walter Annenberg's $500 Million Gift to Public Education, in 2000 (see http://www.edexcellence.net/detail/news.cfm?news_id=41) and CPRE's Children Achieving: Philadelphia's Education Reform, A Second-year Evaluation, in 1998. The current report, along with CPRE's previous efforts, can be found by surfing to http://www.cpre.org/Research/Research_Project_Children_Achieving.htm.
WestEd, 2003
The California-based regional lab named WestEd was commissioned by the Los Angeles Alliance for Student Achievement to help rethink the structure and governance of public education in America's second largest city. The result is a surprisingly hard-hitting report on the many ways that LAUSD's current structure and governance interfere with needed improvements in the city's schools and an impressively bold set of recommendations for birthing a "new charter system" in Los Angeles. WestEd would do this via the California charter school law, both to create needed educational capacity and to demonstrate how public education can be delivered in very different ways. The report is coy about how large such a charter network should be - news reports mention 50,000 youngsters and 100 schools - but it's otherwise comprehensive and well thought out. Will California's fiscal crisis allow something this ambitious to be undertaken in the next few years? We should hope so. There's an original and promising model for urban-education reform visible in these pages. Perhaps less promising - albeit sensible enough - is the section of the report that lays out a quintet of policy and governance changes for the larger LAUSD system itself. You can download a copy (it's 52 pages long) at http://www.wested.org/online_pubs/LA-Alliance-Report.pdf.
Elementary school principals and safety experts say they're seeing more violence and aggression than ever among their youngest students, according to an article by Greg Toppo in USA Today. While federal statistics show only a small increase in the percentage of elementary school teachers who say they've been attacked, a number of states and districts report an alarming rise in very young students suspended for hitting classmates or teachers or committing other "crimes against persons." For more, see "School violence hits lower grades," by Greg Toppo, USA Today, January 12, 2003
Many have remarked upon the double standard that operates in American education when judging the regular school system versus proposed reforms in it. The school establishment insists that would-be reformers prove in advance that their change will work perfectly with no adverse side effects, while the regular school system gets away, seemingly forever, with working badly and producing much collateral damage. Reform ideas are thus held to a lofty standard, the unchanging system to a far lower one. (Indeed, a strength of the No Child Left Behind act is its requirement that ALL schools make their results public and face the accountability music.)
Recent weeks have brought to light another vivid example of double standards at play in U.S. education. I refer to the Washington Teachers Union scandal. It seems that the former leaders of the huge teachers union local in the nation's capital looted its treasury of some $2 million over a seven-year period. Yet I don't hear the public-school establishment demanding immediate corrective action in the specific case plus broad reforms designed to assure that nothing of the sort can ever happen again anywhere. No such demands are audible. Indeed, the silence on this front is deafening.
Now consider the response we always see when someone messes with the finances of, say, a charter school. This hasn't happened often but, when it has, outrage has rolled across the education landscape like a tsunami. We hear loud calls from teacher unions, school board associations, ed schools, etc. for urgent action to impose new rules and red tape on all charter schools, not just the malefactors. "If we don't rein in this entire category of activity," goes the cry, "great harm will be inflicted on America's children."
Why is the Washington Teachers Union case not similar? Yet in this instance there is no outrage. Sure, the FBI and U.S. Attorney are beavering away. Criminal indictments may follow. But where are the denunciations from America's leading public educators and their agents and political allies? Where is the cry for new protections to be imposed on all teachers and union locals? Why the silence?
You know why. The establishment protects its own. The last thing it wants is any basic rearranging of the ground rules. While it may not condone illegal behavior by individuals, it treats such things as regrettable aberrations in a basically sound set-up, not fundamental flaws in the arrangements themselves. When "outsiders" misbehave, however, the response is no mere wrist slap for the miscreants. It's a full-scale attack on the policy arrangements that let outsiders enter the arena in the first place. But not when the perps are union leaders!
So far as I can tell, the National Education Association - usually so trigger-happy - has been a silent bystander to this episode. So has nearly everyone else. Reported The Washington Post on Sunday: "Those who might have raised red flags did not speak out until the money had vanished: the WTU's parent, the American Federation of Teachers; the union's own three-member board of trustees; its 21-member executive board; its membership; and U.S. Labor Department regulators."
The AFT's internal controls and accountability systems turn out to be staggeringly weak - and yes, that would be the same national union that recently urged a moratorium on charter schools because they're insufficiently accountable and because those who monitor them are too lax. Talk about lax: The AFT ostensibly requires each of its locals to conduct a complete internal financial audit every two years, yet the Washington Teachers Union hadn't done one since 1995. Asked to comment, an AFT spokesman told reporters that the union "would consider tightening its oversight."
To be sure, the national union tipped off the FBI in the first place and secretary-treasurer Edward McElroy claims to be "outraged" by "alleged abuses." But imagine the AFT reaction if, say, a charter school or for-profit EMO went seven years without an audit, even if there were no evidence of malfeasance: the unions would not settle for mere voicing of outrage. They would demand legislative and regulatory redress. They would probably insist that some governmental authority be placed in charge of the auditing function, required to conduct these audits on a regular cycle and obliged to make public their results.
What happened inside the Washington Teachers Union was no momentary lapse. "According to an FBI affidavit," Education Week reports, "there is 'probable cause' to believe that the three officials embezzled money, committed tax, mail, and wire fraud, and laundered money" over a seven year period, even as office phone bills and rent went unpaid. All this within shouting distance of AFT national headquarters.
A forthcoming book by Peter Brimelow estimates U.S. total annual teachers' union dues payments at a staggering $1.25 billion. Yet I don't see anyone urging legislation to ensure that these huge sums are properly looked after, to safeguard teachers from the misdeeds of their leaders, or to require that the unions' books - and audit results - be exposed to the sunshine.
Meanwhile, the Washington Teachers Union has applied for a quarter-million dollar bank loan to replenish its coffers. The District's teachers will have to repay that loan - with interest. They will repay it from dues withheld via payroll deduction - i.e. by the school system - using funds supplied by taxpayers.
Where is the outrage over this squalid episode? And over the double standards that apply in American public education and among those who purport to lead it?
In the midst of depressing financial news elsewhere, Philadelphia district officials announced last week that belt-tightening will eliminate a $28.3 million deficit and, in fact, produce an estimated $2 million surplus by the end of the fiscal year. The savings have been produced by eliminating over 1,000 staff positions, reducing all nonschool nonpersonnel budgets by 5 percent, eliminating redundancies in the facilities program, and other measures, according to school CEO Paul Vallas. The district also announced that most of the seven outside managers overseeing Philadelphia public schools this year are asking to oversee more schools next year. After the state took control of the school system a year ago, district officials turned over the management of 45 low-performing schools to three for-profit firms, two universities, and two non-profit groups. Unfortunately, district officials will have to decide whether to give the outside managers additional schools (or take schools away) before the results are in from the district's testing program this spring.
"Officials: School district to save $51m this year," by Mensah Dean, Philly.com, January 10, 2003
"School managers seeking more work," by Susan Snyder, Philadelphia Inquirer, January 8, 2003
As states encounter major revenue shortfalls (due to the sagging economy) and impose freezes or cuts in aid to education, many schools and districts are facing tough decisions about how to allocate their shrinking budgets. California governor Gray Davis has proposed an across-the-board cut in all state programs, which could trigger teacher layoffs or force schools to shut down for fifteen days. In Oregon, many districts have already announced plans to shorten the academic year, a cost-cutting measure that has not been imposed since the Great Depression. In a number of districts, these financial woes are compounded by recently discovered errors in district budgets caused by financial mismanagement or simple bookkeeping mistakes. The midyear budget cuts come at a time when schools face a variety of new education expenses, some linked to the federal No Child Left Behind Act, others arising from voter initiatives and court decisions. They also come at a time when states, districts, and schools are under mounting pressure to produce results. Some say these simultaneous challenges may force state education agencies and large school districts to become more efficient; if so, that would be a welcome silver lining to the cloudy year ahead.
"States brace for tough new year," by Alan Richard and Joetta Sack, Education Week, January 8, 2003
"Budget errors leave schools feeling pinch," by Jeff Archer, Education Week, January 8, 2003
"Schools ending year early to cut costs," by Sam Dillon, The New York Times, January 12, 2003
Earlier this week, U.S. Secretary of Education Rod Paige accepted the resignation of Susan Neuman as assistant secretary for elementary and secondary education. While the office she directed is the organizational home of the No Child Left Behind Act, much of the implementation of that law has been orchestrated from elsewhere in the Department and White House. Before joining the Department, Neuman was a reading researcher at the University of Michigan. For details see "Secretary Paige Accepts Susan B. Neuman's Resignation," press release, U.S. Department of Education, January 14, 2003
This week, Marion Joseph will attend her last meeting as a member of the California state board of education, and a column by Peter Schrag makes clear how large a gap her departure will leave. It was Joseph, a liberal Democrat, who in 1994 sounded the alarm that millions of California schoolchildren were not learning to read. Since then, she has spoken out relentlessly on the importance of rigorous curricula and no-excuses standards for all California students. She will be sorely missed, especially at a time when pressure to roll back academic standards is mounting in the Golden State. "Marion Joseph's decade of education reform," by Peter Schrag, The Sacramento Bee, January 8, 2003
The Center for Education Reform this week released the latest edition of its ranking of the "strength" of the nation's 40 charter school laws. Arizona retains the top ranking this year as Minnesota moves into second place. New charter laws in Tennessee and Iowa garner a C-minus and F respectively. For more information, surf to www.edreform.com/charter_schools/laws/rankingintro.htm
Audrey Amrein and David Berliner
Arizona State University
December 2002
In last week's Gadfly, Jay Greene reported on the failings of a much-publicized study of high-stakes testing by Arizona State University's Audrey Amrein and David Berliner. [http://www.edexcellence.net/gadfly/issue.cfm?issue=6#412] It turns out this dynamic duo has done TWO related studies. The second one, published electronically by the Education Policy Studies Laboratory (and also paid for by teacher unions), reviews data from 16 states with high-stakes graduation exams, and, as you might expect, purports to find all sorts of dire things happening in them: more high school dropouts, more teacher dropouts, etc. This is a cooked-to-order "study" that isn't really worth your time but if you're a glutton for punishment - or just trying to monitor the efforts of the anti-testing crowd and their fellow-traveling Panglosses - you can find it online at http://www.greatlakescenter.org/pub/H-S%20Analysis%20final.pdf.
Boston University School of Education
November 2002
Boston University's School of Education is reviving and remaking its 125-year-old education journal under the editorship of Richard Silberman. It seeks to play a role in education policy debates akin to the roles played by Foreign Policy, The Public Interest and The American Scholar in their respective fields - somewhere between an academic journal and a popular magazine. This debut issue is 117 pages long, not at all glamorous to look at, but has some pretty good stuff in it: a meditation on teaching by Thomas Cottle, a thoughtful commencement address by former education dean Edwin J. Delattre, a discussion of the tension between teacher creativity and accountability, an explanation of why Department of Defense schools do relatively well academically and what local systems might learn from them, a couple of perceptive book reviews, and more. There doesn't appear to be a website but you can request a subscription by writing Journal of Education, Boston University School of Education, 605 Commonwealth Ave., Boston, MA 02215.
Education Week, January 2003
This bulky annual data compilation cum policy analysis from Education Week is getting better. If you can find a shelf large enough for its 182 oversized (and un-foldable) pages, you'll want to retain and refer back to it from time to time. As everybody knows, this year's issue focuses on teachers, in particular on the "teacher gap" (the concentration of less able, less prepared and less experienced teachers in schools serving poor kids vs. the relatively stronger teaching workforce in middle class schools) and on state policies that may boost the supply of "highly qualified" teachers, which are supposed to be the only kind left in U.S. classrooms three years hence. (Besides state-level data, this year the editors surveyed 30 large school districts.) It's somewhat circumscribed in its policy imagination, given that it reports on what exists in American public education today rather than thinking anew about what might be done differently. You will not, therefore, find much discussion of (say) giving school principals sweeping authority over the employment and compensation of their teachers - nor will you find much effort to learn from innovations in charter and private schools. Despite those limitations, this report tallies an impressive array of efforts to solve a tangle of teacher-related problems. Mostly, though, it illustrates how far most states still have to go to get within striking distance of real solutions, and how intractable are a host of local problems (e.g. the sluggish, bureaucratic maze of teacher hiring and placement in urban school systems) that are almost immune to state-level policy manipulation.
Though the press hasn't said much on the topic, this edition of Quality Counts also updates sundry indicators of state policy, practice and performance that bear on achievement, standards and accountability, school climate, resources, etc. For the most part, it deploys the best available data to be found on these topics, including some from less conventional sources than before. Here, too, there's progress to be glimpsed but vast distances yet to be traversed. For example, this year just 19 states have in place the kinds of testing regime that No Child Left Behind will require of every state. Though nearly all states issue "school report cards," fewer than half "disaggregate" their achievement results according to the various demographic categories mandated in NCLB. Just 24 states publish the pass rates of their teacher-training programs (though all must report those rates to the federal government) and, across the 35 states that claim to identify low-performing teacher preparation programs, a grand total of just 59 such programs have as yet been fingered for this dubious distinction. And so on and on. The publication, in other words, is considerably superior to the performance that it reports. You probably already have a copy but, in case not, you can get one by surfing to http://www.edweek.org/sreports/qc03/
Sherman Dorn
Education Policy Analysis Archive
January 2003
In this 30-page study, "published" by Arizona State's "Education Policy Analysis Archive," Sherman Dorn of the University of South Florida's College of Education tries to use 20th Century history to predict the effects of high-stakes tests that must be passed to earn high-school diplomas. You may not be surprised to learn that the author thinks they will cause graduation rates to decline, dropouts to rise, and confusion to persist over the "social meaning of diplomas." Aaargh. See for yourself, if you must, at http://epaa.asu.edu/epaa/v11n1/.
Tom Corcoran and Jolley Bruce Christman
Consortium for Policy Research in Education
November 2002
In this report, Corcoran and Christman examine the impact of the Annenberg Challenge in Philadelphia - and the Children Achieving reform it funded - nearly eight years after it first began. This enormous reform effort began in 1995, when Walter Annenberg awarded $50 million to Philadelphia (which was matched with $100 million in funds raised elsewhere) and then-superintendent David Hornbeck created an ambitious plan to turn around Philadelphia's abysmal public schools. The report reads like a case study in the problems associated with changing a vast and bureaucratic system. The reform began with promise - and the right ideals - as Philadelphia implemented standards, testing, and accountability and then decentralized decision-making to give schools flexibility in meeting these standards. Test scores improved modestly, but the visions of great change never materialized. Ultimately the reform efforts unraveled from both the bottom - where schools were ill-prepared for additional responsibility and teachers were given little time to adapt to the new standards - and the top - where politics and battles over funding disillusioned the community and ultimately led to Hornbeck's resignation. Perhaps the most universal lessons are that implementation is paramount, massive school reform is incredibly unpredictable, and continually cultivating support from teachers and the community is imperative. This report provides a conclusion to the unfinished stories told in previous reports, including a chapter from Fordham's own Can Philanthropy Fix our Schools: Appraising Walter Annenberg's $500 Million Gift to Public Education, in 2000 (see http://www.edexcellence.net/detail/news.cfm?news_id=41) and CPRE's Children Achieving: Philadelphia's Education Reform, A Second-year Evaluation, in 1998. The current report, along with CPRE's previous efforts, can be found by surfing to http://www.cpre.org/Research/Research_Project_Children_Achieving.htm.
WestEd, 2003
The California-based regional lab named WestEd was commissioned by the Los Angeles Alliance for Student Achievement to help rethink the structure and governance of public education in America's second largest city. The result is a surprisingly hard-hitting report on the many ways that LAUSD's current structure and governance interfere with needed improvements in the city's schools and an impressively bold set of recommendations for birthing a "new charter system" in Los Angeles. WestEd would do this via the California charter school law, both to create needed educational capacity and to demonstrate how public education can be delivered in very different ways. The report is coy about how large such a charter network should be - news reports mention 50,000 youngsters and 100 schools - but it's otherwise comprehensive and well thought out. Will California's fiscal crisis allow something this ambitious to be undertaken in the next few years? We should hope so. There's an original and promising model for urban-education reform visible in these pages. Perhaps less promising - albeit sensible enough - is the section of the report that lays out a quintet of policy and governance changes for the larger LAUSD system itself. You can download a copy (it's 52 pages long) at http://www.wested.org/online_pubs/LA-Alliance-Report.pdf.