 Tests in use from Kindergarten through eleventh grade need to have cut scores that denote true readiness for the next grade and that culminate to "college and career readiness." Photo by albertogp123 |
As the U.S. education world eagerly awaits more information about the new assessments that two consortia of states are developing to accompany the Common Core standards, dozens of perplexing and important questions have arisen: Once the federal grants run out, how will these activities be financed? What will it cost states and districts to participate? Who will govern and manage these massive testing programs? What about the technology infrastructure? The list goes on.
The assessment questions that weigh most heavily on my mind these days, however, involve “cut scores.” For if the Common Core is truly intended to yield high school graduates who are college and career ready, its assessments must be calibrated to passing scores that colleges and employers will accept as the levels of skill and knowledge that their entrants truly need to possess. Adequately equipping young people cannot wait ‘til twelfth grade, nor can the assessment sequence. The tests in use from Kindergarten through eleventh grade need to have passing scores that denote true readiness for the next grade and that cumulate to “college and career readiness.”
That’s a daunting challenge for any test maker, but it’s further complicated by widespread fears of soaring failure rates and their political consequences, as well as by Arne Duncan’s stipulation (in the federal grants that underwrite the assessment-development process) that the states belonging to each consortium must reach consensus on those passing scores (in government jargon, “common achievement standards”). All this means, in effect, that Oregon and West Virginia (both members of the “Smarter Balanced” consortium) must agree on “how good is good enough” for their students, as must Arkansas and Massachusetts (both members of PARCC). Can that really happen?
The angst is palpable among state officials—especially the elected kind—over the threat of soaring failure rates, and not just among the poor and dispossessed. We already know from national assessment data that about half of eighth graders with college-educated parents fail to clear the “proficient” bar on NAEP. If (as mounting evidence suggests) “NAEP proficient” is roughly equivalent to “college ready,” and if the new assessments hew to that level of rigor and honesty, many millions of American youngsters will be found unready—and millions more will learn that they’re not on track toward readiness. Such a cold shower should benefit the nation over the long haul, but in the short run, it’s going to feel icy indeed.
Yet that’s only the start. Here are some other perplexing challenges in this realm:
- When will the chilly water hit? PARCC says it doesn’t intend to do any level-setting until after scores come in from the first test-administration in 2015, a classic example of psychometric considerations overriding real-world political considerations. How can states and districts possibly prep their students, their educators, and their publics for new standards (and heightened risk of failure) if nobody knows in advance what sort of performance will be deemed passable? The folks at Smarter Balanced say they’ll set cut scores in advance of that first administration but “confirm” them afterward. What exactly does that mean?
- A single cut-score or several? NCLB says states must set at least three cut scores on their assessments, but the Education Department’s RFP for the new Common Core assessments makes no such demand. It refers only to “college and career readiness.”
I had the honor of chairing the National Assessment Governing Board (NAGB) twenty years back when it was struggling to set “achievement levels” for NAEP. Al Shanker (and others) warned that we’d be wrong to establish a single level. It would either be too high, he cautioned, in which case far too few kids would reach it, or too low, meaning it would amount to little improvement over state-set “minimum-competency” levels. In the end, NAGB set three achievement levels, familiar today as “basic,” “proficient,” and “advanced.” We declared “proficient” to be the level that every student ought to reach—but then, as now, fewer than two in five actually achieved that. “Basic” was intended to be a solid marker along the road to proficient and “advanced” was meant to represent “world class”. Will the Common Core assessments do something similar?
- Will these cut scores be static or will they rise over time? Some states (Texas comes to mind) have had good results by slowly elevating the cut scores on their own assessments. Doing something of the sort with the new Common Core assessments would ease the political backlash. On the other hand, the promise of “college and career readiness” would then remain hollow for years to come.
- Speaking of which, who is doing what to ensure that colleges and employers will actually accept these standards—and cut scores—as evidence of readiness? And how exactly is “readiness” being construed? (The federal procurement was nebulous and, while the Common Core standards themselves are substantively very ambitious, the “cut scores” on their assessments are not obliged to be.) For college-bound youngsters, I see “readiness” as the ability to enter credit-bearing courses in the appropriate subjects on non-selective-admission campuses—i.e., to avoid remediation. But I’ve no idea how to define “career readiness.” Who does?
- What happens in states (about half of them) that already have statewide graduation tests (e.g., Massachusetts Comprehensive Assessment System and the Ohio Graduation Test) with minimum passing scores? These have typically been pegged to levels of academic prowess around old-style ninth- or tenth-grade curricula and have been required (of almost all students) as prerequisite to getting a diploma. Are said states supposed to replace these with the new assessments pegged (presumably) to much tougher demands? Administer both? What about a two-tiered diploma system, at least for awhile? Set the new cut scores high to denote true “readiness,” get the colleges to accept them as such, and confer a “readiness diploma” on youngsters who meet that standard. But for some period of time also continue to hand out “regular” diplomas to those who meet the state’s existing graduation requirements, typically a mix of course completions, test scores, and the like. Phase out the latter if possible. (In the old days, New York State did something like this, distinguishing between an ordinary diploma and a “Regents’ Diploma.”)
- As the “credit-recovery” industry grows, often abetted by online delivery of make-up courses and such, how will such arrangements intersect with the new standards, assessments, and cut scores? (The same question may fairly be asked of the G.E.D.) It may turn out that stiffer academic expectations drive more kids to flunk, drop out, etc., and that credit recovery becomes more important than ever. But will that path turn out to be a short cut to a meaningless diploma or an honorable avenue to meet the higher standards and demonstrate one’s “readiness” for job and college?
- If the ACT and College Board folks build the Common Core into their widely used college-admissions tests, as seems likely, will these new consortium-based assessments even be needed at the high school level? For that matter, what would a “cut score” on the SAT or ACT look like?
I hope some smart people are figuring all this out—but if they are, the answers haven’t yet reached my eyes or ears. Time is getting short.
