Are states dutifully reporting the fraction of students who are on track for college or career? According to a new report from Achieve, a nonprofit organization that assists states in education reform efforts, the answer is no—and Ohio has been one of the worst offenders.
The report documents the discrepancies between proficiency rates on state tests versus the National Assessment for Educational Progress (NAEP). Achieve’s analysis finds that most states “continue to mislead the public on whether students are proficient” by reporting proficiency rates on state assessments that are significantly higher than those on NAEP. This is a serious problem since NAEP—commonly referred to as the Nation’s Report Card—has long been considered the gold standard of measuring student achievement and, more recently, college preparedness. As Fordham’s Mike Petrilli and Chester Finn explained a few weeks ago, there’s now good evidence that NAEP’s proficiency level in reading is particularly predictive of whether students are ready to succeed in college without taking remedial courses.
Ohio’s longstanding definition of proficiency, on the other hand, is predictive of nothing, as far as we can tell. It certainly doesn’t indicate that a student is on track for college. But that’s surely what parents believe when they receive a test score report saying that their children passed at the proficient level.
It’s also worth noting that this isn’t the first time questions have been raised about whether states are misleading parents and the public about student proficiency—back in 2007, Fordham took the questions head-on in our study The Proficiency Illusion.
Achieve’s report notes over half of states’ discrepancies in state versus 2013 NAEP results are more than thirty percentage points. Ohio is one such state. In 2013–14, Ohio’s state tests scores differed by at least thirty percentage points or more in each of NAEP’s main test subjects: fourth-grade reading and math, and eighth grade reading and math. In fact, in fourth-grade reading, the difference between NAEP and Ohio test scores is a whopping forty-nine points. Such large score discrepancies support Achieve’s troubling analysis that many states—including Ohio—are classifying too many students as proficient when they actually aren’t anywhere close to being on track for entering college without remediation.
Fortunately, several states are working hard to decrease what Achieve has dubbed the “honesty gap.” The states that have decreased discrepancies between NAEP and their state test scores appear to have three strategies in common: raising academic content standards, utilizing higher-quality tests, and raising the cut score for proficiency.
For example, many of the states that have raised student achievement while simultaneously lowering NAEP/state test discrepancies are early adopters of Common Core and Common Core-aligned assessments. Kentucky is a prime example. As the first state to adopt Common Core back in 2010, Kentucky has been faithfully implementing the standards ever since. The Bluegrass State’s energetic implementation efforts have led to a significant rise in student achievement, but they’ve also led to smaller discrepancies between state tests and NAEP. In each of the previously mentioned test subjects, Kentucky’s discrepancies are half that of Ohio’s. In fourth-grade math, the discrepancy is as little as eight percentage points, and in eighth-grade reading, Kentucky is one of the five states with the smallest discrepancies (along with Wisconsin, New York, Utah, and Tennessee). Kentucky also offers a good example of transitioning to a more difficult test. After initially signing on to PARCC and Smarter Balanced, Kentucky left both consortia and instead chose to develop its own Common Core-aligned test. These new tests were harder, and Kentucky initially saw a drop in proficiency rates. But those rates are now improving (along with a slew of other measures), and as a result, the state is seeing a rise in proficiency that truly reflects student learning.
New York offers a similar example. While Common Core implementation and a move to higher-quality tests has been more controversial in New York than in Kentucky, the Empire State has benefited from both. Since 2010, when New York first adopted Common Core and its aligned assessments, the difference between its state test scores and NAEP scores has narrowed significantly. But narrowing the honesty gap in New York isn’t merely the result of high standards and rigorous tests. In fact, it’s largely due to raising cut scores. Back in 2010, New York experienced firsthand just how hard it is to tell the truth; cut scores were raised, the number of students who were deemed proficient dropped, and parents and the public were outraged. But after the initial frustration over lower proficiency scores, New York settled in and got to work—and it paid off. In each of the four aforementioned test subjects, New York has one of the lowest discrepancies between state tests scores and NAEP. In fact, New York even has a positive difference in fourth-grade reading and eighth-grade reading and math; in other words, New York has proficiency requirements that are more rigorous than NAEP.
Telling the truth can be scary, particularly when the truth is predated by a history of reporting inflated proficiency rates. For policymakers, it can be tempting to keep cut scores, standards, and test quality low in order to avoid the hard truth that too many students aren’t on track for college or a well-paying career. But students, families, and the public deserve to know the truth about student achievement, even if it’s not pretty. Only after confronting the truth can we move on in the right direction. States with large discrepancies between state test scores and NAEP should beware of how uncomfortable the future will be if they continue to indulge in the false comfort of the honesty gap. Thankfully, Ohio is finally on the brink of closing the honesty gap: PARCC is a more difficult test, and its proficiency cut scores are expected to be quite demanding. Governor Kasich and other Buckeye policymakers should be commended for maintaining their support during a difficult transition—and for their commitment to telling the truth to Ohio students and families.
