In 2003, as part of a broader education reform package, the Commonwealth of Massachusetts began requiring high school students to pass Massachusetts Comprehensive Assessment System (MCAS) exams in math, English language arts, and science in order to earn a diploma. A timely new report examines the impacts of this momentous change in Bay State education policy. Why timely? Because a teachers-union-backed initiative on the November ballot threatens to permanently eliminate the testing requirement.
A team of analysts from the Annenberg Institute’s Educational Opportunity in Massachusetts initiative—a research-practice partnership with the Massachusetts Departments of Elementary and Secondary Education and Higher Education—examined the first twenty years of student cohorts required to pass MCAS exams to determine how the policy has impacted college enrollment, performance, and graduation, as well as labor market outcomes. College data come from both the National Student Clearinghouse and the Massachusetts Department of Higher Education, while labor market data come from the state unemployment insurance system. Limitations include the inability to follow high school or college graduates who leave the commonwealth for work and the fact that the latest testing cohort with a full suite of outcome data took their exams way back in 2012.
The main findings are a mixed bag that echo some larger education trends. First and most importantly, MCAS test scores accurately predict long-term outcomes. The higher the scores, the better the college and career outcomes. While there were analogous findings regarding high school course grades in the first several years of the data (that is, stronger course grades predicted better long-term outcomes), starting around 2011 the analysts noted that the share of students earning As and Bs began a substantial upward trend. Over that time, MCAS scores declined overall (even controlling for several increases in cut scores that were implemented), and the mismatch between course grades and test scores appears to have widened during the pandemic. The upshot is that MCAS scores have become an even better predictor of later success, versus course grades, than they used to be. The authors express concern that many parents and students are being misled about college and career readiness due to the ongoing inflationary pattern in course grades continues. If MCAS is wholly abandoned, their concerns will become a near certainty.
Next, the analysts zero in on students just around the passing margin, trying to get at the causal impact of passage versus non-passage of similar students. They find that those who barely pass the tests on their first attempt have better post-secondary outcomes than those who fall just below the cut scores. (Most students did pass the assessments on their first attempt.) However, nearly all of those who failed were English learners (ELs), students with disabilities, or students with inadequate course preparation (such as those who were taking below-grade-level courses at the time of MCAS testing or who earned very low grades in their on-grade-level courses).
But most students who initially failed went on to pass the tests upon retake. Appeals of non-passing scores after multiple retakes were also generally successful. (An appeal requires students to “demonstrate through their coursework that they have the knowledge and skills to meet or exceed the passing standard.”) The small cohort of students with successful appeals—almost entirely ELs or students with disabilities—earn a “certificate of attainment” rather than a diploma. Those students generally fare worse in outcomes than their peers who pass the MCAS tests at any point during high school. However, the MCAS scores of ELs underpredicted their later earnings relative to other students—whether they ultimately passed or not—which is a notable exception to the general results. The analysts speculate that their test scores reflect a lack of proficiency with English rather than their overall academic ability.
Finally, most high schools that boost test scores also improve their students’ long-term outcomes. However, some schools, including some of the Bay State’s highly-regarded career and technical education schools, improve students’ long-term earnings without raising test scores.
The report’s authors offer no recommendations, and the report was completed before the ballot initiative was approved. If MCAS survives and continues as the central arbiter of graduation status, their findings suggest that Massachusetts schools must better prepare English learners and students with disabilities to take and pass the tests. Inflated course grades and non-standard portfolio projects are not fitting substitutes for rigorous exams, which this report shows are an accurate predictor of long-term success. If the effort to remove MCAS as the pathway to a diploma is successful, however, those may be the only options remaining.
SOURCE: John P. Papay et al., “The MCAS as a Graduation Requirement: Findings from a Research-Practice Partnership,” Annenberg Institute at Brown University (July 2024).
