In August, the Ohio Department (ODE) of Education and the State Board of Education (SBOE) released their five-year strategic plan for education. It included ten strategies aimed at helping the state meet a questionable goal that doesn’t ask much of students or schools. One of these strategies called for identifying “robust and diverse ways to measure performance.” Take a look:
Ohio needs to address challenges related to a reliance on standardized assessments in academic content areas, especially in high-stakes situations. Students should have multiple ways to demonstrate what they know and are able to do. The State Board of Education recognizes this point and is examining the use of alternative tools as validated, reliable methods to assess knowledge. Such tools might include student portfolios, capstone projects, presentations, or performance-based assessments.
Less than a month later, state board members are already debating a draft proposal for a new set of graduation requirements that includes alternatives to assessments. Students would be able to choose how to demonstrate their competency in English, math, and other subjects from a laundry list of options. One of these options is a non-test-reliant pathway called a Culminating Student Experience (CSE).
The draft proposal offers a relatively detailed look at what a CSE could be, though little guidance on what it must be. It is described as a “project or set of activities and experiences” that can be embedded into existing school programming and could include in-school and out-of-school activities. It’s up to students to choose which activities they’re going to complete. Provided examples of such activities include a research or portfolio project, a community service project or experience, a work-based learning experience, or the completion of a career-technical program or an in-demand credential. Despite these examples, the proposal notes that students and their advisors are free to choose “activities and experiences that are aligned to the particular student’s aspirations, interests, and future plans.”
But here’s the real kicker: Depending on the activities they choose, students could use a CSE to meet all of the state’s graduation requirements. For example, to fulfill the English requirement using a CSE, students must complete a writing demonstration and achieve a rating of “satisfactory” on a rubric. For math, it’s a mathematical competency/data analysis demonstration and another satisfactory rating on another rubric. For both the “well-rounded content” requirement (which includes subjects like social studies and science), and the leadership/reasoning and social-emotional-skills requirements, it’s nothing more than presenting a summary of the CSE to a review panel. Students who can complete each of these demonstrations satisfactorily (it’s unclear what that means because the rubrics don’t exist yet) will earn a diploma.
To be clear, the draft proposal is not a formal recommendation from SBOE and ODE. The document clearly states that it was created “for discussion purposes only,” and State Superintendent Paolo DeMaria told the board that details would still need to be worked out if the concept was approved. But any proposal deemed worthy of discussion by a group that’s been so highly critical of current graduation requirements deserves a close look. And these “details” are not small, trivial things that can be waved away as implementation concerns. In fact, stakeholders and advocates should be extremely concerned about this proposal moving forward.
For starters, the proposal does not require certain experiences or activities to be part of a CSE. Instead, in an attempt to give students the freedom to align their CSE with their aspirations, interests, and future plans, it offers blanket approval for pretty much anything: “The Culminating Student Experience could include any meaningful collection of activities and experiences” (emphasis mine). As long as a student can write about what they did, do some math related to it, and present it to a review panel, then they’re free to use it as a pathway to a diploma.
Wanting to give students the freedom to align school to their interests and goals is a laudable idea. A recent survey indicated that only half of students think that what they learn in school is relevant to the real world, and some students struggle to stay engaged in school when they don’t see it as pertinent to their lives. Schools should absolutely be working toward connecting academics to real life. But the state should not be granting diplomas based on an open-ended list of student-selected activities that it cannot prove will develop or accurately measure academic knowledge and skills.
Consider one of the few activities the proposal identifies as a possible CSE: a major community service project/experience, such as a significant leadership role in a student organization. Although such activities are undoubtedly meaningful for students, they aren’t reflective of academic mastery. Johnny may have learned a ton from his time leading a student organization, but writing an essay about his experience to meet ELA graduation requirements overlooks almost every single one of the state’s reading standards. Johnny will graduate without proving that he can make logical inferences, determine central themes, assess how a point of view shapes content and style, or compare and contrast two texts—skills that are absolutely vital in college and a vast majority of workplaces, not to mention typical adult experiences, like deciding which presidential candidate to vote for or reading a lease agreement.
There is no way for the state to ensure that every CSE activity a student chooses is high quality. Instead, policymakers will rely on teachers and administrators to do it—a decision that will heap even more onto the already full plates of folks on the ground. The sheer amount of time and manpower it will take to not only monitor student selections but also grade and evaluate all of the associated demonstrations in addition to daily classes boggles the mind. Professional development and training will cost the state millions.
But that’s not all. In a previous piece, I pointed out the drawbacks of using GPAs as a graduation requirement (which the proposal also advocates for). Many of the same issues with GPAs exist for CSEs: an enormous amount of pressure on the teachers tasked with grading; the potential of score inflation and gaming; and wildly different expectations between schools and districts, and even for different groups of students that make comparison and validity all but impossible. That last point—the impossibility of comparing between districts, schools, and subgroups of students—should terrify equity advocates. The state has a responsibility to ensure that all children are learning, and to do everything in its power to close the achievement gap. That’s not possible if we can’t compare the graduation rates of poor, minority, and underserved children to the results of their more affluent or white peers.
And then there are concerns about reliability. The proposal suggests that students could complete a portfolio project either as part of a CSE or to complete their well-rounded-content requirements. The state of Vermont implemented a similar portfolio assessment program back in the early 1990s. Research into Vermont’s work showed that “rater reliability,” or the extent of agreement between raters about the quality of a student’s work, was low, on average, in both math and writing. The authors of the report note that when ratings are unreliable, “scores tend to spread out more than they would with reliable scoring.” This results in too many students obtaining scores on the extreme ends of the spectrum, and too few students obtaining scores near the middle. In Vermont’s portfolio system, the resulting bias was “substantial enough to undermine reporting of statewide proportions of students reaching each score point.”
Other assessment alternatives proposed by the plan have similarly questionable evidence bases. In a review of the literature on high school capstone courses, authors found that, “while the idea of such courses has been around since the 1990s, such courses have not been widely implemented and virtually no research exists on their effectiveness.” Work-based-learning experiences are promising, but still in their infancy: The ODE-developed framework for how schools should issue high school credit to students completing work-based learning didn’t go into effect until this year. The state is still working on how to reliably measure work-based-learning participation and evaluate performance.
The upshot of all this is clear: The state board and ODE promised to examine alternatives to assessments to see if they were valid and reliable. But there was no examination. In fact, given the speed with which this new draft proposal came out, it seems that their minds were already made up. That stands in stark contrast to other states that are also considering test alternatives but are doing so carefully, via small-scale pilots and extended timelines, and with the worrisome concerns of equity and reliability front and center.
It should greatly concern parents, educators, and the general public that the alternatives proposed in the draft plan not only lack solid evidence of validity and reliability, but are also burdensome, subject to gaming, of questionable rigor, and completely devoid of the state’s reading standards. This is not to say that all alternative assessments are bad; they may very well be excellent assessment tools at the local level, and they could become an important complement to (though not a replacement of) state tests. Nor is the point to champion standardized tests as the be all and end all. Standardized tests don’t tell us everything we need to know about student potential, achievement, or ability. But they are objective, valid, reliable, comparable, and rigorous tools that gauge what students know and are able to do—something that can’t be said for the alternatives offered by the draft proposal.