Editor’s Note: The Thomas B. Fordham Institute occasionally publishes guest commentaries on its blogs. The views expressed by guest authors do not necessarily reflect those of Fordham.
Editor’s Note: The Thomas B. Fordham Institute occasionally publishes guest commentaries on its blogs. The views expressed by guest authors do not necessarily reflect those of Fordham.
Earlier this month, the federal government released the latest batch of national test score data tracking changes in the achievement of nine-year-olds since the beginning of the Covid-19 pandemic. The announcement resulted in ominous headlines in nearly every major national news outlet, with the Washington Post reporting that “American students’ test scores plunge to levels unseen for decades” and the New York Times noting darkly that the pandemic “Erased Two Decades of Progress in Math and Reading.”
The trends documented in these articles are many ways old news, both because the national data merely confirmed what a wave of state assessments and low-stakes diagnostics had already shown a year earlier and because the new figures could not speak to the most policy-relevant questions: Has student achievement begun to recover from the initial pandemic disruptions? And if so, at what rate, in which subjects, and among which student groups?
Fortunately, my new report examining the results from Ohio’s spring state exams answers exactly these questions. The analysis focuses on student achievement in both spring 2021 and spring 2022, comparing each cohort to performance on the same exams among otherwise similar students prior to the pandemic. Overall, it shows both the impressive progress Ohio students made last year, as well as the considerable learning shortfalls that still remain, especially in the oldest grades.
On the positive front, the spring assessments confirmed the encouraging news revealed by the fall round of exams, which showed that English language arts (ELA) achievement among third graders had charted an impressive rebound a year after plunging in the early months of the pandemic. The spring ELA exams, which included many more grades, showed similar gains among most tested elementary and middle school grades. Although students remain between one-third and two-thirds of a year behind compared to pre-pandemic performance of similar peers, they are on track to make up the lost ground within the next two to three years—if the current pace of recovery can be sustained. (That is obviously a big “if.”)
The math results are far less impressive, however. The initial achievement decline in this subject was much larger than in ELA, and the recovery over the past year has been far more muted. Overall, students remain between one-half and one whole year behind in math, corresponding to a proficiency rate that is between 10 to 15 percentage points lower than prior to the pandemic in most tested grades.
In addition to contrasting the exam scores of current students to earlier cohorts, the new report also examines one-year learning gains between spring 2021 and spring 2022 relative to comparable pre-pandemic growth. This allows me to directly quantify the amount of learning “acceleration” taking place in Ohio schools over the past year. The analysis confirms the trends found in the aggregate achievement data, showing that test scores in ELA grew substantially faster than in the years prior to the pandemic (with an important caveat, discussed further below). These gains were broad-based. Unfortunately, there was no consistent evidence that student groups who suffered the most pronounced initial declines in the first year of the pandemic—particularly Black students, economically disadvantaged students, and those attending districts that remained remote the longest—have recovered any faster than their peers. In other words, the achievement gaps that expanded early in the pandemic largely remained last year.
In math, one-year growth was only modestly larger than normal in elementary grades but was also modestly smaller than usual for most older students. Indeed, the data for the oldest students is most distressing. Overall, eighth graders showed no evidence of learning acceleration—meaning the initial learning shortfalls among these students first recorded in spring 2021 did not narrow over the subsequent year. And the test scores suggest that tenth graders may have fallen even further behind, at least in ELA. Since these students have only a few years left before graduation, prioritizing their academic recovery should be at the very top of the agenda for Ohio educational leaders and public officials.
Frustratingly, the assessment results cannot speak to which strategies and investments that districts have pursued over the past year are working and which should be ditched, because there is no consistent data collection being done on these efforts, aside from tracking district spending of federal Covid aid. (Such limited tracking is not particularly useful since money is and has always been fungible.) However, the spring assessments do offer one important—and concerning—hint.
Specifically, most third-grade students take the same ELA exam twice during the academic year, in both the fall and the spring. And although spring 2022 third-grade reading scores showed significant improvement from spring 2021, the exams also revealed no evidence that these students had learned more during the academic year compared to pre-pandemic cohorts. In fact, the fall-to-spring growth rate among third graders last year was approximately 15 percent lower than typical before the pandemic, perhaps due to the disruptions caused by the Omicron wave in January or the widespread bus driver shortages and transportation woes. This suggests that much of the observed gains recorded in the spring were due to expanded programming and services from summer 2021, not improved instruction or tutoring services delivered during the school year.
The message from the spring’s assessments should be loud and clear: Despite some recent gains, Ohio students remain behind academically, especially in math. With Ohio districts still sitting on hundreds of millions in unspent federal aid, it is more important than ever that we understand which interventions are having the greatest impact in moving the needle on student academic recovery, and that policymakers prioritize and target their spending on students and subject that need help the most. The future of Ohio’s students—especially our oldest students—depends on it.
Vladimir Kogan is Associate Professor in The Ohio State University’s Department of Political Science and (by courtesy) the John Glenn College of Public Affairs. The opinions and recommendations presented in this editorial are those of the author and do not necessarily represent policy positions or views of the John Glenn College of Public Affairs, the Department of Political Science, or The Ohio State University.
Unless there’s a political or ideological controversy, curricular decisions in schools and districts rarely make headlines. That’s too bad because these choices are immensely important. Research shows that curriculum plays a critical role in student achievement, and that upgrading curricular materials can be a cost-effective intervention. That’s particularly the case with early reading, where the evidence base is especially strong. State-led efforts have also proved promising: Louisiana’s decision to incentivize schools to adopt and implement high-quality curricula was paying dividends prior to the pandemic, with students showing improvements on state tests, the ACT, and AP exams. Other states, like Tennessee, have followed suit.
Obviously, curriculum reform isn’t a silver bullet. But helping districts and schools make wise curricular choices can pay off in the long run without being prohibitively expensive. Over the last several years, Ohio leaders have implemented several initiatives aimed at making it likelier that schools adopt high-quality curricula and instructional materials. Let’s take a look at two of those efforts.
The Ohio Curriculum Supports Guide
The Ohio Curriculum Support Guide was developed by the Ohio Department of Education in partnership with Instruction Partners, an education nonprofit that works with teachers to “support great teaching and accelerate student learning.” As you’d expect from a nonprofit aimed at helping teachers, the guide is specifically designed for educators who are responsible for selecting curricula and materials for their school or district.
The bulk of the guide is based around a framework of “leadership actions” that educators should take as they work through the curriculum adoption process. The framework is divided into three phases—selecting materials, preparing to launch, and teaching and learning—and each phase contains several “key actions.” For example, phase one includes the following:
Each key action comprises an overarching goal, an explanation about why it matters, and several steps that team members must take to complete it. For example, the second action listed above—establish the vision—includes two steps: train the selection team and review committee, and articulate the vision of instruction and core beliefs. Each of these steps is broken down even further into guiding questions, notes, and resources.
The framework is a thorough resource designed to take the mystery out of selecting high-quality curricula. But the details don’t stop with the framework. The guide also offers a sample implementation timeline that outlines how to progress through the framework, a workbook designed to help educators and administrators work through the process as a team, a diagnostic tool aimed at helping leaders determine the best place in the framework to begin, and solutions to common challenges.
INFOhio
To understand INFOhio, it’s important to understand Information Technology Centers (ITCs). There are eighteen of these nonprofit public agencies in Ohio, and they are charged with providing technology and shared services to the state’s schools. These services include internet connectivity, fiscal systems, student information and EMIS services, and INFOhio.
The overarching vision for INFOhio is to ensure that every student in the state has “equal access to high quality digital resources.” This is accomplished by offering instructional and technical support to schools (mostly through professional development and training for teachers) and by providing educators and families with access to age-appropriate content and instructional materials. Whereas the Ohio Curriculum Support Guide helps educators select curricula for their school, INFOhio offers a digital and searchable library of materials that can be used for free by teachers, school staff, parents, and students.
INFOhio also offers curriculum and instructional reviews in several forms. This is a hugely important development, as these reviews can provide district and school staff with objective, third-party evaluations that are crucial for wise decision-making. INFOhio’s review resources include the Ohio Evidence-Based Clearinghouse, which helps school leaders identify evidence-based strategies for school improvement, as well as instructional reviews that evaluate submissions to Open Space, Ohio’s digital platform for open access and open educational resources. It also includes Ohio Materials Matter Reviews, a site that allows educators to search for high-quality instructional materials that have been reviewed by EdReports, a national organization that boasts a network of more than 300 experienced reviewers from forty-six states.
***
To outsiders, the development of the Ohio Curriculum Support Guide and the advent of INFOhio might not seem like much. But for teachers and administrators charged with selecting a new curriculum for their school, the support guide offers step-by-step guidance that can ease a complicated transition. For parents and teachers looking for supplemental materials, INFOhio offers a plethora of free digital resources. And for schools looking to upgrade their curriculum to improve student achievement, INFOhio offers rigorous reviews on the many, many options that exist. These are critical resources that Ohio didn’t have before—and that’s a huge win.
In late August, the Ohio Department of Education (ODE) unveiled its FYs 2024–25 budget priorities to a state board of education committee. The vast majority of ODE’s budget simply passes through to Ohio schools via funding formulae, and the rest mainly supports the agency’s administrative work. As such, there’s usually not much to write home about when ODE releases its biennial budget request. But this year’s proposal includes some exciting initiatives, including several that would give literacy efforts a lift statewide. The ideas would, of course, need to be included in the state budget legislation, which will be introduced early next year, and passed by lawmakers next summer to become reality.
Following an encouraging national trend, the ODE blueprint emphasizes the science of reading. This term refers to instructional approaches that a large body of research has found to support strong reading. These include, notably, systematic and explicit phonics instruction and a knowledge-rich curriculum that aids reading comprehension. Though researchers have long understood the importance of such practices, schools have not always implemented them and many still rely on discredited literacy models.
The budget proposal would support the science of reading in three key ways:
In addition to the reading initiatives discussed above, ODE also notes a few other promising literacy initiatives that it plans to undertake. They include strengthening its parental resources and working with higher education to ensure that teacher preparation programs are providing instruction in the science of reading. The agency also proposes to continue several programs initiated in response to the pandemic, including funding for summer school and tutoring grants.
To its credit, ODE has given stronger credence to the science of reading through the statewide literacy plan it released in early 2020. Now the agency is asking to put some money where its mouth is by proposing new spending initiatives that support effective literacy practices. The state board—and ultimately the legislature—should approve ODE’s initiatives, as they’d be another step towards ensuring that all Ohio students have the foundational reading skills needed for lifelong success.
Helping students catch up from more than two years of school-closure-related learning loss will be an impossible task if they do not have regular access to grade-level work in their classrooms. While this may seem like a no-brainer, a new report from TNTP indicates that many of our neediest students may be missing out on this crucial opportunity.
TNTP partnered with ReadWorks, a free digital literacy resource used by more than 75,000 schools nationwide, to analyze how teachers were using the platform. Data in this analysis came from the 2018–19 to 2020–21 school years and included more than three million students in over 150,000 classrooms. ReadWorks offers K–12 teachers curated nonfiction passages they can assign to students as supplemental reading practice that can increase background knowledge and improve vocabulary across subjects. Each passage is accompanied by a text-dependent question set that provides practice in inferring, monitoring and clarifying, and questioning. Grade-level appropriateness is determined using qualitative and quantitative analyses of text complexity, and texts are aligned to Common Core State Standards. While the platform encourages teachers to assign grade-level passages to students, the choice is ultimately left up to teachers.
Overall, students working on the ReadWorks platform spent about one-third of their time engaging with below-grade-level texts and question sets, and teachers assigned 5 percentage points more below-grade-level content after pandemic disruptions than before. Students in schools serving more low-income students were assigned the most below-grade-level work. They spent about 65 percent more time on below-grade-level texts and question sets than did their peers in the most affluent schools. And while students overall were just as successful on grade-level work as they were on below-grade-level work, low-income students were given less access to grade-level work even after they had already shown they could master it. The TNTP analysts state, starkly, that “there seems to be nothing many students in high-poverty schools can do to ‘earn’ access to the grade-level work they need to be successful.”
It is important to note that this is supplemental work for students and the report should not be misinterpreted to be an analysis of what may or may not be going on in their actual classrooms. But pre-pandemic research already showed a troubling dearth of grade-level work for many of our neediest students, and poor pandemic era student achievement data clearly suggest that old patterns have persisted—and worsened.
The TNTP analysts are heartened to see that successful performance on grade-level work seems achievable for students—if only they could access it. Thus, they conclude with recommendations focused largely on school culture: Teachers and school leaders must give students as much high-quality grade-level work as possible, be well-prepared to support them in persisting and achieving, and must believe that their students will rise to the challenge.
SOURCE: “Unlocking Acceleration: How Below Grade-Level Work is Holding Students Back in Literacy,” TNTP and ReadWorks (August 2022).
As money and attention focus on career and technical education (CTE) at ever greater levels, researchers can help gauge program effectiveness by digging into the data. Common research questions include: Does CTE lead to college success? Are CTE students better off skipping college and jumping into the workforce? Can CTE students be both college and career ready upon graduation?
A new report published in Educational Evaluation and Policy Analysis is a good example of how data can be analyzed to reach finer-grained conclusions on such questions. Analysts Walter Ecton and Shaun Dougherty start with a rich dataset from Massachusetts’s longitudinal data system and focus on nearly 252,000 first-time ninth graders whose on-time graduation from high school was expected between 2009 and 2017. Their most comprehensive analysis focuses on the first three of those cohorts (graduating 2009–2011), which allows for seven years of post-high school observation. They look at enrollment data, demographics, attendance, residence, state test scores, immigrant status, disability status, and English learner status. College enrollment and completion data come from the National Student Clearinghouse, and employment and wage information come from the Massachusetts Department of Labor’s unemployment insurance system.
Almost 20 percent of students in the sample are classified as “CTE concentrators” based on the federal definition: enrollment in CTE courses for two or more school years at any time during high school. While this is useful shorthand—and indeed many career-based clusters in CTE are two-year sequences with a licensure/certification exam at the end—other research has shown that students can and will take unrelated courses for multiple semesters without such a “concentration” in mind. Ecton and Dougherty wisely break out these “dabblers” from their peers who persist in a cluster sequence. They also distinguish between students who attend CTE-dedicated schools, where a larger and more varied menu of courses is available, and those who take CTE courses in “comprehensive” high schools.
Overall, CTE concentrators are more likely to be male, from lower-income families, and English language learners than their non-CTE concentrator peers. Hispanic students are heavily overrepresented, while Asian students are underrepresented among concentrators. Concentrators score well below the state average on eighth grade tests and are nearly 13 percentage points less likely to attend and graduate from college. Male students are widely overrepresented in the construction, transportation, manufacturing and technology, and information technology clusters; female students are similarly overrepresented in education, health care, and business and consumer science. Students scoring in the lowest quintile of eighth grade test scores are overrepresented in every career cluster studied—highest in transportation, hospitality and tourism, and construction.
CTE concentration generally has no impact on the likelihood of students attending college compared to non-CTE concentrators, but that masks huge variation in different career clusters. Health care concentrators are more than 15 percentage points more likely to attend college than non-concentrators; education concentrators, 13 percentage points more likely. Transportation concentrators are 14.9 percentage points less likely and construction concentrators 12.7 percentage points less likely to attend college. Ecton and Dougherty believe that the expectation of attending college—especially community college—is baked into a number of CTE clusters such as IT and health care in order to complete the required learning.
CTE concentration is associated with a large increase in initial earnings compared to non-concentrators, which persists even seven years after high school graduation. The gap is especially pronounced when comparing CTE concentrators who do not go to college to non-concentrators who also do not go on to college. These results are driven by male concentrators, as females’ initial advantage is smaller and decreases more over time. Lower-income concentrators, Black and Hispanic concentrators, and those with lower prior achievement scores also see earnings boosts compared to their non-concentrator peers.
The highest earnings are seen in construction, transportation, manufacturing and technology, and health care clusters. In most clusters, the advantages begin to decline in years five through seven post-graduation, likely indicating that college-educated students in those clusters are joining the work force. However, non-college goers in health care and construction continue to see higher earnings than their college educated peers. This is an indicator that college is perhaps less germane to these clusters, although schools must make clear to students the real postsecondary prospects that await them. For example, concentrating in the health care cluster does not indicate a path to medical school but to nursing assistant, phlebotomist, and home-health aide jobs.
Finally, concentrators who take courses in CTE-dedicated schools—which enjoy a strong reputation for excellence in Massachusetts—show even higher income boosts than concentrators in comprehensive high schools. In testing for unobserved characteristics such as selection bias (especially for students opting into CTE-dedicated schools) that might have influenced results, Ecton and Dougherty examined dabblers in CTE-dedicated schools and concentrators who only have access to comprehensive high schools. They find very similar patterns to the larger analysis, namely that significant advantage is achieved by concentrators in whichever setting they are educated. Thus they are confident that most of the characteristics driving outcomes are accounted for in their methodology.
While not conclusive proof that CTE course concentration can prepare students for both college and career, this detailed analysis does indicate that positive outcomes in either college or career are likely for CTE students and that these outcomes can be predicted based on concentration level and, especially, on career cluster pursued. Most encouragingly, concentration in nearly any of the CTE clusters typically available can lead to either college enrollment or higher income after high school graduation, an important point as CTE concentrators are often low academic performers whose post–high school prospects might otherwise be bleak.
SOURCE: Walter G. Ecton and Shaun M. Dougherty, “Heterogeneity in High School Career and Technical Education Outcomes,” Educational Evaluation and Policy Analysis (August 2022).
Recognizing the importance of an educated citizenry, Ohio taxpayers have made generous investments in K–12 education. In FY 2021, statewide spending on public primary and secondary education reached a record high of $21 billion or $13,300 per pupil. Last July, the legislature passed a new funding formula that could potentially add another $2 billion per year in state education spending.
But how those funds are distributed to schools and ultimately spent matters enormously to student achievement. In these realms, unfinished business remains.
Our latest policy brief, second in a series of papers, looks at several issues that require attention in the next state budget cycle. The brief also offers recommendations that would continue to move Ohio toward a more efficient, productive, and transparent school funding system.
