The Education Gadfly Weekly

On a recent visit to Xenia schools, State Superintendent Paolo DeMaria said: “I think so many times we underestimate what students can do. Our hearts tell us to be a little more protective—I’m not sure they can reach that level or handle that kind of work. But time and time again as I go across the state talking to teachers, I find that their socks are blown off because kids always are able to exceed even the highest expectations we set.”

Superintendent DeMaria is right, but not everyone in authority seems to agree. As you’ll recall, doomsayers sounded the alarm back in 2016, declaring that hordes of students would never meet the state’s new and tougher graduation requirements, which offer young Ohioans three pathways to a diploma. Of the three, the state test pathway caused the most concern: Students earn one to five points based on their performance on seven end-of-course (EOC) exams, and must earn a total of eighteen out of thirty-five points to graduate. District officials claimed that students were struggling to accumulate eighteen points because the tests were so difficult.

Perhaps unsure how to judge that concern, and open to giving schools more time to adapt to a higher bar, the legislature in June 2017 approved a temporary set of weak “alternative pathways” for the class of 2018, the first affected by the new requirements. There were a few alternative options—like earning credits in College Credit Plus or certain exam scores in an AP or IB course—that required an objective demonstration of academic mastery. But there were also others—like 93 percent attendance during the twelfth grade year or completing 120 hours of work or community service—that had nothing to do with academics at all.

The class of ’18 has now graduated but—depressingly—the State Board of Education remains unconvinced that upcoming classes can meet higher expectations. This spring, board members returned to the legislature and asked that the softened 2018 graduation requirements be extended to the classes of 2019 and 2020. To its credit, the legislature left Columbus for the summer without assenting. Senate President Obhof, apparently remembering that the 2018 changes were intended to be temporary, remarked: “There were a number of members of the caucus who expressed the same concern I did at this point last year about that being a temporary, one-year fix, not a permanent fix.” Representative Andrew Brenner, chair of the House Education and Career Readiness Committee, added: “We want to see more data. We want to see some more results and then we’ll go from there.”

Legislators are right to demand more data. We still don’t know how many students met the original, more robust graduation requirements, even though the class of 2018 received their diplomas a few months ago. The Ohio Department of Education (ODE) hasn’t managed to make this calculation despite the critical importance of having accurate data before changing state policy. The only “data” currently available are ODE projections from January 2018, which predicted that 76.5 percent of the class of 2018 would likely graduate without the alternative pathways, i.e., would reach the higher bar.

The aforementioned doomsayers likely interpret that projection as one in four students being denied a diploma. But doing so purposely ignores that Ohio’s most recent four-year graduation rate was 83.4 percent, not 100 percent. Simple subtraction implies that 16.6 percent of students failed to graduate on time, even under Ohio’s older and more lenient system. The percentage of students who were on track to graduate but hadn’t met the higher standards compared to Ohio’s previous graduation rate is just seven percent (83.4 minus 76.5). Nobody wants to see graduation rates go down, but that sort of drop is far from the “apocalypse” some had predicted.

Legislators are right to resist putting another temporary fix into law. The new requirements were adopted for an important reason that hasn’t gone away: far too few Ohio graduates possessed the knowledge and skills needed to succeed in college or the workplace. Since the requirements represent a significant escalation of expectations, they would surely result in a temporary decline; the same thing happens to proficiency rates when states raise academic standards or administer harder tests—or even just new tests. As time goes on, students and schools adjust to the higher expectations and performance improves. That already appears to be happening in Ohio with proficiency rates on state assessments.

Massachusetts offers a glimpse of what can happen when states stand firm on higher expectations. While discussing how the Bay State implemented its landmark Education Reform Act of 1993, Dave Driscoll, the former Commissioner of Education in Massachusetts, explained that the law called for raising standards and expectations for three “buckets” of people: students, educators, and schools and districts. For students, higher expectations took the form of tests that students had to pass in order to get a diploma.

Just like in Ohio, Massachusetts officials faced a lot resistance. Driscoll recalled that “a lot of legislators were getting cold feet, a lot of superintendents were saying this isn’t going to work, a lot of principals were saying this is not right for kids.” Fortunately, the state’s “governors, speakers of the house, senate presidents…[they] stuck with it. And I think that was the key…sticking to high standards has been a hallmark for the political leadership of Massachusetts.”

It’s a lesson that Ohio’s state board seems not to have learned. If members seriously believe that too few Ohio students will meet state test requirements (or the two other pathways), they could take unilateral action by lowering the minimum number of EOC points needed to graduate. In fact, back in 2016 they considered lowering the number of required points from eighteen to fifteen. Fortunately, they didn’t do that, but the board remains responsible for setting the number of EOC exam points required to get a diploma—it’s still in their power to adjust the threshold.

As long-time advocates of high expectations, we believe that lowering the bar would be a mistake. But if it comes down to a choice between lowering the test-score point threshold temporarily or continuing to push for academic-lite options to be established into law, it’s an easy decision.

When the state offers non-academic graduation pathways, it communicates to students that learning doesn’t much matter—that their actual readiness for the future struggles they face matter less to state leaders than the overall graduation rate. It’s not a coincidence that Massachusetts is the highest achieving state in the country, or that Ohio lags behind. Massachusetts’s success started with instituting high standards—including a set of rigorous graduation tests—and then sticking to them by helping kids who struggled instead of just moving the goalposts. So far, Ohio has failed to do that.

If state board members share Superintendent DeMaria’s view that kids will exceed even the highest of expectations, they’d be wise to focus on setting and maintaining high expectations for all students and making sure that Ohio education policy is laser-focused on helping every young person in the state reach rigorous levels of academic attainment.

Editor’s Note: David Driscoll, former Commissioner of Education for the Commonwealth of Massachusetts, has been a member of the boards of the Thomas B. Fordham Institute and the Thomas B. Fordham Foundation since 2011.

In a recent report, school-funding expert Howard Fleeter analyzes Ohio districts’ revenues stretching back to 1999. His study concludes that, in the two decades after the DeRolph school-finance litigation, Ohio made only moderate progress in creating an equitable funding system. According to his calculations, the gap in state and local revenue that the wealthiest districts generate versus the poorest narrowed from 8 percent in 1999 to 5 percent today. In an interview with the Columbus Dispatch, he sums up the results: “We lowered the gap, but not by a whole lot. Getting them [the poorest districts] to the same (level) of the high-wealth districts still isn’t adequate ... because their costs are higher. They have kids who cost more to educate.”

Those of us at the Thomas B. Fordham Institute agree with Fleeter that schools serving more disadvantaged children should receive more public aid. And we respect his more than two decades of work on school finance; his voice is highly influential in the ongoing funding debates, and appropriately so. But all this doesn’t mean that his results should go without challenge. In this piece, I wish to raise a few questions about his analyses and share some skepticism as to whether the state’s poorest districts are underfunded.

Let’s first take a closer look at the key chart in Fleeter’s report, displayed below. The focus of his analysis is on local and state revenues for districts in quintiles 1 and 5—i.e., the poorest and wealthiest 20 percent of districts, respectively, in terms of property values. In FY 1999, the chart shows that Ohio’s poorest districts received $6,033 per pupil in state and local aid, while the wealthiest received $6,524 per pupil—an 8 percent gap. Fast forward to FY 2019[1] and the poorest districts now receive $7,808 compared to $8,192 per pupil for the wealthiest—a 5 percent difference. Bearing in mind the notion that poorer schools should receive more (not less) in public funding, these results appear to be discouraging in terms of achieving funding equity goals.
 

​

A few observations and questions, however, are in order:

• First and most striking is the decision to deflate FY 2019 dollars to 1999 dollars, rather than inflating the older to the more current (the standard practice). While Fleeter’s method is not technically incorrect, it vastly understates the current levels of school funding. For example, the chart above displays a statewide average of $7,567 per pupil in state and local revenue, even though in FY 2017 this amount was about $11,500 per pupil. Regrettably, the “deflated” amount made it into the Dispatch’s coverage of this study, an egregiously misleading statement of funding levels, especially given that the public already underestimates true amounts of school funding.
• Second, note that federal revenues are excluded even though they are part of the overall equity picture. While these revenues pale in comparison to state and local revenues, poor districts tend to receive more federal aid than wealthier ones, so excluding these dollars results in an incomplete picture.
• Third, the poorest districts in this analysis are defined only by their property wealth. This is one way to look at poverty, and it may have been the only “wealth” variable available from 1999 to 2019. But it’s also possible that this measure may not capture districts that are actually educating the state’s poorest students. For example, Columbus and Cincinnati have relatively high levels of property wealth, but they also serve predominantly low-income children. Would the results change based on different definitions of “poor”?

With these points in mind, let’s take another look at the situation. What follows focuses on funding amounts in FY 2017, the most recent year in which we have full datasets. Instead of just considering state and local revenues, the following figures rely on the “total expenditure per pupil” statistics produced by Ohio Department of Education, which also include the spending of federal dollars. These data represent all operational spending, and they exclude capital expenditures, as well as the spending and enrollments of charter schools (save for a small number of “conversion” charters sponsored by districts).[2] The following set of charts show districts’ average expenditure per pupil based on four indicators of (dis)advantage; districts are ranked along these metrics and then divided into quintiles of roughly equal enrollments.[3]

The first two charts focus on districts’ property values per pupil and their median incomes. Figure 1 shows that the poorest districts in property values—quintile 1—slightly outspend their counterparts in the top quintile. Specifically, they spend $12,771 compared to $12,235 per pupil in quintile 5—a small advantage of $536 per pupil.

Figure 1: Operating expenditure per pupil by district property valuation per pupil (FY 2017)
 

​

Of course, property wealth or the lack thereof is not the only equity measure that policymakers might care about. When examining median incomes, Figure 2 shows that districts with the lowest incomes spend more than those with the highest—a more sizeable advantage of $1,167 per pupil.

Figure 2: Operating expenditure per pupil by district median income (FY 2017)

​

One might also want to consider whether districts educating disproportionate numbers of higher-need students spend more. Key indicators of student need are whether a child is economically disadvantaged (ED), black, or Hispanic. On average, these student groups achieve at lower levels than their peers and may need additional resources. Both Figures 3 and 4 indicate that the highest-need districts—quintile 5 in these charts—spend substantially more than districts with the lowest fractions of ED or black and Hispanic populations (quintile 1). Figure 3 shows that districts serving the highest percentages of ED students spend on average $13,962 per pupil, or $2,454 per pupil more than the lowest ED districts. Meanwhile, Figure 4 indicates that districts enrolling the highest percentages of black and Hispanic students spend upwards of $14,000 per pupil—or $3,810 per pupil more than districts with the lowest percentages of students from these race/ethnicities.

Figure 3: Operating expenditure per pupil by percentage of economically disadvantaged students (FY 2017)

​

Figure 4: Operating expenditure per pupil by percentage of black and Hispanic students (FY 2017)

​

***

These charts, of course, cannot tell us whether funding equity has improved or weakened since the conclusion of the DeRolph cases—one of the main concerns in Fleeter’s analysis. Nothing in the piece here suggests that his calculations of state and local revenues are incorrect. But by looking at total operational expenditures, which include federal dollars, and analyzing different indicators of disadvantage, we see the picture can change dramatically. As other studies have concluded, examined in various ways, Ohio’s funding system looks quite fair.

In the final days of August, the Ohio Department (ODE) of Education and the State Board of Education released their five-year strategic plan for education. It includes a state-level vision, a goal focused on high school graduates, four learning domains, ten priority strategies, and three core principles.

The bulk of the plan is a breakdown of the ten strategies that ODE and SBOE plan to use to achieve their declared goal. Although each of these strategies is worth studying in depth, there are three in particular—those that are aimed at improving standards, assessments, and accountability—that deserve a close look:

Strategy 4: Identify clear learning standards and guidelines that reflect all four equal learning domains.

Strategy 5: Move toward a varied system of assessments to appropriately gauge the four equal learning domains and allow students to demonstrate competency and mastery in ways beyond state standardized tests.

Strategy 6: Refine the state’s accountability system to be a fairer, more meaningful process that reflects all four equal learning domains.

Each of these strategies contains a reference to the “four equal learning domains:”

​

The key word, certainly implied by its use in each of the strategies, is “equal.” Going forward, ODE and the state board intend for these domains to be equally important within the state’s standards, assessments, and accountability system. That’s a pretty significant change because two of the four don’t factor into our current system.

The top two domains cover the kind of traditional, concrete academic competencies that are core to schooling. Ohio already has state standards and assessments for many of these subject areas, and they are included in our state accountability system through annual student assessment in subjects like math and English language arts. But the bottom two domains are far more abstract. The examples provided for these two categories—things like creativity, self-management, and relationship skills—are generally considered to be non-cognitive, or “soft skills.” While these have long been understood as critical to student success, they aren’t part of Ohio’s current accountability system in any formal capacity.

The inclusion of non-academic domains and non-cognitive skills in Ohio’s overall plan is not without merit. Research shows that possessing non-cognitive and social-emotional learning (SEL) skills can affect both academic and life outcomes. Teachers who complement their academic instruction with access to SEL opportunities are helping students develop skills that have long-term benefits. Not to mention the well-documented SEL benefits of extracurricular activities. In fact, these skills are so important that the Organization for Economic Co-operation and Development (OECD), which is responsible for administering the PISA assessment, has taken an interest. They recently released a working paper that includes a conceptual framework for their new international study on social and emotional skills, and the survey’s findings will be released in 2020.

Given the research and international attention, it’s no surprise that ODE and the state board have included SEL as part of their plan for educating the “whole” child. The decision to commission a standards advisory group to recommend SEL standards for all grades (Ohio only them for kindergarten through third grade at the moment) is a wise move. Ohio should be giving teachers and school leaders guidance and resources on SEL.

Unfortunately, the plan goes beyond just establishing standards and collating resources. Under strategies 5 and 6, it suggests that these soft skills should be assessed and included in the state accountability system.

That would be a serious mistake. Ohioans have been vocal about their desire for fewer assessments and a less complicated report card. Numerous state board members have expressed grave concerns about over-testing. Superintendent DeMaria has recommended cutting a number of state tests. Legislators have proposed bills aimed at streamlining report cards. Why, then, would ODE and the state board publish a strategic plan that increases the number of assessments students must take and adds even more measures to state report cards? Even if the state opts for non-traditional tests—like student portfolios, projects, or experiences—it doesn’t change the fact that they are tests that will take time to administer, prepare for, and collect data on.

More concerning, the plan ignores the advice of leading researchers who have repeatedly warned about the danger of including SEL measures in high-stakes accountability systems. A 2017 report from the Learning Policy Institute concluded that although states could use SEL measures at the local level to inform teaching, learning, and program investments, “states should not use measures of students’ social-emotional competencies for high-stakes accountability purposes.” This is because they are still too new, weren’t designed for cross-school comparison, and are particularly vulnerable to reference bias—meaning that a person’s individual frame of reference or beliefs can influence how they report on or evaluate themselves and others.

Angela Duckworth, the researcher who is credited with popularizing the concept of grit, has also conducted research that demonstrates the danger of reference bias in SEL measurements. In 2016, she wrote a piece for The New York Times that carefully outlined why measures of character shouldn’t be included as part of high-stakes accountability metrics. She explained: “Policy makers who assume that giving educators and students more reasons to care about character can be only a good thing should take heed of research suggesting that extrinsic motivation can, in fact, displace intrinsic motivation. While carrots and sticks can bring about short-term changes in behavior, they often undermine interest in and responsibility for the behavior itself.” Carol Dweck, whose research covers fixed and growth mindsets, has also expressed reluctance about using mindset measures for high-stakes accountability.

And then there are the implementation issues. The plan’s assessment strategy says that skills within the non-academic domains “can be observed by individuals trained to do so.” There are no details about who these individuals are or what kind of training they would have to undergo, but considering how tight Ohio’s purse strings were during the recent budget cycle, ODE probably won’t be hiring a bunch of professionals to conduct these assessments. That means all the work is probably going to fall on teachers. Given that SEL measurements are prone to reference bias—and all kinds of subjectivity— there should be some serious concern about how the state plans to overcome differing norms and prevent efforts to game the system. After all, there’s a reason that not a single state opted to measure SEL for school performance despite new flexibility under ESSA.

To be fair, the strategic plan is just that—a strategy. There are no mandates and few details included in its contents, so it’s not clear how ODE and the state board will assess SEL skills or incorporate them into school report cards. In fact, the plan is carefully worded to say that embracing the four domains will merely “inspire the state to explore innovative approaches to assessments that go beyond academic content” (emphasis mine). At the end of the day, the power to add assessments and change the accountability system still rests solely with the General Assembly.

But it should worry lawmakers, advocates, families, and the general public that ODE and the state board appear to be recommending adding assessments and changing the accountability system in this way. There is no question that both organizations should be promoting SEL. They should create a clearinghouse of resources for teachers and leaders. They should invest in effective and meaningful teacher training. They should consider all the evidence and keep a close eye on upcoming research from respected organizations like OECD.

But they should not be calling for SEL skills to become part of our assessment and accountability system. And they especially shouldn’t be “shying away” from traditional academic competency to do so. SEL is a key component of student achievement and growth, but it’s not the only component. Content mastery matters too, as do “hard skills.” NAEP scores, ACT scores, and remediation rates indicate that Ohio still hasn’t figured out how to make sure all of its students are achieving at high academic levels. Until it does, policymakers should be focused on how SEL skills can complement academic mastery, not replace it.

Editor’s Note: As Ohioans prepare to elect a new governor this November, and as state leaders look to build upon past education successes, we at the Fordham Institute are developing a set of policy proposals that we believe can lead to increased achievement and greater opportunities for Ohio students. This is the first in our series, under the umbrella of creating transparent and equitable funding systems. You can access all of the entries in the series to date here.

Proposal: For funding purposes, Ohio should decouple the identification of economically disadvantaged (ED) students from eligibility for federal free and reduced-priced lunches (FRPL). Instead, the state should identify low-income students through their family’s participation in other means-tested programs, such as the Supplemental Nutrition Assistance Program and/or Temporary Assistance for Needy Families—a process known as direct certification.

Background: With fewer resources at home, low-income students require more public funds to support their education. Recognizing this, Ohio provides additional state aid to districts serving more ED students (see table 1 above). In FY 2018, the incremental amount is $272 per pupil, with an adjustment that steers more dollars to districts serving the highest proportions of ED students. Ohio generally identifies ED students based on their eligibility for the federal program of free or reduced-priced lunches (FRPL), a program that has traditionally been linked to family income. But a recent change in federal law now allows higher-poverty districts to offer meals at no cost to all students, regardless of their income. Known as Community Eligibility Provision (CEP), this program has led to a growing number of districts reporting all students as ED—even though a significant number may not come from low-income households. In 2016–17, forty-nine Ohio districts reported blanket ED coverage (> 98 percent), compared with just seven such districts four years earlier. Though CEP is a worthwhile meals initiative, linking ED identification to FRPL eligibility increasingly misdirects state aid intended for low-income children to those with higher incomes.

Proposal rationale: Ohio rightly provides additional funds to districts and charters serving low-income students. However, the implementation of CEP has compromised the use of FRPL eligibility as a method to identify low-income children, leading to a misallocation of state funds. By identifying ED pupils based on participation in another means-tested program—something that several other states, including Indiana and Massachusetts, have been able to do—Ohio would more accurately target funding intended to support low-income students.

Cost: The state may incur an indeterminate amount of administrative expenses to link K–12 students to other government benefit programs. Once links are made, the state should consider increasing funding amounts for ED students. Changes in the percentage of students identified as ED would affect district funding levels, so a phase-in strategy may be needed to mitigate impacts on district funding.

Resources: For more background on providing additional funds to support low-income students, see Matthew Chingos and Kristin Blagg’s report Do Poor Kids Get Their Fair Share of Funding?, published by the Urban Institute (2017); for details on CEP, see ODE’s “Community Eligibility Provision”; and for information on other states’ methods for delivering additional aid for low-income students, see EdBuild’s Poverty Funding: Policies in Each State.

Among the most frequently heard concerns around charter schools is that they drain money from traditional districts, potentially harming students who stay behind. Yet another school of thought theorizes that charters encourage district improvements by injecting competition into a largely monopolistic system. A new study conducted by Matthew Ridley and Camille Terrier puts both these claims to the test using data from Massachusetts, a state that recently held a hotly debated and ultimately unsuccessful referendum on expanding charters.

To examine charters’ effects on district spending and achievement, the researchers rely on a 2011 reform that allowed charter school expansion in underperforming Massachusetts districts. They identify nine “expanding” districts, including Boston, where the charter share increased markedly post-reform, and then compare their spending and test-score growth in math and English language arts (ELA) to “non-expanding” districts whose charter share remained flat. The analysts use various statistical techniques, including a “synthetic control method,” to estimate the impacts of charter expansion in the years after reform (2011–12 through 2014–15).

In terms of fiscal impact, their study finds that charter expansions increased districts’ per-pupil expenditures. Post-reform, total per-pupil spending in expanding districts rose at a faster clip than non-expanding ones, and by 2015 their expenditures were 4.8 percent higher. When broken down by expenditure type, the analysts find that fixed-cost expenditures, like maintenance and interest, increased by 6.2 percent and instructional expenses rose by 7.2 percent relative to the control group. Yet support service expenses slipped in expanding districts, suggesting that districts responded to greater competition by reallocating resources from support to instructional activities.

There is, however, an important policy twist to these financial results. During this period, Bay State districts with increasing charter enrollments received state reimbursements that compensated for the financial losses associated with transfers, explaining much of the rising expenditures. Yet this is also a costly approach that funds districts for students not in their classrooms. And it may be an unsustainable one as well; as the authors note, in more recent years—starting in FY 2015 to 2017—Massachusetts has cut its reimbursement rates. Nevertheless, whether the fiscal results of this study apply in states without reimbursement policies remains less certain.

Turning to the effects on student achievement, Ridley and Terrier conclude that charter expansions led to modest improvements among district pupils. Their analysis reveals achievement gains in both math and ELA among the expanding districts relative to those whose charter shares did not increase. Specifically, by 2015, expanding districts saw achievement gains of 0.03 standard deviations in math and 0.02 in ELA—what Ridley and Terrier call a “small positive effect.” They note that these gains are similar to those uncovered in a few other charter studies that also found mild but positive competitive effects.

This study challenges the notion that charters damage the well-established district school system. And it’s a pity that critics continue to push this canard to coax voters into opposing efforts to expand charters. (Can we redo the Massachusetts referendum in light of this study?) Instead, an accumulating body of evidence indicates that, through healthy competition—and collaboration as well—the expansion of charter schools can indeed contribute to a rising tide that lifts all boats.

SOURCE: Matthew Ridley and Camille Terrier, Fiscal and Education Spillovers from Charter School Expansion, School Effectiveness & Inequality Initiative (2018).

Across the nation, urban public charter schools are posting impressive student achievement gains. Under pressure to achieve similar results, many traditional public districts have chosen to follow in charter schools’ footsteps and grant some of their schools more autonomy. These hybrid schools are still operated and supported by district officials but are permitted to opt out of certain district policies and practices in staffing, learning models, curriculum, budgeting, school calendar and schedule, and professional development.

But does giving traditional public schools more autonomy actually raise student achievement? A new report from the Progressive Policy Institute answers this question by analyzing state standardized test scores in four districts that operate autonomous schools: Boston, Memphis, Denver, and Los Angeles. The authors control for race and ethnicity, language proficiency, socio-economic status, and special education status.

The data show that a positive relationship exists between school autonomy and student achievement. Students at some autonomous schools in Boston and Los Angeles were more likely to be proficient than their counterparts in traditional public schools, but independent charters outperformed all autonomous school models in Boston, Los Angeles, and Denver.

Memphis was the only exception. In 2010, the state of Tennessee created the Achievement School District (ASD), a state-run turnaround district made up of schools that scored in the bottom 5 percent on state exams. The same legislation that created the ASD permitted districts to create innovation zones, or iZones, for other low-performing schools that would remain under district control but be given increased flexibility and support in exchange for greater accountability. Results indicate that students at independent charters and Memphis’s iZone schools did as well as students at traditional public schools, and better than those at ASD charters. The authors suggest that these unique results may be due to an influx of funding, support, and talent at iZone schools.

To close the report, the authors offer five possible explanations for the findings, including that charters have true autonomy, are schools of choice, and are held accountable for student performance via closure or replacement, unlike their district counterparts. The report also offers recommendations for districts interested in implementing autonomous schools. 

SOURCE: David Osborne and Emily Langhorne, “Can urban districts get charter-like performance with charter-lite schools?” Progressive Policy Institute (August 2018).