Most Ohio Gadfly readers know that we typically offer in-depth commentary one topic at a time. This tendency assumes (pardon the holiday metaphor) that one huge present is preferred—like the Lexus tied up in a bow. We recognize that other folks might prefer a bundle of gifts. So, for those yearning for a little more diversity in their inbox, this one is for you.
Most Ohio Gadfly readers know that we typically offer in-depth commentary one topic at a time. This tendency assumes (pardon the holiday metaphor) that one huge present is preferred—like the Lexus tied up in a bow. We recognize that other folks might prefer a bundle of gifts. So, for those yearning for a little more diversity in their inbox, this one is for you. (No white elephants, we promise.)
A win on ESSA accountability
In late November, the U.S. Department of Education released its revised and final regulations on school accountability under the federal Every Student Succeeds Act (ESSA). In a victory for high achievers, the feds made it crystal clear that states are permitted to use a performance index—as Ohio has long done—as an indicator of student achievement. Regrettably (see here and here for why), the previous draft regulations would have likely forbidden performance indices and forced states to use proficiency rates instead. Now it’s full steam ahead on the performance index as Ohio drafts its ESSA state plan.
Information in the palm of your hand
Kudos to state leaders who are making Ohio’s report card data useful and accessible to policy wonks and the general public alike. A recent Data Quality Campaign (DQC) publication spotlights Ohio’s school report cards as exemplars for providing “data that is valuable to my community” and displaying clear information. The report also notes that Ohio’s database houses the large majority of the data elements DQC deems important for public review (fifteen out of twenty-three). Meanwhile, the Ohio Department of Education last week launched a smart phone app where users can receive updates and check out school report cards anywhere and anytime. Is your local school board member—or maybe real estate agent—waxing poetic on how lovely the schools are? Now you can get the lowdown on the data and see for yourself. As President Reagan once said, “trust but verify.”
No thanks on the similar students measure
In the recent charter reform legislation, state lawmakers ordered the Ohio Department of Education to “conduct a study to evaluate the validity and usefulness of using the ‘similar students measure.’” In a report issued in late November, the Department concluded after said study that the measure was “neither valid nor useful” for use in the Ohio’s accountability system. The measure, pushed by a charter advocacy group and ECOT, adjusts a school’s achievement rate depending on its demographics (for more, see here). One of the central problems, however, is that the measure would set lower proficiency expectations for disadvantaged children. As Chris Woolard, the Department’s accountability chief, told the Columbus Dispatch: “Our system right now has high expectations for all students. This [measure] violates that basic principle that we want all students to be able to succeed.”
Auditor Yost on inter-district open enrollment
Ohio Auditor of State Dave Yost recently released a report on the fiscal impact of inter-district open enrollment. The main takeaway: Districts should weigh the costs and benefits of accepting additional pupils via open enrollment. Under state law, districts are not obligated to accept open enrollees—though state funding follows students, offering districts a financial incentive to do so (no local dollars transfer however). According to his cost-benefit calculations for four Northeast Ohio districts, one posted a net loss of $1,282 per incoming open enrollee, while another gained a whopping $4,563 per open enrollee. The fiscal impact, as the Auditor explains, depends in large part on capacity. When a district has “empty seats,” the cost of educating an open enrollee is minimal—teachers wouldn’t need to be hired, for example—but it would gain the funding tied to the student. The reverse might be true when a district is at or near capacity: Marginal costs could exceed the benefit. The Auditor understands the finances of open enrollment, but this analyst at least wonders whether economic concerns could be used as an excuse by public schools to not accept all comers. (“Sorry kid, we just don’t have the capacity.”) This begs a couple questions: a) just how many districts in Ohio are at full capacity, including suburban ones that prohibit open enrollment altogether; and b) for districts without excess capacity—but facing increasing demand—should the state support expansions, so they are not turning away students?
AP scores of 2 = proficient?
In an amendment to Senate Bill 3, a deregulation bill that passed last week, state legislators added language that would deem an Advanced Placement (AP) score of 2 equivalent to proficiency on certain state end-of-course exams (EOCs). Ohio high schoolers may substitute AP test results for EOCs in the following content areas: US History, US Government, and science (substitutions are not allowed in math or English). This raises an eyebrow, because an AP score of 2 is typically considered mediocre—the second lowest on AP’s 1-5 scoring scale. It’s a score that colleges and universities won’t accept for course credit—a minimum 3 or 4 is required. In addition, an AP score of 3 is needed for schools to earn credit on Ohio’s Prepared for Success report card component. It may be true that an AP score of 2 is technically a closer equivalent to EOC proficiency than a 3 (AP tests are likely more difficult), but it does seem peculiar to call an unsatisfactory AP score “proficient.” Did the student demonstrate proficiency in the AP course? According to the test results, it’s not clear she did. Or maybe this predicament calls into question the notion that different standardized tests are so easily substitutable.
We hope you enjoyed this package of ed news gifts. Stay tuned in the New Year as we continue to track these stories and much more!
One of the big Ohio education stories of 2016 was the growing popularity of College Credit Plus (CCP), a program that provides students three ways to earn college credit from public or participating private colleges: by taking a course on a university campus; at the student’s high school where it’s taught by a credentialed teacher; or online. Many students and families have found that the program saves them time and money and provides valuable experience. For families with gifted or advanced students, it is a chance for acceleration even as early as seventh grade; for students in high-poverty rural and urban areas, it may be the only way to take high-level courses in basic subjects, let alone electives.
Before registering, students in grades 7-12 must be admitted to the college based on their readiness in each subject they plan to take a class in—a decision made by each higher education institution and determined by GPA, end-of-course (EOC) exam scores, and other available data. Once admitted, students can register for any course the school offers, except for those that are considered remedial or religious. (The latter restriction is presumably intended to keep church and state separate while a child is enrolled in a public school.)
Most of the media coverage of the growth of College Credit Plus has focused on its cost, but in October, the state released an overview of preliminary information gathered during the first year (and part of the second year) of the program. Here are a few of the most interesting data points:
The preliminary data suggest a few areas that need attention as Ohio works to ensure that CCP is functioning as intended.
Pay close attention to passage rates
Eyebrows should rise when seeing that over 90 percent of courses taken by CCP students resulted in credits earned. Other data points—such as state test scores and ACT scores—show a troubling lack of proficiency that one might expect would translate to a smaller percentage of students earning credit. Similarly, average scores on Advanced Placement (AP) exams indicate that far fewer than 90 percent of AP courses taken result in college credit earned or even in what the College Board terms a “qualifying score” (3 and up). Why, then, are CCP’s passing percentages so high?
One reason may be that CCP’s eligibility requirements permit only college-ready students to enroll. By restricting enrollment to students who have successfully demonstrated that they are college ready, usually through widely accepted measures of college readiness like the ACT and Compass, CCP passing percentages may be high because the college readiness requirement is acting as an effective gatekeeper. But this could also be considered suspect (i.e. the requirement may be a so-low-as-to-be-meaningless bar) considering how many of the participating post-secondary institutions are “open enrollment” campuses. It’s also possible that CCP courses—most likely those taught on a high school campus by a secondary instructor—just aren’t rigorous enough. Passing these courses is determined by the teacher, not by an external review such as Advanced Placement uses, and we live in an era of grade inflation.
How to ensure that courses taught on high school campuses are rigorous?
The majority of CCP courses (nearly 61 percent) were taught on a high school campus by a secondary instructor—an educator who is already teaching at the high school but has earned additional credentials. Although the state data showed that student GPAs varied only slightly based on class location, that’s no reliable gauge of course rigor. The state’s report notes that “monitoring quality and participation when the course is taken on the high school campus” is an “item to discuss.” Policymakers should talk to representatives from K-12 and higher education for ideas on how to maintain rigor, including how best to train secondary teachers to teach post-secondary classes and to evaluate student work by post-secondary standards rather than K-12 criteria.
Maintain entrance requirements for students
Although some folks have bemoaned the challenges that students face in qualifying for CCP, the college-readiness restriction is critical for two reasons. First, it ensures that only students who are academically prepared for the rigors of college are able to participate, a requirement that, if forcefully and dutifully applied, should prevent students from the double-whammy of a failing grade on both their high school and college transcripts. Second, students who are ineligible for CCP one year can still become eligible the following year if they are able to demonstrate that they have achieved college readiness; this could provide students with more motivation to work hard to reach the bar. A college freshman who isn’t college-ready, on the other hand, has no options except expensive, non-credit-bearing remedial courses. Still, we must keep in mind the softness of a “college readiness” criterion when determined and applied by an open-access college.
Keep CCP and co-requisite remediation separate
A bill now before the General Assembly (House Bill 474) would create a “CCP Co-requisite Remediation Pilot Program.” This would aim at high school seniors in need of remediation in math and English by allowing them to “simultaneously enroll in a remedial course and an introductory college course in the same subject area, or enroll in an introductory college course that incorporates remedial curriculum.” Whatever the merits (and weaknesses) of co-requisite remediation, it’s illogical to push college-level remediation into high school via a program that is, by law, intended only for college-ready students. Remediation on college campuses occurs because students didn’t learn what they needed to learn in high school. Why, then, would the state allow students who are still in high school—and thus still have a chance to prepare for college prior to enrollment—to take college level work for which they are unprepared? Why not encourage high schools to do a better job preparing their students for college, rather than shoving those same students further along the path? The risks for students, whose grades in CCP courses appear on both their high school and college transcripts, are just too high.
Take a deep dive into the underrepresentation of minority students in CCP
Participation gaps aren’t only a CCP problem; AP courses face a similar issue. Although a ton of analysis has been done on AP participation gaps, it’s more difficult to diagnose the cause of CCP’s participation gaps (mostly for African American students) based solely on the information released by the state. Analysts should gather more information and investigate what could be causing the discrepancy. We can assume—based on state test scores—that too few minority students are prepared to qualify as college ready while still in high school. But there could be additional factors at play. Is more and better outreach needed in particular schools? Are some schools subtly discouraging participation or less likely to facilitate the high school-located classes intended to minimize transportation challenges? Answers to these questions could begin to narrow participation gaps.
***
CCP is new and we should expect glitches and growing pains. There are no easy solutions to all the problems that it faces, but the initial uptake by students suggests that College Credit Plus is worth continued attention and improvement.
From the latest issue of the journal Economics in Education Review comes a fascinating paper in which author Metin Akyol creates mathematical models that simulate the effects of private school vouchers on the overall education system. It is not a study of an actual voucher program, but instead a thought experiment meant to test whether both universal and targeted voucher programs can increase the efficiency of the education system as a whole. As strange as this may seem to lay readers, there is in fact a long history of such econometric analyses—and their findings are often worthy of consideration.
Akyol’s complex model can’t be fully explained in this short review, but some features are worth noting. It incorporates the findings of empirical voucher studies to increase its reliability. It simplifies the real world in an effort to find the signal in the noise. Every household therefore has only one child, and the hypothetical school district has neither magnet schools nor charters. And one of its defining assumptions is that more efficient public school spending is an effective proxy for increased educational quality. In other words, it presumes that the money saved by greater efficiency can be reinvested in ways that improve outcomes.
Regardless of how one feels about all this, the model ends up producing outcomes that are very similar to empirical findings regarding actual programs. In one important example, the positive effects on voucher-eligible students who do not opt to leave their district school (found empirically by David Figlio in Ohio) are predicted in Akyol’s targeted-voucher model. Public schools in the model are observed to “up their academic game” to retain students when voucher competition is introduced. Additionally, the model predicts that students lowest on the income scale will be least likely to use vouchers, even in a model where vouchers are universally available and not means-tested. This stands to reason, considering that real-world vouchers often fall short of full private school tuition. (To some extent, it was also borne out in Figlio’s research.)
Also interesting is the difference in effects between a universal voucher program and a targeted one. Akyol ran models that replicate the prime goal of vouchers—make private schools affordable for more children—in two ways: by manipulating the voucher availability and by simply changing the family income distribution. The results were not the same. The universal voucher model led to an observable decline in “peer group quality” for those students at the lowest end of the income spectrum who did not take the vouchers. This decline in quality was also present to a lesser extent in the model where vouchers were targeted at low-income students. But it was absent in a model that gave high-ability students lower voucher amounts than their lower-ability peers. This appears to be the theoretical sweet spot that results in the most favorable overall outcome: more students were able to access private schools, and public schools felt the competition keenly enough to improve their academics for the students who remained.
None of this means we must redesign real-world voucher programs based on any one of the mathematical models presented in this paper. But to the extent that modeling can predict real-world outcomes, choice advocates and policymakers ought to consider the results, which can elucidate the potential benefits and challenges of particular voucher designs. If the incoming Trump/DeVos education department is going to prioritize vouchers as a means for improving education, Akyol’s mathematical models have at the very least led him to offer some sage advice: “…the outcomes of a voucher program hinge on its design.”
SOURCE: Metin Akyol, “Do educational vouchers reduce inequality and inefficiency in education?” Economics of Education Review (December, 2016).
Ohio’s charter school reform discussions have mostly focused on sponsors—the entities responsible for providing charter school oversight. Overlooked are the important changes in Ohio’s charter reform law (House Bill 2) around operators. Operators (aka management companies) are often the entities responsible for running the day-to-day functions of charter schools; some of the responsibilities they oversee include selecting curriculum, hiring and firing school leaders and teachers, managing facilities, providing special education services, and more. (To get a sense of the extent of operator responsibilities, read through one of their contracts.)
Extra sunshine on operators has been especially needed in a climate like Ohio’s, where operators historically have wielded significant political influence and power not only with elected officials but even over governing boards. For instance, one utterly backwards provision pre-HB 2 allowed operators to essentially fire a charter’s governing board (with sponsor approval) instead of the other way around—what NACSA President Greg Richmond referred to as the “most breathtaking abuse in the nation” in charter school policy.
HB 2 installed much-needed changes on this front, barring the most egregious abuses of power and greatly increasing operator transparency. The legislation required that contracts between charter boards and operators be posted on the Ohio Department of Education (ODE) website; that operators collecting more than 20 percent of a school’s funding provide a detailed statement of expenditures; that ODE post a simple directory of operators so the public could know which operators were affiliated with which charter schools—information surprisingly difficult to come by outside of inside charter circles; and that ODE publish an annual academic performance report for operators. These new provisions were at once somewhat obvious, yet revolutionary. Such is the Ohio charter story.
The new performance reports are out, and that’s a great step forward for Ohio where public information on operators has been historically lacking. But the reports are disappointing in their lack of depth and breadth. The image below shows one report in its entirety; fifty-three operators received a similar half-page report delineating academic performance, attendance, student demographics, and staffing data.
[[{"fid":"117775","view_mode":"default","fields":{"format":"default"},"type":"media","attributes":{"height":"359","width":"500","class":"media-element file-default"},"link_text":null}]]
Here are a few observations about the reports and where they could be improved.
Ohio evaluates sponsors in significant part based on their schools’ performance, and these evaluations include detailed information about schools’ academic results as well as compliance with various rules and laws. For operators, however—entities that are actually running schools day to day, and in some instances collecting more than 90 percent of schools’ public funds—there is very little information.
HB 2’s operator transparency provisions are necessary to provide valuable information to governing boards, sponsors, taxpayers, the public, and parents more broadly. Taken together, the newly available information on operators is a step forward for Ohio’s charter sector, and ODE deserves credit for creating the first operator performance reports and doing it on time. However, there is still much room to improve the report. In the interest of transparency, Ohio should move toward a much more robust and detailed 2.0 version.
One in seven adults’ ages 18-24 in Ohio lacks a high school diploma and faces bleak prospects of prospering in our economy. Dropouts earn $10,000 less each year than the average high school graduate according to the U.S. Census Bureau, are almost twice as likely to be unemployed, and typically earn an average annual income of $20,241 which hovers just above the poverty line for a family of three in Ohio. Dropouts also drag down the Ohio economy; over the course of their life, they consume an estimated $292,000 in public aid beyond what they pay in taxes.
To mitigate the number and cost of dropouts, Ohio has permitted the creation of ninety-four dropout prevention and recovery schools. Collectively, these schools enrolled sixteen thousand students in the 2015-16 year. They serve at-risk and re-enrolling students—pupils who previously dropped out but are now re-entering the education system—with the aim of graduating students who might otherwise slip through the cracks.
To hold these schools accountable for successfully educating at-risk students, Ohio has created an alternative report card. This report card assigns an overall rating of “Exceeds,” “Meets,” or “Does Not Meet” standards based on the school’s state assessment passage rate, graduation rate, ability to achieve progress from year-to-year, and the achievement gaps between student groups. Prior to 2012-13, dropout-recovery schools were rated on the same report-card indicators as all public schools.
Whether this new alternative accountability framework appropriately captures the success of these schools is up for debate. This past summer, a committee of legislators and civic leaders debated the definition of quality, heard from community members and school leaders, and reviewed the components of the current report card. The committee failed to recommend any changes (it had to meet a legislative deadline of August 1), though a new committee convened in November to continue this important work. (Disclosure: Fordham’s Chad Aldis has been named to this newly reconstituted committee.)
Should the committee choose to maintain the state’s recently created alternative report card, some adjustments are needed to ensure that high-performing dropout-recovery schools are distinguished from schools that continually fail to improve the learning of at-risk students.
Attention should be paid to one component in particular of the accountability rating—the progress measure that, generally speaking, gauges whether dropout-recovery students are making at least one year of academic growth. A very large majority of dropout-recovery schools appears to be falling short of growth expectations. In 2015-16, just seven schools exceeded the progress standards, eighteen met them, and an overwhelming majority of schools—sixty-nine of them—failed to meet the state’s standard for academic progress. In the previous year (2014-15), only one school exceeded standards, thirty-three met standards, and fifty-nine failed to meet the growth expectations. Given these results, the committee should review this measure’s methodology and confirm that the norm-referenced group used to calculate student growth along the NWEA’s Measures of Academic Progress test is appropriate for dropout-recovery students. Ohio law requires dropout-recovery schools to use a norm-referenced exam, not state exams, to gauge student growth over time. Ensuring that we accurately and fairly capture student progress, especially for pupils who may be years behind, should be a high priority.
Additionally, the way Ohio evaluates graduation rates should make certain that schools are not punished for taking in students who “drop-in” years after the expiration of their expected four-year graduation rate. (Four-year graduation rates, along with extended rates—up to eight years—are included in the alternative accountability system.)
Many also take issue with dropout-recovery schools being measured against the adjusted cohort graduation rate, as dropout-recovery schools face the consequences of a student’s previous school passing them on from one grade to the next without accomplishing adequate academic progress. A student’s transcript may report that they are in the eighth grade when they have really only mastered reading and math skills at the sixth-grade level. Yet, dropout-recovery schools are held accountable for graduating that student in four years. Testimony from school leaders during the summer’s dropout prevention and recovery school study committee emphasized the time crunch schools feel as soon as students who are academically far behind step through their doors. School leaders should not face perverse incentives to reject or rush students through the curricula because they are on the hook to meet four- or five-year graduation rates.
Finally, the performance standards should be adjusted to reflect high yet attainable standards. Currently, for dropout-recovery schools to attain a “Meets” graduation rate, they must graduate just 8 percent of all eligible students in four years. This standard is much lower than that of traditional schools, which must graduate 84 percent of seniors to earn a C rating and 93 percent to earn an A on the four-year graduation indicator. However, dropout-recovery school’s overall graduation rate standard could be set so low to account for the challenges these schools face in meeting typical adjusted cohort graduation rate timelines. Moving forward, the committee should evaluate the performance standards alongside their respective measures to ensure these are aligned and appropriately rigorous. Should the committee decide to phase in higher performance standards, they should also consider what supports, like re-engagement programs, could be implemented to help schools meet these targets.
As the new dropout-recovery committee works into 2017 to define what quality means for these schools, they should give thought to Ohio’s current system and explore ways to better distinguish high-performing from low-performing dropout-recovery schools. The quality of education for at-risk students, and by extension, Ohio’s long-term economic condition, is at stake.
