The Education Gadfly Weekly

On February 25, 2016, Ohio released report cards for the 2014-15 school year—the first in which the state administered next generation assessments. In conjunction with these new exams, state officials raised the minimum test score needed for students to be deemed “proficient.” As a result of these transitions, proficiency and achievement-based ratings fell across the state—a necessary reset of basic accountability measures in a time of rising expectations. This year’s report provides an overview of these changes, along with a presentation of data from national exams, suggesting that policymakers should go further to match Ohio’s definition of proficiency with a true college and career ready benchmark.

Since 2005, the Fordham Institute has conducted annual analyses of Ohio’s school report cards, with a particular focus on the performance of urban schools, both district and charter. This year’s analysis again takes a deep-dive look at the student achievement and school quality in the Ohio Big Eight areas. The key findings are as follows:

• College and career readiness rates are extremely low in Ohio’s high-poverty urban areas—in the Big Eight cities, roughly 10 to 25 percent of students are reaching rigorous benchmarks.
• According to the state’s achievement-based school ratings, urban schools almost universally receive low ratings (Ds and Fs). But when examining results from Ohio’s student growth measure (value added), variation in school quality emerges. Both urban charter and district schools receive high value added ratings, indicating the presence of schools that are helping students catch up with their peers.
• Still, too many students in urban areas are trapped in low quality schools (receiving poor ratings on both the performance index and value added). Taken together, approximately 150,000 students in Cincinnati, Cleveland, Columbus, and Dayton attend a low-quality school.

Download the report to learn more about the performance of Ohio’s public schools, statewide and in its eight largest urban areas.

National news outlets including Slate, Politico, Esquire, and the Washington Post have predicted that charter schools might be a growing thorn in Governor John Kasich’s side as he competes for the Republican presidential nomination. Kasich is being criticized for the overall poor performance of Ohio’s charter school sector, as well as for last year’s scandal over authorizer evaluations and its aftermath (including a hold placed on Ohio’s $71 million federal Charter School Program grant).

But by calling charters Kasich’s “little problem back home”—or, more boldly, claiming that his track record with them is “terrible”—national reporters are missing big pieces of the story. If these journalists had dug a little deeper, they would have realized that Kasich mostly deserves praise, not scorn, for the steps he’s taken to improve Ohio charter schools. In fact, any real examination of the candidate’s record on charters would reveal that no Ohio governor has worked harder to strengthen oversight of the charter school sector.

Kasich inherited a charter sector that was notorious for conflicts of interest, regulatory loopholes, self-dealing, and domination by powerful special interests. The mediocre performance of Ohio’s charter sector precedes Kasich’s tenure as well: CREDO’s 2009 charter study rated Ohio among the lowest-performing states.

In his first year in office, Kasich attempted to prohibit authorizers from opening new schools if they had any school rated the equivalent of D or F (a move that would have essentially frozen the sector entirely). This was later modified to prohibit the bottom one-fifth of authorizers from opening schools, based on a ranking system that, for the first time ever, evaluated them on the academic performance of their schools.

On the heels of two studies Fordham released pointing out the remaining weaknesses in Ohio charter law and sector performance, Kasich continued pushing for reforms. In remarks to the Ohio Chamber of Commerce in December 2014, he said, “We are going to fix the lack of regulation on charter schools. There is no excuse for people coming in here and taking advantage of anything. So we will be putting some tough rules into our budget.” His remarks were likely surprising to leaders in the Republican-controlled legislature, as well as for-profit charter school operators with a history of powerful influence at the statehouse.

It wasn’t empty rhetoric. Released two months later, Kasich’s budget bill outlined unprecedented reforms for charter schools and authorizers, many of which were embodied in the final reform bill that passed with overwhelming bipartisan support in October 2015. The legislation is significant: It builds on the authorizer performance system that Kasich had begun building in 2011, aligning incentives so that high-performers will be rewarded and low-performers will be shuttered. It eliminates conflicts of interest and installs new transparency measures for governing boards and operators. Credit also belongs to several lawmakers for seeding reform ideas and getting the legislation across the finish line—but it’s indisputable that Kasich set the tone and put pressure on lawmakers to get the job done.

Kasich also has been slammed for the actions of former Ohio Department of Education official David Hansen, who was charged with implementing the world-class authorizer evaluation system. Hansen made a serious error in judgment by excluding the failing grades of online charter schools from authorizers’ evaluations and resigned shortly after the omission came to light last July. Despite zero evidence that Kasich was aware of Hansen’s actions, many believe he’s guilty by association. (Hansen is married to Kasich’s campaign manager.)

However, the education department during Kasich’s administration has played a far more active charter oversight role than ever before. In 2013, the governor’s hand-picked state superintendent ordered the closure of charter schools for egregious health and safety violations; the next year, the department investigated underperforming authorizers and prevented them from opening poorly vetted schools. By the end of 2015, a new authorizer evaluation system with more emphasis on academic performance was approved.

Critics have also pointed to the political sway of Ohio’s academically struggling e-schools, exercised through sizeable donations to Republicans. Politico, as evidence of influence, points out that Kasich was responsible for lifting the state’s moratorium for online schools. But that move enabled competitors to enter the market, which didn’t exactly benefit the powerful e-school oligopoly. Since then, three new online schools have opened, subject to an additional layer of approval by the department. The law has also capped the annual growth of existing Ohio e-schools. Most importantly, the state is currently holding the line against the anti-reform efforts of the e-school lobby more than at any other point in our charter history. Recent efforts to water down accountability are being met with strong resistance, evidence that the power of Ohio e-schools is dwindling.

Finally, no Ohio charter school discussion would be complete without mentioning funding. While charter detractors might suggest that Ohio charters made money hand over first under Kasich’s administration, such claims are bogus. State funding for charters expanded overall during his tenure, but that’s primarily because the number of students enrolled in charter schools has increased. The base state allotment for charter schools has gone up in precisely the same manner it has risen for district schools. Charters, deprived of access to the state’s school facilities program and denied the ability to raise funds locally, were awarded a meager $150 per pupil toward facilities and may apply to a competitive $25 million facilities state grant if they rank among the top performers. That’s hardly pork barrel politics at work.

There are many other aspects of the governor’s education record that most school reformers would laud. Kasich was instrumental in bringing Teach For America to the state and was at the helm when Ohio designed award-winning report cards; he also improved learning standards, raised the bar on proficiency, expanded a program to help high schoolers earn college credit, and created an innovation fund. In sum, Governor Kasich has been a powerful advocate of charter school reform in a state whose sector has long been deeply troubled, and it’s shameful that the national media has created the opposite impression. In fact, Ohio could become a leader on charter quality if the latest reforms are implemented well. Rather than haunting his political career, the turnaround of Ohio’s charter sector could be another example of Kasich beating the odds. 

[Editor’s note: This is the fourth post in a series on improving teacher preparation programs. See here, here, and here for prior ones.]

Millions of families in America depend on education as a pathway toward upward mobility. We owe it to these families and their students to provide highly trained teachers who are ready on the first day. Unfortunately, way too many teachers learn how to teach during their first year in the classroom instead of before it. For example, of all the preparation programs examined in NCTQ’s most recent Teacher Prep Review, not a single one met the standard for effectively training teachers to plan lessons. Only 11 percent of programs met the standard in classroom management techniques. Student-teaching is the only real clinical experience that many teacher candidates receive, yet only 10 percent of programs met NCTQ’s standard for a strong student-teaching experience. In short, most preparation programs are doing a lackluster job of teaching their candidates how to teach.

Education schools are often hesitant to focus on clinical training because it seems too similar to vocational training. Instead, they spend considerable amounts of time on education theory, philosophy, and personal reflection. These are valuable pursuits, to be sure, but a competent teacher is one who can effectively teach. Teacher preparation programs would be much better off replicating on-the-job learning while candidates are still enrolled in their programs. That means not just teaching how to write a lesson plan, but requiring candidates to actually teach the lesson plan, reflect on its strengths and weaknesses, modify it, and then teach it again. It means not just reading about how to manage a classroom, but actually practicing strategies for engaging sleepy or bored kids, responding to minor and major misbehaviors, and dealing with emergencies. It also means that the things a first-year teacher usually learns on the fly—how to determine grades; how to effectively differentiate instruction; how to manage makeup work, students who finish early, and students who need extra time—should be explicitly taught and practiced. Teaching candidates may not be full-fledged teachers, but that doesn’t change the fact that they can benefit from the same type of development as currently teaching teachers. It’s not just about acquiring knowledge and skills, it’s about practicing what they learn, studying the effects, and then adjusting. Current teachers benefit immensely from coaching, and it stands to reason that the same is true for teacher candidates.

All of these skills shouldn’t just be taught and practiced in a clinical setting. They should also be evaluated based on competency instead of time served. Doctors don’t fill out anatomy charts and then start treating patients, and pilots don’t recite plane parts and then climb into the cockpit. Teaching is no different. Competency means that teachers must demonstrate that they can do the core work of teaching—intentional planning, effective execution, and building relationships—before they’re allowed to take charge of a classroom. Student-teaching should be the culmination of years of clinical practice graded on competency—not a candidate’s first foray into the classroom.

To make competency-based clinical training a reality, teacher preparation programs need access to schools with kids. Teacher candidates need plenty of opportunities, starting in their first year, to do the work of actual teachers rather than just reading about and observing it. A recent report from Education First examined partnerships between school districts and preparation programs and offered recommendations for how to shape teacher pipelines through collaboration. This kind of partnership is the key to unlocking competency-based clinical training. Here’s an overview of a potential competency-based, clinical model for teacher preparation.

Years 1–3: Working with hybrid teachers

I’ve written before about hybrid teachers—those who instruct students and hold additional leadership responsibilities. One such responsibility could be working with a group of teacher candidates. It’s a win-win for all participants: Highly effective teachers stay in the classroom part-time and continue to impact kids while simultaneously impacting teaching candidates; candidates absorb the knowledge, experience, and coaching of current teachers; and teacher preparation programs offer the kind of real-world, hands-on experience that they often get criticized for neglecting. After all, nobody offers real classroom experience better than teachers who currently teach.

With that in mind, I imagine a program that looks something like this: During candidates’ first three years in a preparation program, their fall semesters are spent completing traditional college courses, including those related to subject content, education theory, philosophy, current research, and policy. For their spring semesters, small cohorts of candidates are paired with a highly effective hybrid teacher from a local district (rather than a teacher from a lab school). Two days each week, candidates continue to take traditional college courses, including one taught by the hybrid teacher on basics like lesson planning, data analysis, assessment, and behavior management. The remaining three days are spent in the classroom with the hybrid teacher, interacting with children and practicing what they’ve learned.

The responsibilities given to candidates follow a gradual release model and align with what they are learning. For instance, while learning about lesson planning, candidates craft a mini-lesson plan and co-teach it with the hybrid teacher. Afterwards, the hybrid teacher and other candidates offer feedback, which the candidate uses to revise the lesson. Eventually, candidates are solely responsible for delivering a full lesson. When learning about classroom management, candidates could start with one-on-one tutoring and advance to small group projects that require them to engage kids and keep them on task. Learning about IEPs and differentiation would allow candidates to plan for and lead small group remediation or enrichment, and learning to analyze data could occur after unit tests or exams. Each of these exercises would culminate in a performance-based assessment that measured competency rather than completion.

Candidates could gain additional experience and knowledge by monitoring lunch and recess, taking part in parent-teacher conferences, and assisting with field trips and extracurriculars. While all of these experiences are vitally important to the development of candidates, they also represent a key support system for schools and students—candidates become an instrumental part of school operation and provide students with additional mentorship and academic support.    

Year 4: Teacher residency

During their fourth year of the teacher preparation program, rather than taking part in traditional student-teacher teaching, candidates complete a teacher residency. Teacher residency programs are based on the model that doctors use; residents spend a year embedded in a school, co-teaching alongside a teacher of record and gradually taking on full ownership of certain aspects of the classroom. While there are several residency programs across the country (including programs in Boston, Memphis, Fresno, and Los Angeles), most of these programs are post-baccalaureate partnerships that offer a teaching credential to candidates who haven’t completed a traditional teacher preparation program. In my proposed model, residency would target candidates passing through traditional programs rather than those interested in alternative certification.

After three years of coursework and significant experience in a hybrid teacher’s classroom, candidates in this program would be better prepared for residency than traditional candidates are for student-teaching. By the time the fourth-year residency finishes, these aspiring teachers would have far more hands-on experience than graduating traditional candidates. This extra experience makes them extremely attractive to districts looking to hire new teachers—not just because they can hit the ground running, but also because they are more likely to understand the demands of the profession and thus less likely to quit after a few years. Even better, the sink or swim mentality that characterizes the first year of teaching would cease to exist; candidates would master important skills before they became official teachers, which could improve student achievement, overall teacher quality, and teacher retention.

***

Implementing this model will undoubtedly be difficult. For starters, it would likely be more expensive than traditional training. Teacher preparation programs would have to rework course requirements in order to ensure that candidates aren’t bogged down by courses that don’t meaningfully contribute to their development. Selecting the right hybrid and resident teachers would be also hugely important, since their tutelage could make or break the development of candidates. Lastly, getting districts and teacher preparation programs to collaborate could be a heavy lift. But the payoffs would be massive, particularly for the students who would benefit from amply prepared first-year teachers. Preparation programs looking to improve their development of candidates should consider piloting this model with a small cohort of willing candidates—and closely studying the results. 

Last year, a few early childhood advocates blasted the Common Core State Standards for their “harmful” effects on kindergarteners, particularly in reading. While a careful examination of the standards reveals this claim to be overstated, the notion that we are killing kindergarten was gaining traction long before Common Core came onto the scene (2010 and thereafter). Until now, this narrative has been informed largely by anecdotal evidence, idealism, and good old-fashioned nostalgia. Noting that “surprisingly little empirical evidence” has been gathered on the changing nature of kindergarten classrooms, this paper attempts to fill the void by comparing kindergarten and first-grade classrooms in 1998 and 2010—capturing the changes in teachers’ perceptions of kindergarten over time.

Using the Early Childhood Longitudinal Study, researchers compared survey response data from public school kindergarten teachers in 1998 and 2010 to investigate changes across five dimensions: teachers’ beliefs about school readiness, curricular focus and use of time, classroom materials, pedagogical approach, and assessment practices.

Overall, researchers found that kindergarten has become more like first grade. When asked to rate the importance of thirteen school readiness skills, 2010 teachers tended to rate all of them as more important than their 1998 counterparts had. This was true for academic skills (identifying letters, counting to twenty) and non-academic ones (being “sensitive to others’ feelings,” problem solving). The most striking change in beliefs was related to reading: only 31 percent of 1998 teachers believed that children should learn to read in kindergarten; by 2010, that figure rose to 80 percent. In short, teachers expect more from kindergarteners than they did in the 1990s—not necessarily a bad thing given that their expectations have an enormous influence on student behavior and achievement.

However, the findings also point to an unfortunate narrowing in curriculum. While the number of teachers conducting daily reading and math instruction didn’t shift significantly, the percentage of those teaching music and art each day went down (by eighteen and sixteen percentage points, respectively). It’s not that music or art disappeared altogether; it just happened less often. An overwhelming majority of teachers in 2010 still reported teaching music and art weekly. (On a positive note, more teachers in 2010 than in 1998 reported that children had daily recess.) Kindergarten teachers also reported an increasing reliance on didactic instructional activities (use of worksheets, workbooks, and textbooks) as well as fewer centers for hands-on learning (particularly water or sand tables, art stations, and dramatic play and science areas). Schools serving more low-income and non-white children were more likely to use didactic instruction and less likely to provide hands-on learning opportunities.

Several looming questions remain and point to the need for further study. For one thing, there’s an ongoing debate regarding how much academic content is appropriate for kindergarteners—some argue that too much is damaging, while others posit that exposure to advanced content in kindergarten can benefit students. Given the wide range of developmental maturity among kindergarten-age students, it seems both imprecise and unproductive to generalize in either direction. (Instead, educators and policy makers may want to explore solutions, like a transition year between kindergarten and first grade or competency-based groups instead of age-based ones, to help mitigate concerns that less developmentally mature children are harmed by increasing rigor.) Additionally, the paper briefly mentions that “low-performing teachers in high-stakes grades are disproportionately reassigned to untested early elementary classrooms.” This trend deserves urgent attention because low-income children can’t afford underperforming teachers in the early grades. Finally, the report alludes to—and rightly rejects—the notion that academically oriented classrooms crowd out exploration, social skill development, and play. In fact, high-quality kindergarten teachers do both—yet low-income students, again, are more likely to receive poorly structured pedagogy and be ultimately shortchanged.

Today’s kindergarten classrooms are increasingly rigorous. Unfortunately, youngsters who come to school unprepared are at an even greater disadvantage than in the past. The study reminds us of the importance of high-quality, targeted preschool, as well as teacher policies that ensure that kindergarten students—especially those with skills deficiencies—don’t get short shrift.

Source: Daphna Bassok, Scott Latham, and Anna Rorem, “Is Kindergarten the New First Grade?” American Educational Research Association (January 6, 2016) 

If you’re reading this, you are probably a subscriber to the Ohio Gadfly Biweekly email newsletter. Kudos. You have excellent taste in email newsletters.

We love an in-depth analysis and an insightful review just as much as the next person, but there’s more to Fordham Ohio’s work than just what you see in the biweekly Gadfly. (Hard to believe, I know.) Check out the Ohio Gadfly Daily for additional coverage, such as:

• The Proper Perspective: Fordham’s own Jamie Davies O’Leary discusses education issues with Stephen Dyer, an education policy fellow at Innovation Ohio, in a point/counterpoint style that helps provide insight and understanding. Recent topics include testing opt-outs and the Third Grade Reading Guarantee. Best of all, there’s a forum where you can weigh in too.
• Developing news and rapid analysis: Sometimes a news item or development in education demands a quick and concise response from Ohio’s education gadflies. The same analysts who produce long-form research and commentary also bring their insight to these “quick-hit” blog posts as developments warrant. Recent examples include a look at Ohio State University’s creditable efforts at closing the black/white college attainment gap and a first glimpse at long-awaited state report card data.
• Education news clips: Three times each week, we scour news outlets across the Buckeye State for education news and opinion, collating, and annotating them to bring you an irreverent snapshot of education voices. It’s called Gadfly Bites—read a few and you’ll understand why. It’s posted on Mondays, Wednesdays, and Fridays, and you can even subscribe to it and have it delivered right to your inbox.
• Events and videos: Missed seeing one of our Ohio events live? You can find wrap-ups and photos on our daily blog and videos on our YouTube channel. Mike Petrilli’s “Education for Upward Mobility” event, held in Columbus on March 23, can be viewed in its entirety here.

So you see, while we are the Ohio Education Gadfly, we are so much more than that. Check out Ohio Gadfly Daily today.

Regular Gadfly readers know that we usually rely on two metrics when analyzing school performance—Ohio’s performance index and the value added measures. However, the state assigns A–F ratings along several other measures, including one called the “gap closing” component (a.k.a. annual measureable objectives, or AMOs). This measure deserves scrutiny now that state policy makers have the opportunity to retool the accountability system under the Every Student Succeeds Act of 2015 (ESSA).

First implemented in 2012–13 as a modification to the federal Adequate Yearly Progress (AYP)provisions, AMOs are meant to hold schools accountable for the proficiency of student subgroups (e.g., low-income or race/ethnicity). Specifically, the measure compares the proficiency rates of a school’s subgroups to statewide proficiency targets—the “measureable objective.” The AMO methodology also gives partial (up to full) credit to schools when subgroup proficiency increases from year to year. The idea of AMOs is to maintain pressure on schools to close longstanding gaps between low-achieving subgroups and their peers.

Shortly after ESSA passed, the U.S. Department of Education notified states they were freed from using AMOs in their accountability systems. That doesn’t mean that low-achieving students will be forgotten: The new federal law requires an “indicator of proficiency” split out by subgroup and based on “ambitious, State-designed long-term goals.” But the law does not set forth specifications around the indicator’s design nor does it precisely define “proficiency.” With new leeway on how to implement this measure, state policy makers should now consider what to do with AMOs—an important decision, since AMOs are currently a high-profile A–F graded component and slated to become a significant part of Ohio’s overall rating system.

Should Ohio continue with AMOs, modify them, or discontinue them altogether? If seeking a different path, what are some alternatives?

The case against AMOs

As relics of the bygone No Child Left Behind era, the first step state policy makers should take is to scrap AMOs and start over. Let’s review three flaws of the AMO measure as presently implemented in Ohio.

First, they rely on proficiency rates—the percentage of students reaching an acceptable level of achievement as determined by the state. In high-stakes accountability, there are serious problems with relying too heavily on proficiency rates: For example, researchers in Chicago found evidence that proficiency-based accountability promoted a stronger focus on students near the proficient threshold, at the expense of very low- or high-achievers. Due to problems such as these, in Fordham’s ESSA Design Competition, no one suggests using simple proficiency rates in school accountability; most recommend a performance index (a weighted measure of achievement) or using scaled scores.

Second, using year-to-year changes in proficiency, as AMOs do, is problematic. For example, let’s suppose a school (call it Lincoln Elementary) in which 40 percent of low-income students were proficient in 2013–14, while 45 percent met that bar in 2014–15—a five-point increase. Should we understand this increase as legitimate improvement, even “gap closing,” or has the composition of the subgroup simply changed? At Lincoln Elementary, what if the increase in proficiency was due an influx of high-achieving, low-income kids? Because a school’s subgroup composition can change, researchers have urged caution when interpreting such changes in proficiency. We don’t know whether one-year increases in subgroup proficiency (reported at the school level) are evidence of real improvement—or just a mirage.[1]

Third, high-poverty schools—even when they earn exemplary scores on Ohio’s student growth measure (value added)—overwhelmingly receive failing grades on the AMO rating. Consider the following table, which displays the AMO ratings of high-poverty schools that earn value-added index scores within the top 10 percent of the entire state. On this list, you’ll notice that thirty-four out of the thirty-eight schools (89 percent) received F ratings on AMOs. If we believe—as many researchers do—that value added is a truer gauge of school effectiveness, why do so many high-performing, high-poverty schools receive Fs on AMOs? Are these schools “widening the achievement gap”? Sadly, even when they contribute positive learning gains (which, if accumulated over time, would narrow the achievement gap), they are deemed as failing AMOs.[2]

Table: AMO ratings of high-performing, high-poverty schools in Ohio, 2014–15.

[[{"fid":"115857","view_mode":"default","fields":{"format":"default"},"type":"media","link_text":null,"attributes":{"height":"728","width":"752","style":"width: 500px; height: 484px;","class":"media-element file-default"}}]]

Source: Author’s calculations based on data from the Ohio Department of Education. Notes: “High-poverty” schools have greater than 80 percent economically disadvantaged (ED) students. To be included as a “high-performing” school, its value-added index score must be within the top 10 percent statewide (total number of rated schools in Ohio = 2,520). Charter schools are italicized. For reference, the statewide distribution (school-level) of A-F ratings for AMOs is as follows: A-20%; B-15%; C-8%; D-9%; F-48%. The distribution of A–F ratings for high-poverty schools (greater than 80 percent ED) is as follows: A-5%; B-4%; C-2%; D-3%; F-86%.

Ideas for improvement

In my view, AMOs are unsalvageable. Relying purely on a proficiency measure in a high-stakes accountability system narrows the focus to kids on the cusp of passing ; year-to-year changes in proficiency may be no more than an illusion of improvement (and as my colleague Robert Pondiscio suggests, incentivizes poor instruction, at least in reading); and something is amiss with the AMO methodology the state currently uses. In short, it’s not clear that AMOs are accomplishing the important function they were designed to serve—accurately telling us which schools are serving our most disadvantaged students well and which aren’t. To be clear, state policy makers are not entirely responsible for these problems: Recall that AMOs are part of the former federal accountability system. But it’ll take their action to make revisions to state law and undo AMOs.

Subgroups still matter. Not only does ESSA still require some type of “indicator” that breaks out achievement results by subgroup, it’s also critical that Ohio never go back to a time when the academic achievement of any subset of students could be ignored. What to do in a post-AMO world? Here are four thoughts.

• Implement a subgroup performance index. If allowed by federal officials (and it should be), Ohio ought to disaggregate subgroup achievement using the state’s performance index (PI). As a measure that takes into account students across the entire achievement distribution, PI alleviates some of the problems associated with focusing strictly on the proficiency bar. In the ESSA design competition, Josh Boots of Empower K12 puts forward the notion of an “achievement gap index” that would essentially assign PI scores for each school’s subgroup and then compare that score to a statewide benchmark. Ohio should pursue a measure such as this.
• Create reasonable statewide goals for subgroups. Due to its harsh design, AMOs doomed virtually all high-poverty schools to failure—even those contributing well over a year’s worth of learning. It will be important for policy makers to set rigorous but attainable statewide goals for the achievement and continued improvement of disadvantaged subgroups—goals that deserving schools can meet and exceed.
• Don’t make subgroup achievement a standalone A–F component. Ohio’s report cards already have two achievement-based A–F ratings that typically reveal gaps between schools (the performance index and indicators met, with a third—a composite of the first two—soon to come). A separate graded component for subgroup accountability, such as AMOs, may unnecessarily pile on the Ds and Fs for schools serving disadvantaged populations. Policy makers should consider implementing ESSA’s “subgroup indicator” as a non-graded component, or placing the subgroup indicators within the state’s larger achievement component. The point is not to conceal subgroup achievement, but rather to ensure the report card system is fair to schools.
• Resist the urge to evaluate schools on “gap closing.” Attempting to evaluate whether a school closes gaps is fraught with peril. The state’s current measure of gap closing (AMOs) is imperfect, and its results are perplexing. But a wholly different concern is that we might be pitting one subgroup against another, especially if we use a higher-achieving group as a benchmark. (It goes without saying that one perverse way to narrow the gap—particularly at the school level—is to depress the achievement on the top end.) The most prudent path for Ohio would be to avoid measures that explicitly aim to measure gap closing. Those measures invariably suggest that education is a zero-sum game, but we know that all students can and should achieve at higher levels.

Breaking down achievement (and growth data) by subgroup is an important dimension of a school evaluation system. Overall averages shouldn’t be used to hide the low performance of certain groups of students. Moreover, subgroup data can reveal a school’s strengths and weaknesses—a foundational element for school improvement. But accountability at a subgroup level must be done carefully and thoughtfully as well. Now that federal law puts Ohio firmly in charge of its own accountability system, policy makers should use this opportunity to refine policies in a way that makes even better sense for Buckeye schools and students.

Social Impact Bonds (SIB), also known as “pay for success” loans, are a novel form of financing social service interventions, including education initiatives. First piloted six years ago in the United Kingdom and now making their way to the U.S, SIBs aim to leverage private funding to start new programs or scale proven ones. Broadly speaking, the instrument works like this: Private lenders and philanthropists deliver dollars—the bond—to a nonprofit provider that, in turn, implements the intervention. A government agency pays back the bond principal with interest, but only if the program achieves pre-specified results.

In its ideal form, an SIB has the potential to be a triple win: Governments receive risk-free funding to test or expand social programs that could help them save money; investors reap a financial return if the program works; and providers gain access to new sources of funding. To ensure the deal will benefit all parties, due diligence occurs on the front end, including selecting a program provider, estimating government savings, and developing an evaluation method. 

To date, the discussion on SIBs has been largely conceptual, engaging both supporters and skeptics alike. But a fascinating new report written by MDRC President Gordon Berlin provides a first-hand look at an early SIB experiment. In July 2012, Bloomberg Philanthropies, Goldman Sachs, MDRC, and New York City teamed up to create a four-year, $9.6 million SIB that funded a behavioral therapy program at Rikers Island jail. The objective was to reduce adolescent recidivism by at least 10 percent. The deal was structured in such a way that after three years (summer 2015), the program evaluators would determine whether to continue into a fourth year or pull the plug.

The program, however, fell short of its goal and folded early. Using quasi-experimental methods, independent evaluators found that the intervention did not reduce recidivism for its participants. Berlin describes the implementation challenges, among them the harsh prison environment and hurdles to program completion (just 9 percent of participants completed the full intervention). The null findings resulted in Goldman taking a loss of about $1.2 million (Bloomberg’s grant protected the bank from absorbing the entire loss). Meanwhile, the New York City government was freed from payment: Under the SIB terms, it would have paid back the principal if the program met the 10 percent threshold—and more (principal plus interest) if the reduction had been greater than 10 percent.

Based on this experience, Berlin provides helpful lessons for those contemplating SIB financing. To highlight just a few: First, he notes that SIBs, as currently structured, tend to carry more risk on the lenders’ side. SIBs are commonly conceptualized as high-risk/high-return “venture capital” for the social service sector. Yet the “bond-like returns” governments may be willing to pay aren’t commensurate with the risk that SIB investors have to bear, thus limiting their viability. Second, Berlin discusses the incredible intricacy of SIB deals and how the “tyranny of inflexible loan agreements” can stifle on-the-ground flexibility. For instance, the Rikers Island program found it more difficult than expected to recruit participants; consequently, the parties had to negotiate changes in the already complex SIB contract. He concludes, “Each SIB will confront this tension between inflexible contract terms and the need to respond to unexpected operational challenges.”

This report is important reading for anyone interested in SIBs, including their potential use in financing education interventions like pre-K or scaling high-quality schools. (Utah is presently using an SIB to fund pre-K for 595 low-income children, with the aim of reducing special education placements in elementary school.) To be sure, SIBs hold the potential to unlock non-traditional financing to help grow and replicate innovative programs; they also rightly focus on results and rigorous evaluation—features not traditionally emphasized in the public sector. But as this report acknowledges, such arrangements must be done thoughtfully and with a strong dose of realism. Berlin observes, “Reality is turning out to be more nuanced than either proponents or detractors have promised.”

Source: Gordon L. Berlin, “Learning from Experience: A Guide to Social Impact Bond Investing,” MDRC (March 2016).