KIPP Columbus achieves extraordinary outcomes for its students, predominantly students in poverty and students of color. Led by Hannah Powell and a visionary board, the school has a rare knack for forging powerful partnerships at every turn—ones that strengthen KIPP students, their families, and the entire community near its campus.
KIPP Columbus achieves extraordinary outcomes for its students, predominantly students in poverty and students of color. Led by Hannah Powell and a visionary board, the school has a rare knack for forging powerful partnerships at every turn—ones that strengthen KIPP students, their families, and the entire community near its campus. We invite you to read this profile of Steve, a KIPP graduate, an immigrant and first-generation college student now attending Vanderbilt University. Steve’s entire family has been uplifted by the school and his story shows powerfully what is really possible in a high-quality charter school.
Eleven weeks ago, in High Stakes for High Achievers: State Accountability in the Age of ESSA, the Fordham Institute reported that current K–8 accountability systems in most states give teachers scant reason to attend to the learning of high-achieving youngsters. We coupled that bleak finding with a reminder that the new federal Every Student Succeeds Act (ESSA) creates a rare opportunity for state leaders to rethink their accountability systems and thereby set matters right.
Now we’re back with a companion paper, High Stakes for High Schoolers, which appraises state accountability regimes as they affect high-achieving students in high school. We examined states’ current (or planned) accountability systems and rated them based on whether they incorporate under ESSA the following principles to incent schools to prioritize high achievers:
We find that most current (and planned) state accountability systems provide high schools with few incentives to focus on their high-achieving students. In fact, our analysis indicates that just four states—Georgia, Ohio, Pennsylvania, and Texas—have truly praiseworthy systems when it comes to focusing attention on these students, though four others—Alabama, Idaho, Louisiana, and New York—are clearly moving in the right direction in their proposed frameworks.
See the tables below for two sets of ratings: one for the thirty-nine states (plus the District of Columbia) that calculate (or intend to calculate) summative school ratings and one for the eleven states that don’t (or don’t plan to) take this step.
Click on each state’s name for an analysis of its high school accountability system as it pertains to high-achieving students.
 |
Idaho*, New York*, Ohio |
 |
(None) |
 |
California*, Kansas, New Jersey, Tennessee |
 |
Maryland, Montana, North Dakota, South Carolina |
*Indicates that a state's rating was based upon a planned system.
*Indicates that a state's rating was based upon a planned system.
Fortunately, states now have an opportunity to put America’s schools on the right path. It will take leadership and courage, however, as naysayers will always insist that any attention given to high achievers is inherently elitist. They are wrong. There are hundreds of thousands of American teenagers ready to work harder, reach higher, and go further, if only we give them the chance. Many are children of color and come from poor families. They deserve our attention. State accountability systems can send strong signals about who matters. The right answer is everyone—including high achievers.
Amendment: Due to a data oversight, West Virginia's rating was initially represented as one star out of four in this report's initial publication. As of November 16, 2016, all materials associated with this report have been updated to reflect West Virginia's ranking of two stars for its proposed framework under ESSA. We regret the error.
KIPP Columbus achieves extraordinary outcomes for its students, predominantly students in poverty and students of color—a fact worth celebrating by itself. In 2015-16 in Ohio’s Urban Eight cities, KIPP Columbus was in the top five percent of all schools (district and charter) on student growth and among the very best (top 3 percent) in Columbus. But it’s not just KIPP’s academic data that are impressive. KIPP Columbus, led by Hannah Powell and a visionary board, has a rare knack for forging powerful partnerships at every turn—ones that strengthen KIPP students, their families, and the entire community near its campus. This year, KIPP launched an early learning center in partnership with the YMCA of Central Ohio to serve infants, toddlers, and pre-school aged youngsters. In a neighborhood lacking high-quality childcare and early learning opportunities, it’s an investment not just for KIPP students, but for the community at large. KIPP Columbus also partners with the Boys and Girls Club of Columbus, Battelle Memorial Institute, and other community organizations.
This profile is about KIPP graduate Steve Antwi-Boasiako, an immigrant and first-generation college student now attending Vanderbilt University, whose entire family has been uplifted by the school. His story illustrates the depth of KIPP’s commitment to students’ long-term success. The “KIPP Through College” program tracks data on what colleges turn out to be a good fit for students (many of whom are first-generation college-goers), assists families with applications and financing information, and even partners with 83 colleges and universities nationally so that cohorts of KIPP students improve their odds of successful completion. In a world where just nine percent of low-income students attain a four-year college degree, KIPP students’ attainment rate (44 percent) is truly remarkable, and even ten percentage points higher than that of the population at large (34 percent).
Steve was an inspiration to younger students while attending KIPP: Columbus;
he now attends Vanderbilt University
We urge you to read Steve’s story. In it, KIPP Columbus’s mission to “prove what’s possible” truly comes to life. Indeed, his remarkable trajectory reminds us that not only will students meet our expectations when we place them high, but will often exceed them beyond our wildest hopes.
The Every Student Succeeds Act (ESSA) has put the future of teacher evaluations firmly in the hands of states. Ohio is now in full control of deciding how to develop and best implement its nascent system.
It should come as no surprise to folks in the Buckeye State that the Ohio Teacher Evaluation System (OTES) has significant room for improvement. Since its inception in 2009, approximately 90 percent of Ohio teachers have been rated in the top two categories and labeled “skilled” or “accomplished.” Unfortunately, there isn’t significant evidence that the system has impacted the quality of Ohio’s teacher workforce, perhaps because there is no statewide law that permits administrators to dismiss teachers based solely on evaluation ratings. Meanwhile, OTES also doesn’t appear to be delivering on the promise to aid teachers in improving their practice.
A quick glance at the ODE-provided template for the professional growth plan, which is used by all teachers except those who are rated ineffective or have below-average student growth, offers a clue as to why practice may not be improving. It is a one-page, fill-in-the-blank sheet. The performance evaluation rubric by which teachers’ observation ratings are determined doesn’t clearly differentiate between performance levels, offer examples of what each level looks like in practice, or outline possible sources of evidence for each indicator. In fact, in terms of providing teachers with actionable feedback, Ohio’s rubric looks downright insufficient compared to other frameworks like Charlotte Danielson’s Framework for Teaching.
Another problem with OTES is the way it incorporates student learning. Ohio law requires that all teacher evaluations include a student growth component, which consists of test results. For teachers with a valid grade- and subject-specific assessment, that means value-added measures. Unfortunately, only 20 percent of Ohio teachers are able to be measured using the state assessment either in whole or in part.[1] Another 14 percent receive growth scores from a separately administered vendor assessment that increases the testing burden placed upon students and schools. The student growth component for the remaining 66 percent is based on locally developed measures that tend to be both ineffective and unfair: shared attribution, which evaluates teachers based on test scores from subjects they don’t teach, and Student Learning Objectives (SLOs), which are extremely difficult to implement consistently and rigorously and often fail to effectively differentiate teacher performance. In short, the state hasn’t quite figured out how to fairly evaluate all teachers using student achievement data.
A meaningful overhaul of Ohio’s system should aim to solve four significant problems. First, it should address the current framework’s failure to fairly evaluate all teachers. Second, it should do a far better job of differentiating teacher performance. Third, it should provide actionable feedback to all teachers. And finally, it must positively impact the overall quality of the workforce. Crafting a system that does all this is easier said than done. Fortunately, there’s evidence that focusing solely on a rigorous classroom observation cycle, rather than student growth measures, could be the solution.
In a recent piece for The 74, Matt Barnum examined research on teacher evaluation work in Chicago, including an analysis of a pilot system that focused solely on classroom observations and the system’s impact on the labor market. The analysis found that the first year of the pilot resulted in an 80 percent increase in the exit rate of the lowest-performing teachers; the teachers who replaced exiting educators proved to be higher performing than those who exited. Overall, the findings suggest that evaluation systems based solely on rigorous observations of teacher practice can impact the quality of the workforce. This type of system would also remedy the biggest problem with Ohio’s evaluation structure, which is that current student growth measures unfairly evaluate teachers in many subjects and grade levels.
The second and third problems with the current system—effectively differentiating teachers and offering better feedback—are also improved by zeroing-in on an improved observation cycle. In general, observations provide more detailed information about the complex job of teaching than a list of raw scores ever could. More information means more opportunities to pinpoint variances in performance, but only if the system uses a high-quality rubric and takes advantage of multiple perspectives by including outside observers and peer observers. Improving observer training and ensuring a mix of announced and unannounced observations is also important.
When it comes to offering better feedback, it’s widely acknowledged that teachers find evaluations most helpful when they're given actionable feedback on their practice. This type of feedback only comes from observation of practice. Plenty of other sectors understand this. Professional football teams prepare for their next opponent by studying game film. Players study their future opponents, but they also study their own performance from the previous game—the choices they made, what they could have done better, and what they need to continue doing. Teacher evaluations should offer the same opportunities. Teacher coaching, teacher collaboration (which research says can lead to student achievement gains), and peer reviews—all of which have been found to improve teacher practice—are only effective if they include rigorous observation of practice.
It’s true that assessment results are a form of feedback. But as a former teacher, I can attest to the fact that studying test results (value-added or otherwise) was hardly the most effective way for me to improve my pedagogy. I needed to know why my students did or didn’t do well, and that answer couldn’t be found on a data spreadsheet no matter how hard I looked. A far better use of my time and the best way to make me a better teacher faster would have been actionable feedback that came from observing my practice—which, after all, is what most impacted my student’s test scores in the first place.
In summary, research shows that evaluation systems based solely on rigorous observations of teacher practice can impact the quality of the teacher workforce. Research also shows that improving teacher practice can be done through observations conducted by well-trained observers using high-quality frameworks and rubrics. Taken together, it seems that one way to improve Ohio’s teacher evaluation structure is to pilot a system that focuses solely on rigorous classroom observations. Stay tuned for a detailed explanation of what such a system could look like.
[1] The 20 percent is made up of teachers whose scores are fully made up of value-added measures (6 percent) and teachers whose scores are partially made up of value-added measures (14 percent).
Many prior studies have found that low-income students have less qualified teachers based on measures such as years of teaching experience, teacher licensure test scores, certification status, and educational attainment, but they say very little about how these differences relate to closing the achievement gap, nor do they examine the magnitude of how differences in access to effective teachers might impact performance.
Yet a new Mathematica study is full of surprises. It examines low-income students’ access to effective teachers in grades four through eight over five years (2008–09 to 2012–13). “Low income” is defined as being eligible for free and reduced-price lunch (FRPL), and “high income” includes everyone else (so not much nuance there). The sample includes twenty-six geographically diverse, large school districts across the country, with a median enrollment of 70,000. And analysts measure the effectiveness of each teacher in the district using a value-added model.
There are five key findings.
First, contrary to conventional wisdom, teachers of low-income students are nearly as effective as teachers of high-income students on average (a difference of one percentile point). Specifically, the average teacher of a low-income student is just below the fiftieth percentile, while the average teacher of a high-income student is at the fifty-first percentile.
Second, high- and low-income kids have similar chances of being taught by the most and least effective teachers. For example, 10 percent of both high and low income kids are taught by one of the top 10 percent of teachers in a district.
Third, teachers hired into high poverty schools are equally effective as those hired into low poverty schools. Though both the new hires are less effective than the average teachers, and high poverty schools have more new hires than low poverty schools, neither makes much of a difference because those differences are already small and the performance of new hires improves fast: on average, they become as effective as the average teacher after one year.
Fourth, not surprisingly, on average, teachers who transfer to schools that are higher in poverty than the one they left are less effective than the average teacher. Yet those differences don’t impact equity much because just under 4 percent of all teachers transfer to schools in a higher or lower poverty category anyway (a little more than 4 percent move between schools with similar poverty rates).
Fifth and finally, teacher attrition doesn’t much affect access to effective teachers among high- and low-income kids because the leavers are equally effective among high- and low-poverty schools. Only in a small subset of districts (three out of twenty-six) did they find inequity in access to effective teachers—and it was in math only. In those three districts, if you provided high- and low-income kids with equally effective teachers from fourth to eighth grade, you’d see a reduction in the student achievement gap by at least a tenth of a standard deviation, which is equivalent to four percentile points over a five-year period.
With all that said, the sample the study uses is not nationally representative, even though it is geographically diverse and mostly includes large districts that are lower performing. It also mirrors the types of achievement gaps we see nationally, including in NAEP performance. Therefore, these findings may not hold in small districts or rural areas, for example.
Furthermore, it’s possible that the poorest children in the country (say, those at the tenth percentile of the income distribution) are in fact getting less-effective teachers than the richest kids (those at the ninetieth percentile, for example). But this study couldn’t examine that question because it relied on a binary definition of socio-economic status (i.e., whether a student was eligible for FRPL or not)—and again, findings are not nationally representative.
Still, analysts conclude with a simple summary: The achievement gap arises from factors other than students’ access to effective teachers.
Given that this bottom line finding is the result of an expensive study commissioned by a federal agency and conducted by a well-regarded research shop, it represents a big debunking of conventional wisdom.
SOURCE: Eric Isenberg et al., “Do Low-Income Students Have Equal Access to Effective Teachers? Evidence from 26 Districts,” Institute of Education Sciences, U.S. Department of Education (October 2016).
As another year ends, we want you to tell us what you think were the most important Ohio education stories in 2016 and what you predict will be the top story next year.
This is the easiest task you’ll be asked to do today. It’s only two questions and should only take a minute to complete. You can preview the questions below. When you’re ready to take the survey, click here or on the image below.
Just like the voting booth, whatever you submit will be confidential. Of course, if you want to write and tell us why, we may even feature your piece on our blog.
Thanks for your participation.
As a form of credentialing, high school diplomas are supposed to signal whether a young person possesses a certain set of knowledge and skills. When meaningful, the diploma mutually benefits individuals who have obtained one—it helps them stand out from the crowd—and colleges or employers that must select from a pool of many candidates.
In recent years, however, Ohio’s high school diploma has been diluted to the point where its value has been rightly questioned. One of the central problems has been the state’s embarrassingly easy exit exams, the Ohio Graduation Tests (OGT). To rectify this situation, Ohio is phasing in new high school graduation requirements starting with the class of 2018. Under these new requirements, students must pass a series of seven end-of-course assessments in order to graduate high school, or meet alternative requirements such as attaining a remediation-free ACT score or earning an industry credential.
The end-of-course exams have proven tougher for students to pass than the OGT, leading to concerns that too many young people will soon be stranded without a diploma. One local superintendent called the situation an “apocalypse,” predicting that more than 30 percent of high school students in his district would fall short of the new standards. He wasn’t alone, as an estimated 200 superintendents and school board members recently voiced their concerns at a Statehouse rally. An analysis by the Ohio Department of Education suggests that statewide, almost one in three pupils from the class of 2018 aren’t on a sure track towards a diploma.
This has put the state in a bind. On the one hand, in an era of heightened standards, no one wants to backtrack and hand out meaningless credentials. On the other hand, policymakers are right to worry about leaving thousands of pupils without a diploma. In today’s economy, such students are likely to struggle to find employment and are unable to join the military.
Can Ohio move forward on high standards—even re-inflating the value of the diploma—without leaving young people behind? Yes, but it will take some rethinking about how the state awards its high school credentials.
The most reasonable alternative, also suggested by several prominent education analysts (including our own Checker Finn), is for Ohio to pursue a multi-tiered approach to awarding diplomas. This would help Ohio maintain high achievement standards in the face of pressure to lower them while also building an incentive structure that could push students to achieve at higher levels. Ohio already has an honors diploma for students who go above and beyond in their coursework—a good start. Yet the honors diploma does not rely on state assessment results nor is it widely recognized as a measure of the accomplishments of Ohio’s highest achievers.
Here’s how a beefed-up, tiered system of awarding diplomas could work. At the base level, Ohio could create a standard-issue diploma signifying that pupils have persevered through thirteen years of school—a certificate of completion more or less. These students would have met their core coursework requirements yet fallen short of the stringent benchmarks of Ohio’s end-of-course exams. If we’re being frank, this is where Ohio has been with tying diplomas to the OGTs over the past decade or so (and perhaps also with its predecessor exam, the Ninth Grade Proficiency Test). One step up would be a college- and career-ready diploma that indicates students have demonstrated readiness, either by meeting rigorous academic targets on state exams or completing a demanding industry certification. This lines up more closely to Ohio’s new graduation requirements. Finally, the state could award a third diploma—a certificate of exceptional accomplishment—that the most academically able students receive. This diploma’s benchmarks could be geared to the expectations of the state’s most selective colleges and universities and would be a cause of celebration for students, parents, schools, and communities. The state could also tie the diploma to a merit-based college scholarship program.
A tiered approach would have several benefits. First, the state could maintain high expectations for all graduates, yet simply award various diplomas depending on whether pupils fell short, reached, or considerably exceeded the end-of-course exam standards. Second, by issuing at least a basic-level diploma, the state could avoid the repercussions of potentially denying one in three students a diploma. Third, by awarding a diploma with distinction, the state may incentivize some pupils to accumulate more “human capital.” For instance, consider a junior who has already secured a college- and career-ready diploma. She may not feel motivated in her senior year—a case of “senioritis.” But with an honors diploma in play, and clear benefits for earning one, there is more reason to work hard. Fourth, the state could allow pupils meeting the criteria for the college- and career-ready diploma as juniors (or earlier) to graduate, and then offer them the funds saved by foregoing their senior year to defray the cost of college. Fifth, a tiered diploma would help young people as they enter the workforce. For college students, it could help them in the competition for internships; likewise, a more meaningful diploma should aid those seeking full-time employment directly after high school.
Ohio’s old and outgoing high school diplomas didn’t signal much of anything to anyone. The requirements were ridiculously easy and practically everyone got a diploma. This diminished its value. Now the state is ratcheting up its graduation requirements: As State Superintendent Paolo DeMaria told the Columbus Dispatch, “This is all about giving greater meaning to a high school diploma.” State leaders should not back down on rigorous graduation standards simply to accommodate more diplomas. But neither should they be hard-headed. The best way forward for Ohio is to remake the diploma and reject the notion that there is only one way forward.
Italy has an achievement gap—one that may sound familiar to Americans. PISA scores show a marked gap between Italian students and those of other OECD countries in both math and reading. Digging into the data, Italian education officials found their own in-country gap: Students in the wealthier north perform far better than students in the poorer south. As a result of all of this, starting in 2010, schools in Southern Italy were offered an opportunity to participate in an extended learning time program known as The Quality and Merit Project (abbreviated PQM in Italian). A new study published in the journal Economics of Education Review looks at PQM’s math and reading intervention, which consisted of additional teaching time after school in four of the poorest—and lowest-performing—regions in the country.
A couple of things to note: PQM intervention was focused not on improving PISA test scores, but on improving scores on the typical tests taken by students in lower secondary school (equivalent to grades six to eight in the U.S.). There is no enumeration of which/when/how many tests these students typically take and the researchers are not attempting to make a connection between the intervention and PISA test scores. We as readers should not either. The poor performance of Italian students on PISA simply shone a light on poor performance elsewhere, and perhaps more importantly, unlocked the funding (from the European Union’s Regional Development Fund) that paid teachers to implement an intervention aimed at closing the detected gap.[1] Deciding to initiate the PQM intervention was voluntary on the part of schools. That allowed researchers to match schools that participated with similar schools that didn’t participate. Using results from the typical tests taken by lower secondary school students, the analysts compared changes in test scores before and after the intervention.
The report had two key findings. First, PQM had a positive effect on average test scores in math, but no impact on reading scores. Second, the impact differed depending on pre-intervention achievement: students in the lowest-achieving schools—in the bottom third—made significant gains on math due to the program. For students attending schools in the top two-thirds of achievement, the impact of the after-school program was null in both math and language. According to this evaluation, then, the program worked in a narrow sense—in just math and for the lowest-achieving students.
Researchers conclude that additional in-class instruction time as an intervention in reading is not particularly helpful to students in grades six through eight. “This result is consistent with other studies in the literature showing that it is much harder to intervene on reading and comprehension skills,” they write, “rather than on skills involving practice, like maths, because a large part of literacy work takes place through general vocabulary training in the home environment.” In other words, improving the “skill” of reading is much more than a matter of spending more time on it once fluent decoding has been learned. (We would add that it also relates to content knowledge—something that certainly can and should be taught in school.) However, this research indicates that quantitative reasoning and mathematical knowledge—increased by repetition and “skill building”—responds positively through more time spent on task, especially for low-achieving students.
We need to be careful about the conclusions we draw based on the numerous caveats and unknowns here (not to mention the differing culture and language), but a detailed look at the benefit to students of additional time on task is no bad thing. A longer school day is often seen as a cure-all for students with poor test scores and is sometimes the raison d'être of certain school types. Perhaps a more targeted approach to additional seat time the proper approach.
SOURCE: Erich Battistina and Elena Claudia Meroni, “Should we increase instruction time in low achieving schools? Evidence from Southern Italy,” Economics of Education Review (December, 2016).
[1] As befits this particular journal, the economics of the PQM intervention is addressed in the report. There is not enough space to summarize it here, but it could be instructive to American policymakers about ways to use data to improve outcomes.
