Skip to main content

Mobile Navigation

  • National
    • Policy
      • High Expectations
      • Quality Choices
      • Personalized Pathways
    • Research
    • Commentary
      • Gadfly Newsletter
      • Flypaper Blog
    • Events
    • Scholars Program
  • Ohio
    • Policy
      • Priorities
      • Media & Testimony
    • Research
    • Commentary
      • Ohio Education Gadfly Biweekly
      • Ohio Gadfly Daily
  • Charter Authorizing
    • Application
    • Sponsored Schools
    • Resources
  • About
    • Mission
    • Board
    • Staff
    • Career
Home
Home
Advancing Educational Excellence

Main Navigation

  • National
  • Ohio
  • Charter Authorizing
  • About

Ohio Menu

  • Topics
    • Accountability & Testing
    • Advanced Education
    • Career & Technical Education
    • Charter Schools
    • Curriculum & Instruction
    • ESSA
    • Evidence-Based Learning
    • Governance
    • Personalized Learning
    • Private School Choice
    • School Finance
    • Standards
    • Teachers & School Leaders
  • Research
  • Policy
  • Commentary
    • Ohio Gadfly Newsletter
    • Ohio Gadfly Blog
    • Events

The Education Gadfly Weekly

Sign Up to Receive Fordham Updates

We'll send you quality research, commentary, analysis, and news on the education issues you care about.
Thank you for signing up!
Please check your email to confirm the subscription.

Ohio Gadfly—The I Promise School’s troubling academic results

Volume 17, Number 18
8.7.2023
8.7.2023

Ohio Gadfly—The I Promise School’s troubling academic results

Volume 17, Number 18
view
I Promise update blog image
Standards & Accountability

An in-depth analysis of the I Promise School’s troubling academic results

In 2018, basketball superstar LeBron James opened the I Promise School (IPS) in his hometown of Akron, Ohio. IPS is a joint effort between Akron Public Schools (APS), the I Promise Network, and the LeBron James Family Foundation. It’s overseen and operated by APS, the state’s seventh largest school district.

Jessica Poiner 8.7.2023
OhioOhio Gadfly Daily

An in-depth analysis of the I Promise School’s troubling academic results

Jessica Poiner
8.7.2023
Ohio Gadfly Daily

Mayoral control has been good for Cleveland schools

Aaron Churchill
8.1.2023
Ohio Gadfly Daily

How Ohio helps students into tech-based internships

Jessica Poiner
7.27.2023
Ohio Gadfly Daily

Student acceptance of computer-adaptive testing

Jeff Murray
8.8.2023
Ohio Gadfly Daily

Assessing the operations and impacts of Teach For America in Indianapolis

Jeff Murray
8.3.2023
Flypaper
view
Mayoral control Cleveland blog image

Mayoral control has been good for Cleveland schools

Aaron Churchill 8.1.2023
Ohio Gadfly Daily
view
HS tech internship blog image

How Ohio helps students into tech-based internships

Jessica Poiner 7.27.2023
Ohio Gadfly Daily
view
Computer adaptive testing image

Student acceptance of computer-adaptive testing

Jeff Murray 8.8.2023
Ohio Gadfly Daily
view

Assessing the operations and impacts of Teach For America in Indianapolis

Jeff Murray 8.3.2023
Flypaper
view
I Promise update blog image

An in-depth analysis of the I Promise School’s troubling academic results

Jessica Poiner
8.7.2023
Ohio Gadfly Daily

In 2018, basketball superstar LeBron James opened the I Promise School (IPS) in his hometown of Akron, Ohio. IPS is a joint effort between Akron Public Schools (APS), the I Promise Network, and the LeBron James Family Foundation. It’s overseen and operated by APS, the state’s seventh largest school district. But thanks to additional funding and support from the foundation, it also offers a host of resources that most district schools do not—things like free uniforms, tuition to the University of Akron when students graduate high school, and family services like GED classes and job placement assistance. The school also has some unique characteristics that are typically associated with schools of choice: an extended school day and year, a STEM curriculum, and alternative working conditions for teachers.

During the school’s inaugural year, it welcomed 240 students in third and fourth grade. The original plan was to add students and grade levels each year so that the school would serve students in grades 1–8 by 2022. But in 2019, the school announced that, although it would continue adding a grade level each year up until eighth grade, it would not add first and second grades. Students are admitted in third grade via a lottery of second graders, who must already attend APS and fall into the bottom 30 percent of reading scores. The school’s website reinforces its focus on academically struggling students by declaring its dedication “to those students who are already falling behind and in danger of falling through the cracks.” In several pieces published by the Akron Beacon Journal (ABJ), its noted that the school’s goal is to “move students forward at a rate of one and a third years’ worth of growth in one year. That would get a student who is two years behind in third grade back on track before they reach high school.”

The school’s initial results were promising. In April 2019, it announced on Twitter that 90 percent of its students had met or exceeded their expected growth in math and reading on NWEA MAP, a national computer adaptive assessment that measures academic growth over time. Coverage in USA Today called the results “extraordinary” and noted that scores at IPS increased at a rate higher than ninety-nine out of 100 schools based on NWEA norms.

As anyone who works in education will tell you, though, multiple measures matter when it comes to evaluating a school’s performance. To their credit, most media outlets recognized that. The New York Times pointed out that “time will tell whether the gains are sustainable and how they stack up against rigorous state standardized tests at the end of the year.” And they were right. State test results would matter immensely.

But those showed promise, too. When state report cards were released in the fall of 2019, IPS had earned an overall grade of C. In the age of grade inflation, folks accustomed to nothing but A’s might be displeased. Under the circumstances, however, a C was actually a good thing. It meant that, overall, the school was meeting the state’s expectations. In fact, for a brand-new school that was purposely targeting students performing a year or two behind grade level, a C was pretty impressive. It was even more praiseworthy considering the school’s demographics: The overwhelming majority of students were economically disadvantaged, 29 percent were students with disabilities, 15 percent were English language learners, and over 80 percent were minorities.

A more detailed breakdown of state report card components showed a mixed bag. The school earned an F on the achievement component, which measures students’ overall performance on state assessments. But it also earned an A on the progress component, which determines the growth of students based on their past performance and gives credit when students—including those below grade level—exceed growth expectations. In English language arts, IPS students made progress similar to statewide expectations. In math, however, students made more progress than expected. The school also earned high marks for progress in certain subgroups: a B for students identified as the lowest 20 percent statewide in reading, math, or science, and another B for students with disabilities.

The big takeaway after one year was that both state test results and in-house exams measuring growth indicated that students at IPS weren’t just learning, they were learning a lot. If that kind of growth continued, the school would be well on its way to meeting its goal.

And then the pandemic hit.

At IPS, as elsewhere, it had a massive negative impact on student academic performance. School closures, remote learning, and a plethora of other issues combined to produce whopping learning losses, especially among students from disadvantaged communities—the very same students that IPS makes it a point to serve. IPS students were already years behind their peers before the pandemic hit. Learning loss of any kind would be catastrophic to their outcomes. Add to that the fact that IPS was just a year and a half old when the pandemic hit (staff were still trying to establish the school’s culture) and was experiencing some leadership turmoil (the school lost its founding principal during its third year and then had three interim principals in just a year and a half), and it would’ve been a miracle for IPS students to escape the pandemic academically unscathed.

Here, as elsewhere, the pandemic also made it more difficult—though not impossible—to keep track of results over time via publicly available data. State tests weren’t administered in 2019–20, so there was no way to easily compare the school’s second year to its first. State tests were administered in 2020–21, but ratings weren’t assigned, so the IPS report card from that year is similarly unhelpful.

But there is one important exception to the pandemic era’s dearth of data: Ohio maintained its performance index measure, which considers the state test results of every student and awards schools points based on the state’s six achievement levels. The higher a student’s achievement, the more points the school earns. For example, a student who scores at the advanced level earns their school 1.2 points, while a student testing at the basic level (just below proficient) earns 0.6 points. Schools weren’t graded on performance index during the pandemic, but the state did calculate it during the years that tests were administered. During the 2018–19 school year, when IPS was showing lots of promise (pun intended), it posted a performance index score of 53.5. Two years later, in the wake of the pandemic, IPS’s performance index score had dropped to 27.5, a loss of nearly half.[1]  The school’s score rebounded slightly in 2021–22 to 38, but that’s still far below the score of its inaugural year.

To be clear, IPS isn’t the only school that saw its performance index scores plummet. In fact, between 2018–19 and 2021–22, average performance index scores dropped for the state overall, for districts, and for every student subgroup. Most notably, there were double-digit declines among economically disadvantaged students, Black students, and students with disabilities. As noted earlier, these students make up a majority of the IPS population.

Because the performance index scores at IPS show a similar trend to districts and schools throughout the rest of the state, it seems reasonable to say that the lackluster results coming out of IPS could be attributed to the pandemic. That’s especially true in light of the school’s demographics and the fact that it specifically enrolls students who are academically behind. Of all the schools in Ohio, IPS might have the strongest argument that the pandemic derailed its efforts, and that it shouldn’t be judged too harshly on recent results.

But maybe not. As a public school under the umbrella of Akron Public Schools, IPS received a share of the billions of dollars in federal relief funding that was sent to states to combat learning loss. And while other schools might not have known what to do with all that money, IPS should have. First, because the school leaders should have been accustomed to being financially flush. The LeBron James Family Foundation spent roughly $2 million during the school’s first year, including startup costs, and committed to spending around $2 million or more per year once the school had grown to capacity. Unlike a standard district school, IPS has always had extra money at its disposal. Second, one of the school’s defining features is that it offers students an extended day and school year. Research shows that one of the best learning loss remedies is extra time, including time before and after school and during the summer. While other schools might have struggled to figure out how to provide kids with instruction during periods when buildings were typically closed and teachers were gone, IPS didn’t have that problem. Extra time was built into its DNA.

And then there’s the wraparound services. Another defining feature of IPS is that it offers students more than just an education. It also offers counseling, health checkups, food pantries, clothing, tutoring (which, when done correctly, is another evidence-backed remedy for learning loss), and even housing. Lacking access to these services can impact students’ academic outcomes in both the short and long term. For students at IPS, though, this lack of access shouldn’t have existed—at least not as often as it did for their peers in other schools. And while it’s true that IPS was especially vulnerable to pandemic-related impacts because of its demographics, the school was and is set up to blunt those impacts in some pretty big ways. All things considered, it seems reasonable to expect IPS, of all schools, to catch its kids up at a pretty rapid pace.

But that doesn’t appear to be happening. At least not according to the ABJ, which recently published a piece entitled “Promises kept? Akron school board questions I Promise School’s poor test scores.” It covers a recent school board meeting where board members were presented with academic data from IPS. The opening sentence of the article is a doozy: “This fall’s class of eighth graders at the I Promise School hasn’t had a single student pass the state’s math test since the group was in the third grade.” Meanwhile, two of the school’s biggest student subgroups—Black students and those with disabilities, subgroups that made notable progress during the school’s first year—are testing in the bottom 5 percent of the state.

The coverage from the ABJ noted that, although the school receives the same base funding as any other Akron public school, the LeBron James Family Foundation has given “up to $1.4 million each year for additional tutors and more teachers in younger grades to lower class sizes.” That additional funding means that, not only did IPS likely have more money at its disposal than other Akron schools, the money was designated for academic supports that should have gone a long way to catching students up. And yet, according to the ABJ, “comparing I Promise students to their peers who qualified for the school but attend elsewhere shows that I Promise students are doing worse in some cases, despite the extra staff and wraparound services.”

APS officials didn’t offer the state test results for students who met IPS admissions criteria but didn’t attend the school. That’s a shame, as such a comparison would shine a much brighter light on how IPS is doing. But the district did offer another form of data that makes it possible to compare. Like many districts in Ohio, Akron uses iReady assessments as an internal measure of student growth. And according to Keith Liechty-Clifford, the district’s director of school improvement, IPS students in grades 3–8 performed two points better in reading but six points worse in math than other district students who were also in the bottom 25 percent.

Unfortunately, it gets even worse. The ABJ notes that Liechty-Clifford presented data at the school board meeting showing that, while some grades of students have improved over time based on state test results, others have lost ground. Last year’s sixth graders, for instance, went backward, from 7 percent proficient in reading in fifth grade, to just 2 percent in sixth grade. Last year’s fifth grade students did better—they went from less than 6 percent proficient in reading in fourth grade to 13 percent in fifth. But that’s still not comforting. If only 13 percent of fifth graders can pass the state test, and that’s considered a bright spot for the school, then we have some problems. Especially if just 2 percent of students in the next grade level up are also proficient.

It should go without saying that all these data weave a big red flag. In response, the school’s new principal, Stephanie Davis, provided a statement via the foundation. According to the ABJ, Davis claims that there are data that “wasn’t provided” at the board meeting and which show that students are, indeed, growing. She notes that, although students might not be proficient and on grade level, according to their i-Ready scores, 42 percent grew at least a year’s worth in the last year in math, and an additional 4 percent reached a “stretch” goal beyond that. Meanwhile, 32 percent of students met a goal for typical annual growth in English, and 11 percent exceeded it.

In her statement, Davis also emphasized that “the type of growth that is important to us is not made overnight.” And she’s not wrong. Growth does take time. Especially when we’re talking about proficiency, which—as Davis acknowledges—is “based on mastering grade-level standards.” When students are behind, mastering grade-level standards is far more difficult. That’s why it’s crucial to consider both proficiency and growth when evaluating school performance.

Still, the data cited by Davis should give us pause. First, that’s because they represent the percentage of students who, over the course of the last year, grew a year’s worth in math or met a goal for “typical annual growth” in English. In lay terms, that means they grew the amount they were supposed to over the course of a year. Given how many schools in Ohio fail to ensure that their students grow a year’s worth in reading and math, that’s nothing to sneeze at. But it’s also not something that should be treated as impressive. To do what IPS promised to do, kids need to make more than one year’s worth of growth each year—specifically, one and a third. APS staff knows that. “We signed up for how we attract, who we allow in, and there can’t be any excuses for why they don’t perform at that rate,” Tamiko Hatcher, the district’s director of specialty programs, told the ABJ in 2022. By Davis’s own admission, though, only 4 percent of kids in math and 11 percent in English grew more than a year. Furthermore, more than half of IPS students in both reading and math fell short of a year’s growth.

Second, although schools might need time, that doesn’t mean kids can spare it. The fourth graders from the school’s inaugural year will start high school this fall. They spent five years of their lives at IPS, a school that promised to catch them up by high school. Unfortunately, that’s not what happened. In fact, according to the ABJ, only 11 percent of the school’s first graduating class of eighth graders was proficient on the state’s ELA test.

As someone who used to teach ninth grade English, that number gives me heartburn. And as someone who grew up in Akron and still considers it my hometown, I’m deeply concerned. I understand the outsized impact of the pandemic on student outcomes. And I don’t think it’s fair to ignore that the pandemic likely played a significant role in getting IPS to the troubling place in which it now seems to live. But IPS, far more than most schools, was designed to blunt out-of-school factors. It’s always been set up to give kids the kind of intensive, in-depth intervention they need. If, when school report cards come out this fall, IPS isn’t well on its way to a significant and meaningful rebound, then it will be well past time for the school to answer some very tough questions.


[1] The PI calculations award zeros when students do not participate in state assessments; if the number of untested students at IPS was significantly higher than normal (which was the case statewide in 2020–21), that may have contributed to depressed PI scores.

view
Mayoral control Cleveland blog image

Mayoral control has been good for Cleveland schools

Aaron Churchill
8.1.2023
Ohio Gadfly Daily

A recent headline in Education Week suggested that mayoral control is a “fading school reform.” The piece noted the impending transition to an elected school board in Chicago, as well as efforts in Boston to wrest control from its mayor. The idea of mayoral control does seem to be greeted with more skepticism these days than in the early 2000s when education reformers—observing the troubles of elected boards—were encouraging this alternative governance structure.

But mayoral control shouldn’t be quickly cast aside, judging from the past decade of reform in Cleveland. Led by a reform-minded mayor and savvy district leader, the city’s public school system has made progress after many decades of mismanagement and political backbiting under an elected board. Academic performance is still far from satisfactory, but improvements are evident, more high-quality public school options are available to families, and the district is now in decent financial condition.

As a quick rewind, the Cleveland school district was in complete disarray prior to start of mayoral control in 1997. During the previous two decades, it had a staggering twelve superintendents—one of whom tragically took his own life, citing “petty politics.” Finances were a mess, leading the state to declare a  fiscal emergency. Less than half of students graduated high school, and state test scores were abysmal. In 1995, a federal judge ordered state supervision of the district because of its governance, fiscal, and academic woes.

The crisis also precipitated the legislative action that brought mayoral control to Cleveland. Under state law, the mayor appoints a nine-member board of education, each of whom are first nominated by an eleven-member panel. That panel, which include appointees and representatives of various stakeholders, nominates a slate of candidates for each board vacancy from which the mayor may choose. The appointed board hires the district CEO—akin to superintendent—but it must do so with mayoral concurrence (which is also required to fire a CEO).

While the first decade of mayoral control in Cleveland (1998–2010) has received mixed reviews, the model has worked to stabilize district leadership over the past decade, something that has been highlighted in national profiles of recently departed CEO Eric Gordon. With backing from longtime mayor Frank Jackson, the Gordon era lasted twelve years—double the tenure of the average large-district superintendent. Shortly after the 2021 election of Justin Bibb as the new mayor, Gordon chose to resign and the board selected Warren Morgan as his successor.

Gordon’s longevity alone wouldn’t warrant many accolades but his accomplishments during this time were many. They include implementing reforms via the Cleveland Plan (more on it below), building confidence in city schools as evidenced by  passage of three levies, and forging partnerships with community leaders as well as high-quality public charter schools. I can’t prove it but it’s doubtful that any of these steps forward would have occurred under an elected school board. Consider the following:

  • Stabilizing leadership: Mayor Jackson and CEO Gordon presented a unified vision for Cleveland schools. School board politics seemed mostly tame during this period, and while there were likely some naysayers, no widely known attempts were made to oust Gordon during his tenure. Contrast this with the turbulent politics of district reform efforts in nearby Youngstown and Lorain via academic distress commissions, another whole-district turnaround model that Ohio has tried. That initiative left intact the elected boards, while at the same time creating a state-led commission to oversee the district and to appoint a CEO who’s empowered to implement an aggressive reform plan. Unfortunately, the elected boards have undercut the CEOs tasked with improving the districts. In the end, the influence of the elected boards helped to quash reform efforts, destabilize leadership, and swiftly dislodge the CEOs.

  • Implementing the Cleveland Plan: Mayoral control encourages a city’s top official to engage in K–12 education, rather than just sit on the sidelines. And that’s what Jackson did. He understood the city’s urgent need for better schools, so he rolled up his sleeves and initiated the Cleveland Plan. This package of reforms, which state lawmakers facilitated with legislation in 2012, aimed to grow high-performing schools, cut bureaucratic red tape, increase parental engagement, and reward effective teachers. It quickly became clear that Gordon was Jackson’s right-hand man for carrying out the plan. And for more than a decade, he strove to implement—and make updates to—the plan under the mayor’s watch. It’s doubtful that the Cleveland Plan (or something like it) could have been developed under a fractured elected board. On the other hand, the unified vision of the mayor, appointed board, and CEO was instrumental in developing and executing the Cleveland Plan.

  • Passing levies: After failed levy attempts in 2004 and 2005, the district passed property tax levies in 2012, 2016, and 2020. The support of Mayor Jackson may have helped turn the tide in the district’s levy fortunes. Of course, it’s certainly possible that these would’ve passed without mayoral support. Other big-city districts pass levies, too, and the earlier Cleveland failures also happened when under mayoral control (though pre-Jackson). But having a popular mayor, who was heavily invested in the city’s schools and leading a bold reform plan, could have nudged some voters to say yes to the levy requests.

  • Partnering with quality charters: Speaking of levies, Cleveland is the only district in Ohio that shares a portion of local tax revenue with public charter schools located within its borders. This is an element of the Cleveland Plan, and helps support the growth of high-performing charters. Today, the district partners with sixteen charter schools that meet certain qualifications. The unique partnership probably couldn’t happen with an elected board. Mayors, with their broader constituencies and responsibilities, likely think more comprehensively about solving problems in their community. As such, the mayor may have been more open-minded about charters than an elected board, which tends to be protective of district interests. Bottom line: Cleveland families and students have benefitted from a more holistic strategy to grow excellent schools.

Governance revamps alone can’t solve all that ails public schools, including those in Cleveland. But they do impact the leadership of a district, its political climate, and the opportunities for reform. Elected boards can easily get embroiled in political food fights—just look around the country right now—and they can also be captured by special interest groups like the teachers’ unions. While not immune to these pressures (or mayors that are hostile to reform), mayoral control can help a district rise above these squabbles, allowing for stronger and more coherent leadership that focuses on what’s best for families and students.

With reform-minded leadership in place, Cleveland’s version of mayoral control has helped right a once-sinking ship. It also gives us reason for optimism as Mayor Bibb and CEO Morgan take the reins and work to put their stamp on education in Cleveland. (Here’s a few ideas on how.) Far from being a fading reform, mayoral control might just be the governance structure that more cities and towns need to improve their schools.

view
HS tech internship blog image

How Ohio helps students into tech-based internships

Jessica Poiner
7.27.2023
Ohio Gadfly Daily

Since he took office in 2019, Governor DeWine has consistently prioritized work-based learning and workforce development. In conjunction with the legislature, his team has launched initiatives like the Innovative Workforce Incentive Program (IWIP), which provides funding to school districts to help them establish new programs for students to earn qualifying credentials in “priority” industry sectors like advanced manufacturing and information technology. It also awards schools an additional payment of $1,250 when a student earns such a credential. During fiscal year 2022, Ohio students earned a total of 5,744 eligible credentials, and 123 school districts received over $7 million in incentive payments.

Such impacts are worthy of praise. But so, too, is an under-the-radar internship program under the purview of the Governor’s Office of Workforce Transformation. The High School Tech internship began in 2021 as a pilot program. It was designed to provide high school students with valuable work experience in technology-related roles while giving businesses a competitive opportunity to bolster their talent pipelines (among other benefits). Under this program, employers who hire high school interns in tech-related roles can be reimbursed for wages if students are employed for a minimum number of hours (150 under the first iteration of the program, 120 under the latest guidelines) and are paid at least $12 per hour.

At the outset, two education pilot sites were selected in each of the state’s six JobsOhio regions to facilitate relationships between employers and students. Site staff were responsible for recruiting employers, working with schools to identify students to apply for internships, developing work-based learning agreements for those who were hired, and monitoring interns’ progress. The initial goal was to place 100 high schoolers during the summer of 2021 in roles focused on software development, data, cybersecurity, and cloud and IT infrastructure. According to the program’s 2021 education pilot site report, more than seventy students were successfully placed.

The third year of the program is now underway. (Internships can start at any time once funding is approved, though no student can be placed after June 15.) Media reports indicate that up to 535 students statewide are participating in tech-related internships thanks to the program. While 500 students won’t transform the state, the significant increase since year one suggests an appetite from students and employers for this type of opportunity. Furthermore, without this program, many students might not have access to work-based learning experiences at all.

The state also deserves kudos for being attentive to feedback. Consider the following:

  • The 2021 education pilot site report notes that most sites identified “timeline challenges” as an issue, and suggested starting the employer recruitment process in January. This year, applications for the program started in January.

  • The report indicates that sites with intermediary organizations were more successful at placing larger numbers of students. This year, the state is piloting an intermediary model in two JobsOhio regions—Dayton and Cincinnati—with the Strategic Ohio Council for Higher Education (SOCHE) and the INTER Alliance of Greater Cincinnati serving as intermediaries that will facilitate the program and serve as a resource for educational entities and businesses.

  • The report emphasizes that financial incentives for employers can help businesses “mitigate the financial risks” of providing opportunities to students. Maintaining wage reimbursement has been crucial. But there are also additional incentives at play. Under 2021 guidelines, employers were eligible to receive an extra $100 in reimbursement if an intern earned a credential on the IWIP list. The most recent program guidelines upped that amount to $1,000. With a financial incentive that’s ten times bigger, it seems likely that more businesses will prioritize helping students earn qualifying credentials while they participate in internships.

The latest program guidelines also include additional incentives for educational entities (districts, career-tech centers, and ESCs are all eligible to participate in the program). For example, participating schools may be eligible to use qualifying credentials earned by students during these internships to obtain the $1,250 IWIP incentive. Educational entities are also eligible for a separate incentive payment based on the number of interns they place who work a minimum of 120 hours: Placing four interns is worth $400, six is worth $750, and eight can net an additional $1,200. 

Compared to some other initiatives overseen by the Governor’s Office of Workforce Transformation, the High School Tech Internship program isn’t big or flashy. But it does benefit students, schools, and employers, and it seems to be getting better with age. That’s exactly the kind of program Ohio needs right now—and hopefully, more programs like this will soon be available.

view
Computer adaptive testing image

Student acceptance of computer-adaptive testing

Jeff Murray
8.8.2023
Ohio Gadfly Daily

Computer-adaptive testing (CAT) is on the rise in K–12 schools, from Seattle to Virginia and everywhere in between. Proponents of the algorithm-driven testing model say that having variable problem sets geared toward the demonstrated capabilities of individual students allows for more accurate assessment, especially for very high and very low achievers—with less gaming and guessing—along with a focus on student growth rather than raw scores. Opponents, meanwhile, contend that those features play into the fears of test-averse students and can depress their scores. New research from Japan may offer some clues to help unravel this conundrum.

Japanese education officials have been resistant to using computer-adaptive testing for several reasons, including those noted above, but also some that are peculiar to Japan. There, large groups of students taking the same test in the same place at the same time is a longstanding cultural touchstone that would be entirely eliminated by administration of tests that must be taken on electronic devices, have different questions for each student, and likely require differing amounts of time for students to complete. Nevertheless, CAT’s expansion into international arenas like PISA and the Test of English as a Foreign Language (TOEFL) has made the Education Ministry take a second look. Three researchers from institutions across the country developed a computer-adaptive testing protocol designed to determine not only how students performed, but also how they perceived the testing method and whether their own attitudes toward testing were reinforced or allayed by the experience.

The researchers conducted two experiments. The first was to determine whether students who have testing anxiety performed more poorly on computer-adaptive tests than those who do not. They compiled a proxy measure of anxiety based on students’ self-reported achievement goals (whether they are focused on performance or mastery, à la Dweck) and approach to learning (deep vs. surface). A total of 870 students (474 fifth graders, 174 sixth graders, 163 seventh graders, and 58 students from an unknown grade) from the Kansai region were recruited between June and September 2020 to participate in the first experiment. (Japanese schools mostly remained open during the pandemic.) A total of 415 students were male; 383 were female. Data indicate that a student’s performance goal did not directly predict their test outcome, implying that the computer-adaptive test did not reinforce any preexisting anxiety about test taking. Additionally, students were surveyed post-test regarding their attitudes toward three types of assessments: the computer-adaptive version they just took, the traditional fixed-item version with which they were familiar, and a putative human-adaptive test which is a hybrid of the two. Despite limited familiarity with CAT, students in the first experiment expressed more positive attitudes toward it than toward traditional fixed-item and human-adaptive versions.

The second experiment, conducted in Kansai from March to July 2021, involved an additional 745 students (540 fifth graders, 90 sixth graders, 100 seventh graders, and 15 students from unknown grade levels), roughly half male and half female. The basics of the previous experiment were repeated with the same results. However, this time students were given more detailed versions of the two surveys, aimed at identifying potential mechanisms at work. Results revealed that students who felt that the value of testing was primarily to show improvement or to get them to develop study habits were least accepting of computer-adaptive testing, although the difference between them and their peers was minimal. Students who felt that the value of testing was to determine comparative achievement levels preferred a putative human-adaptive test over the other two options.

What might all this mean? The researchers believe it means that computer-adaptive testing does not make students who are prone to anxiety regarding their test performance any more or less anxious than a traditional fixed-item test would. On the contrary, most students seem unopposed to the new model regardless of their level of test anxiety. In the highly-traditional Japanese education sector, that’s saying something. More specifically, if the purpose of testing is made clear to students—that is, what their teachers and school leaders are going to do with the resulting data—nearly any type of test seems to be acceptable to them. And, as noted above, the data obtained from a computer-adaptive test are more focused on growth measures rather than raw or scaled scores, which can be compared among students. As long as teachers, administrators, and policymakers are good with that new focus, it seems that CAT could be expanded in K–12 education without concern.

SOURCE: Takayuki Goto, Kei Kano, and Takayuki Shiose, “Students’ acceptance on computer-adaptive testing for achievement assessment in Japanese elementary and secondary school,” Educational Psychology (July 2023).

view

Assessing the operations and impacts of Teach For America in Indianapolis

Jeff Murray
8.3.2023
Flypaper

The Indianapolis branch of Teach For America (TFA Indy) was established in 2008, expanding the national organization’s mission—to build and deploy a corps of high-quality education leaders to support high-needs students—into the Hoosier State. Since that time, hundreds of TFA teachers have served students in district and charter schools in the area, assisted by local philanthropic entities, like the Richard M. Fairbanks Foundation. In 2021, the foundation commissioned RAND to assess the operations and impacts of the program and the resulting report was recently released.

The RAND team looked at academic outcome and teacher employment data from the Indiana Department of Education and specific information on TFA members’ employment trajectory during and after their participation, using data provided by TFA Indy. The analysis covers 2008 through 2022. While some TFA Indy members were placed in schools outside of Marion County, the report deals only with traditional district and charter schools located therein, an area of greatest interest to the Fairbanks Foundation. This sprawling county includes both the city of Indianapolis, whose public school student population is characterized by higher levels of household poverty and lower academic achievement than is typical statewide, and several smaller suburban and exurban districts. There are dozens of charter schools in the county, the vast majority of which are within the borders of Indianapolis Public Schools. TFA Indy teachers were more likely than other teachers to work in charter schools, in more-disadvantaged school contexts, and in schools serving more-diverse student populations.

Among the teachers themselves, TFA and non-TFA instructors during this period were similar on racial, ethnic, and gender dimensions (though non-TFA teachers were slightly more likely to be white). TFA Indy teachers were split almost evenly among elementary, middle, and high schools in the initial two-year cohort under study, but were far more prevalent in elementary schools by the final cohort.

The impact findings largely mirror previous TFA research across the country: TFA Indy teachers were, on average, more effective at increasing student achievement than their non-TFA peers. One year of instruction by a TFA Indy teacher was associated with 1- to 2-percentile increase in student performance in English language arts (ELA) and math, respectively (e.g., moving from the 50th percentile to the 52nd percentile).

The researchers also note that TFA Indy teachers who worked in schools with five or more TFA peers were even more effective at increasing student achievement, both overall and relative to non-TFA peers within the same school. Instruction by these TFA Indy teachers was associated with roughly 2- and 5-percentile gains in student achievement in ELA and math, respectively. More on this below.

While TFA Indy teachers left teaching earlier in their careers than non-TFA teachers, often as soon as their initial two-year commitment ended, more non-white TFA Indy teachers remained in the field (and in the county) than their non-TFA peers. Overall, TFA Indy teachers were promoted to school and district leadership roles at similar rates as non-TFA teachers, but non-white TFA Indy teachers were less likely to be promoted than non-white non-TFA peers. Among alumni who left teaching, non-white TFA Indy alumni were more likely to continue working in education-related careers than were white TFA Indy alumni.

The researchers note a couple of limitations, including the likelihood of unobserved variables around teachers entering and leaving the field, the geographic limitations imposed, and a simplified value-added model that imprecisely attributes student achievement to individual teachers. However, they note that “most of our findings tell a consistent story, and, in many cases, the effects we observe are highly statistically significant.” Thus, they conclude with reasonable confidence that hiring a TFA teacher in comparison to a non-TFA teacher results in a net-positive effect on student achievement. This includes negative effects from higher TFA Indy teacher turnover rates, which are far exceeded by the positive effects accrued by TFA teachers’ differential impacts on student achievement.

As to the finding that having more TFA teachers in individual schools led to even greater gains for students, TFA Indy staffers had some interesting thoughts. Among other hypotheses, they suggested that TFA teachers at these schools may be encountering school leaders and coaches who share similar mindsets and instructional strategies aligned with TFA training, or that TFA Indy teachers in these schools were part of larger incoming cohorts of novice teachers, and the camaraderie and collaboration generated more concerted effort. Very collegial of them not to suggest the simpler, more obvious possibility: TFA-trained teachers are better at serving students than are traditionally-trained teachers, and the more of them you have, the better the outcomes will be for kids.

SOURCE: Benjamin K. Master et al., “Teaching and Leading: An Evaluation of the Impacts and Professional Trajectories of Teach For America Participants in Indianapolis,” RAND Education and Labor (July 2023).

Fordham Logo

© 2026 The Thomas B. Fordham Institute
Privacy Policy
Usage Agreement

National

P.O. Box 110
Burke, VA 22009

202.223.5452

[email protected]

Ohio

P.O. Box 82291
Columbus, OH 43202

614.223.1580

[email protected]

Sponsorship

130 West Second Street, Suite 410
Dayton, Ohio 45402

937.227.3368

[email protected]