Aaron Churchill, Ohio research director at the Thomas B. Fordham Institute, joins Mike Petrilli to discuss what’s next for education policy in Ohio and beyond. Following last week’s election results, what does the political shift from red to blue mean for education reform? Drawing on Aaron’s recent report, The Next Chapter: Ohio’s Implementation of the Science of Reading—and Where to Next, they debate whether Ohio’s governor-elect, Amy Acton, will continue the literacy initiatives championed by Republican Governor Mike DeWine. Can Science of Reading reforms maintain bipartisan support? And which other education policies—from curriculum reform to school choice and accountability—could face challenges under new leadership?
Then, on the Research Minute, Amber Northern examines a WestEd study finding that Massachusetts middle schools that adopted the Illustrative Mathematics curriculum before the pandemic saw significant gains in math achievement, while schools that adopted it during or after the pandemic saw no comparable improvements.
Recommended content:
- The Next Chapter: Ohio’s Implementation of the Science of Reading—and Where to Next —Aaron Churchill, Thomas B. Fordham Institute
- How the Midterm Results Could Reshape Some States’ Reading Policies —Sarah Schwartz, Education Week
- Reform Democrats return to power —Michael J. Petrilli, SCHOOLED
- Election reflections —Michael J. Petrilli, Thomas B. Fordham Institute
- Impact of the Illustrative Mathematics Curriculum on Math Achievement —Noman Khanani, Dalia Zabala, Shannon Stackhouse, Jena Gu, and Kirk Walters, WestEd, 2026
Feedback Welcome: Have ideas for improving our show? We would love to hear them. Send them to [email protected]
Transcript
This transcript was generated with the assistance of AI and lightly edited for clarity and readability.
Introduction
Mike Petrilli [00:00]
Welcome to The Education Gadfly Show.
I'm your host, Mike Petrilli of the Thomas B. Fordham Institute.
This week, we're joined by Fordham's own Aaron Churchill to discuss the future of Ohio's Science of Reading reforms following last week's election.
Will Governor-elect Amy Acton continue the literacy initiatives championed by Republican Governor Mike DeWine?
And which reforms are still doable in Ohio and other states that have flipped from red to blue?
Then on the Research Minute, Amber Northern examines a new study on how the timing of schools' adoption of the Illustrative Mathematics curriculum relates to student math achievement.
All that and more this week on The Education Gadfly Show.
Ed Reform Update: What's next for education reform in Ohio?
Mike Petrilli [00:53]
Hello, this is your host, Mike Petrilli of the Thomas B. Fordham Institute, here at The Education Gadfly Show and online at FordhamInstitute.org.
Now, please welcome today's special guest, Aaron Churchill.
Aaron, welcome back to the show.
Aaron Churchill
Thanks, Mike. It's an honor to be here, as always.
Mike Petrilli
Yeah.
Aaron is Fordham's Ohio Research Director.
Aaron knows more about Ohio education policy, finance, research than anybody else alive and recently moved back to the state.
You know, I don't know. We try not to advertise it that you were doing this job from out of state for a while while your wife was getting trained to be a doctor and other things.
But now you're back in a rural hamlet of Ohio.
Aaron Churchill
That's right.
We live in a small town in the USA.
It's a little town called Dalton, Ohio.
People would like to call it Dalton, but it's actually Dalton.
Mike Petrilli
Okay.
Aaron Churchill
In Wayne County, Ohio, in the middle, roughly between Columbus and Cleveland.
Mike Petrilli
Okay.
Love it. Love it.
Well, we're excited to have you on.
I will say, David couldn't join us today.
He is busy helping to design the new NAEP Civics Assessment, which is a good thing for him to be doing.
And anyways, three Fordham people would maybe be too much.
But we, of course, have to talk about something that happened last week.
I hear there was an election and maybe it's bringing changes to Ohio.
And as part of that, I want to talk more about the future of some of the big reforms in Ohio, including around reading.
We had a big study that came out about that this summer.
We want to talk about all of that.
Let's talk about that on Ed Reform Update.
Mike Petrilli [02:36]
Okay, so, Aaron, you know, when this airs, it'll be basically a week after everybody woke up to this blue tsunami that, among other things, swept in Amy Acton as Ohio's governor-elect.
Just tell people who are not from Ohio and maybe haven't paid attention to this race a little bit about Amy Acton.
What do we know so far, and what has she said about education?
Aaron Churchill
That's right.
Yeah, so the blue tsunami did hit Ohio.
Governor Amy Acton will be our next governor starting in January, taking over for term-limited Governor DeWine, who served our great state for eight years and did a lot of great things in education reform.
And we'll talk a little bit about that, I think, today.
Governor Acton was the former public health director under Governor DeWine during COVID.
So she had a lot of publicity and notoriety for leading the state through that particular time.
And she ran as a Democrat.
I think it was a close election.
I think a lot of people thought it'd be a coin flip throughout the campaign season against Vivek Ramaswamy, who was the GOP candidate.
And she won pretty decisively in November.
I think it was around 52 percent, 48 percent, somewhere in that, in terms of the margin.
So she will be the next governor of Ohio.
On education, as you might expect, coming from the public health sector, she hasn't said a lot of things on education.
But she has talked a lot about trying to fund schools, public education, at higher levels.
She's also kind of taken a lot of the Democratic and teachers unions' talking points around school choice and being critical of the private school choice program that Ohio has and saying that, you know, we need to support public schools.
And that's, you know, that means prioritizing the kids who go to the district schools.
So it has been a pretty district-centered campaign so far around what she's talked about in education.
But we shall certainly see kind of what happens in the next year or two.
Mike Petrilli
Yeah, no, very, very well said, Aaron.
And yeah, you know, so it's interesting.
I mean, so one of the things you said there is she was a DeWine appointee, right?
And, you know, on paper you think, oh my gosh, how is the person who is the public health director during COVID, you know, who was one of the more aggressive ones early on?
You know, Ohio was shut down really before just about, before almost anybody else, stayed shut down, not just the schools, but a lot of the other businesses for a long time.
And so pretty remarkable that she was able to run and win anyways, despite that.
But it does mean she has a very good relationship with the current governor.
And he has already met with her, and they look like they're very chummy, of course, as you would expect somebody that had worked for him.
And he's already made it clear some of the Republicans want, in lame-duck session, to try to clip her wings and, you know, take away some power from the governor's office.
And he's already expressed that isn't something he thinks is appropriate.
The voters have spoken.
So kind of old-school, you know, Republican, put-character-first Mike DeWine.
The other thing that Mike DeWine is famous for is literacy.
He has been out front leading the charge in Ohio on Science of Reading.
This is a legacy project for him.
And this summer, like I mentioned, you had a new report that came out tracking the implementation of Ohio's Science of Reading law.
Ohio, that's what it's called, right?
The Next Chapter: Ohio's Implementation of the Science of Reading—and Where to Next.
So tell us, you know, if you were going to brief Governor Acton or her staff and try to help them get up to speed on how things are going and what they need to be thinking about, if she's serious, and it sounds like she is, to continue making progress, what would you tell her?
Aaron Churchill [06:30]
Yeah, no, great.
So you are correct.
The Science of Reading was really a signature item for Governor DeWine.
He was really passionate about it, to his credit, and is still passionate about it.
He's still governor for the next few weeks.
And he really put a lot of positive momentum for the Science of Reading.
He would be out there in the schools saying, like, you know, look at these classrooms transitioning to the Science of Reading.
It's really benefiting kids.
And he has really been a powerful force for the state and for good, I think, in terms of pushing literacy, especially in early elementary grades.
I think, you know, moving and transitioning to the next governor, I think it's important to kind of understand what Governor DeWine did and the General Assembly did with their Science of Reading law we passed in 2023.
It has many of the components that the other states moving to the Science of Reading have put into place.
Mississippi, of course, and Louisiana are the exemplars in that area.
And certainly we followed in the footsteps of them.
It has a requirement that elementary schools adopt high-quality instructional materials.
So these are curricula that are aligned to the Science of Reading, do not have the three-cueing kind of components to it.
It also had a bunch of professional development for our educators.
So more than 100,000 educators have been trained in the Science of Reading.
It has audits for our institutions, institutions of higher education that train teachers.
So now our colleges of education have to demonstrate that their instruction for prospective teachers is aligned to the Science of Reading.
And finally, we have the reading coaches component as well.
So about $20 million a year, the state is spending on reading coaches.
And these coaches are supposed to be helping the educators in their lowest-achieving schools.
So I think moving forward, implementation, as we've said until our face turns blue, is crucial to making sure that kids actually benefit from these reforms.
I think a couple areas where the next governor is going to lead, certainly in terms of bully pulpit, that is one.
Be out there like Governor DeWine has been.
Say this is important for the state.
Continue to push for it just at a leadership level.
In terms of, like, kind of the details, one of the really interesting things that Governor DeWine has proposed over the past year is that the State Department of Education actually creates sort of an inspectorate-type system where you actually go in and look at whether the schools are implementing these programs and their curriculum and the Science of Reading in alignment with what's expected of them.
And so starting this fall, there have been support teams going out to schools, kind of taking a look under the hood to see whether the schools are implementing the Science of Reading well.
So I think that is one area where the next governor can kind of take it up a notch and make sure that those inspectorate teams are really doing a bang-up job to make sure that our elementary schools are teaching properly and aligned to the Science of Reading.
And the other thing is just continuing the reading coaching program.
I think that'll be an early test.
Is she going to continue to allocate funding for the reading coaches?
Is she going to continue to hold those coaches accountable for improving results in the elementary schools?
I think that'll be really telling and interesting to see with her first biennial budget that will come out later this year.
Mike Petrilli
Yeah.
And the coaches thing is interesting.
You know, we had Liana Loewus on a few weeks ago talking about a new report on Alabama and, you know, a huge investment there.
750 coaches statewide in a very small state, all at the state level.
And that's serious money, you know.
And so that is another question.
And look, Ohio's done a lot of things right, but let's be honest.
I mean, there's some things there that also, they at least give me pause, right?
Our high-quality instructional materials list that the districts choose from is really long now.
I mean, Alabama, we talked about, has two programs on their ELA list, right?
And Ohio has, like, two dozen, something like that, right?
Aaron Churchill
That's correct, yeah.
Mike Petrilli
Yeah.
And so that's, you know, you worry.
You're like, okay, well, at that point almost anything qualifies.
And then the other thing is, you know, a few years ago Ohio Republicans, let's be clear, backed off the third-grade retention policy.
Right?
This idea that says, hey, if a child reaches the end of third grade, still cannot read, still is struggling, they need to be held back and provided tons of extra support so that they do, you know, they get what they need and they can move forward with more confidence.
And guess what?
Huge backlash, as there is everywhere.
People don't like this, even though there's good evidence that leads to big changes in schools.
Because, you know, in places where they've stuck to it, the educators, they hate seeing their kids held back.
And so it seems like it is a motivating force for the superintendent, for the district, for everybody to get their act together and put in place good interventions to help those kids get what they need in first grade, second grade, and throughout third grade so they don't get held back.
Right?
Ohio went backwards on that.
We were hoping that maybe a Governor Ramaswamy would push for it to come back.
I guess with Amy Acton, we'll have to see.
Aaron Churchill
That's right.
I mean, I think, as you mentioned, you know, third-grade retention is probably the most controversial piece of their literacy because it actually entails some accountability for the adults working in our elementary schools and in our leadership positions in our schools.
So that tends to generate the most pushback from the school establishment.
And we certainly saw that a few years ago when they walked back third-grade retention.
I think you are right that the Republican candidate, Vivek Ramaswamy, had talked about bringing back retention, possibly restoring that type of policy.
I think under Amy Acton, I think that is a little bit more of a, that would be a bigger challenge to have that, just because of, you know, where the politics are on that issue, being such a tough issue to get past the school establishment.
But you are correct that there is some good research and good evidence that, even from Ohio, that it did change the behaviors of the teachers and the educators in their school system to really focus on the early literacy component and make sure the kids who are behind get the interventions.
You mentioned the high-quality instructional materials in Ohio too.
It is probably a little longer than I think you or I would wish for.
One thing that is positive on that front in Ohio is that Core Knowledge Language Arts is the most-used elementary school reading program now.
I think there has been movement thanks to, you know, I think an understanding from their Department of Education and trying to encourage, like, the knowledge-rich and content-rich material that the schools did pick up when they were required to transition curriculum.
Core Knowledge Language Arts, Wit & Wisdom is also up there among the top five most-used curricula today.
But you're correct that, you know, that is going to be some, an area that there is going to be pressure, I think, to increase the list and ever, you know, kind of expand, expand, expand, while you don't really sort of roll back or repeal any instructional materials off the list.
Mike Petrilli [14:06]
And this, you know, if we kind of expand this conversation beyond Ohio, you know, I always think about, okay, you know, what can we expect in these states where the governors, the Democratic governors, gubernatorial-elects ended up flipping red states to blue or to purple, right?
Or some of these places now where they've gone from split government, like Ohio will have divided government, to having a trifecta.
You know, we're going to have trifectas in Pennsylvania, Michigan, Wisconsin, Arizona, right?
And then the divided government with the Democratic governor in Ohio, obviously, but also in Texas and in Georgia.
So three big states there.
So, and look, my own opinion has always been that there are certain reforms that are just hard because of union politics, right?
The teachers unions are the big 10,000-pound gorilla in our space.
They have a lot of power.
They're hugely influential in the Democratic Party.
They have a lot of money and resources and manpower to spend on elections.
And so when they don't want something, it's hard to get Democrats to fight them.
Some Democrats will on some issues, right?
But they don't like non-unionized charter schools.
They don't like non-unionized private schools getting public money, right?
I mean, and so beating them on that, it's just about power.
But then there's these other issues that's not necessarily there.
They're still hard, but they're hard for different reasons, not necessarily because of the unions, more maybe because of ideology, right?
I mean, that was the reason we thought for so many years that it was hard to get Science of Reading kinds of things in place.
The ed schools, you know, were true believers in whole language.
And so they would fight it or they would train administrators and teachers to embrace whole language and not the Science of Reading.
Or they were against the content-rich curriculum stuff when it was, you know, E.D. Hirsch out there making the case for it.
And so there it's more about, you know, changing hearts and minds.
And there's other stuff.
It's just technically difficult.
I mean, there in Ohio, you have 600 districts and this whole tradition of local control.
You get these coaches and these teams and these efforts from the state level, but at the end of the day, you're talking about 600 school districts with elected school boards, each with their own superintendent, their own chief academic officer.
It just takes time to try to get all those people on board and then help a teacher get what they need so that they can walk into a classroom and confidently teach this stuff effectively.
All of that is hard, right?
So I guess what I'm saying, Aaron, I don't know if this sounds right, but there's a certain category of reforms that maybe we can be hopeful that these Democrats will keep moving forward on.
Especially around literacy, maybe on math, maybe on career tech, maybe on things that aren't that controversial, maybe, or politically, things that aren't strongly opposed by the teachers unions.
But then there's these other sets of issues, certainly private school choice, charter schools, differential pay, some of these other reforms where, you know, their interest is in trying to defend what they think is best for their members and for their own organizations.
They're going to fight that stuff.
I mean, does that seem right?
Like we should be thinking about, okay, in the next couple of years, how do we keep making progress on the, maybe on the curriculum stuff and just acknowledge that the other stuff we're going to have to play defense?
Aaron Churchill
And I think that's very interesting.
I think, you know, literacy does, you know, it does feel like the Science of Reading has more bipartisan support than, you know, certainly some of the other policies that you mentioned, like differential pay and certainly, like, you know, private school vouchers and those types of things.
And it does seem, and thanks to, you know, the blue states that have adopted Science of Reading policy like Massachusetts and Pennsylvania, you know, I think it does give more of a bipartisan flavor to the Science of Reading, and that this is not just, you know, Mississippi and, you know, very deep-red states that are pursuing the Science of Reading.
And I think there are a number of legislators on both sides of the aisle in our statehouse that have embraced the Science of Reading and understood its importance.
I think the challenge, you know, as always, is when, you know, sort of push comes to shove, you know, do the legislators stick with sort of accountability on some of these things?
So we mentioned retention as sort of, it did collapse, you know, and that was under a Republican trifecta.
And, you know, some of these other initiatives within the Science of Reading, if you have a school district that just says, no, we don't want to do this.
We're going to do, you know, whole language, or we're going to do both, you know, phonics and whole language.
I mean, what happens in those types of situations?
So I think, you know, I think broadly speaking, I think it does have a lot of bipartisan appeal.
The Science of Reading, moving to phonics, making sure that students are learning how to sound out words.
I think people understand the common sense of that.
I think the challenge, again, is when push comes to shove, whether our policymakers will have the leadership and the courage to do what's right for students.
Mike Petrilli
Yeah, no, that's right.
And look, there's still ways that the unions are going to want to get in on some of this.
Like, are those coaches, are they unionized or not, right?
Pretty sure Mississippi and Alabama, those people are not unionized.
So, you know, if you hire a reading coach, gosh, I mean, Alabama hires 750.
Surely even if your process is great, you're going to hire a few duds.
You know, you're going to figure out quickly that, oh gosh, they're not really that good at being a coach or they don't have the expertise.
And so we need to be able to tell you, hey, sorry, this didn't work out.
You know, in Ohio, I don't know.
Some of the State Department of Ed people are unionized.
Some of the people at these education service centers that I think hire the coaches are sometimes.
You know, how does that kind of stuff...
I mean, so maybe I try to, I guess what I'm saying here is I try to put these things in these two very clean buckets, right?
And it's maybe not as clean as I think.
Like, even in the Science of Reading, it will probably just have, it's going to be harder in places where unions have more power if they are trying to, you know, stop some of this.
Or they're trying to say, hey, okay, you want our cooperation on Science of Reading.
Here's what we want in return.
We want a sweetener in our pensions.
We want to change the collective bargaining laws in this way.
Those are things that we have to...
Aaron Churchill
Yeah, I think that's right.
And I think it's also just around priorities.
I mean, Governor DeWine really did make literacy, like, I would consider his number one priority in his second term of administration.
Mike Petrilli
Not just education, overall.
Aaron Churchill
I mean, it was in his State of the State speeches.
He was out there in schools and he really made that the number one priority.
I think the question is, you know, under Acton, is it still going to be a number one priority?
Is it going to fall to number two?
Is it going to fall to number five?
I think that is one question that we'll just kind of have to wait and see.
We'll obviously continue to push that and be a number one priority, but there are a number of other things that I think could easily swamp it.
Mike Petrilli
Making sure kids can read is pretty important.
Aaron Churchill
Yeah, yeah.
Mike Petrilli
But you can imagine, there's going to be a budget negotiation.
She has to deal with the Republican General Assembly.
They're going to want to defend the school choice programs, probably.
She's going to want more money for traditional public schools.
And then the question is, it's like, okay, well, where does that money come from?
And does she look at the coaching, for example, and say, oh well, let's take some of that money and put it into the general fund, into the funding formula?
I think that's the kind of thing where it's a risk, right?
And we need a champion to say, hey, no, no, no.
You need to protect that pot because that's important.
Aaron Churchill
That's right.
Yeah.
Mike Petrilli
Yeah.
All right.
Good stuff, Aaron.
Again, Aaron Churchill, our Ohio Research Director.
Do check out his report.
It is called The Next Chapter: Ohio's Implementation of the Science of Reading—and Where to Next.
It really is excellent.
And I hope that other states with these laws have somebody like Aaron watching how it's going in the real world, because, of course, that's all that matters.
Thanks, Aaron.
Look forward to having you come back sometime soon.
Aaron Churchill
Okay.
Thanks, Mike.
Mike Petrilli
All right, now it's time for everyone's favorite, Amber's Research Minute.
Research Minute: Does Illustrative Mathematics improve student achievement?
Mike Petrilli [22:22]
Amber, welcome back to the show.
Amber Northern
Thank you, Mike.
Mike Petrilli
Just had a great conversation there with Aaron.
And, you know, you'd be interested in this and have something to add, given your experience there in Virginia.
You were maybe a year ahead of us on this blue wave.
You know, we're talking about which kinds of reforms are still likely to be able to go forward in states with Democratic governors.
You know, my argument, basically, that things like Science of Reading and the things that maybe, you know, where the unions, you can get the unions on board.
It's not a union-power sort of thing.
Certainly more doable than stuff where they have a clear resistance, you know, like charter schools, let's say.
And I don't know, like in Virginia, I mean, do you feel like curriculum reform?
Surely, I'll talk about CTE, right?
Amber Northern
Yeah, CTE, that's a good one.
Yeah, that's a big one.
I think we see a lot of interest on the right and left around CTE and getting it right.
And there's a lot of agreement around we don't want, you know, IRCs that don't have a lot of potential for kids.
So that's a big one.
I also think that, you know, beginning to think about how we incentivize teachers to go to high-performing schools, a little touchy.
But I think in theory we've got a lot of support for that.
Obviously, the mechanics of how it's done, you know, makes a difference.
So, yeah, I think that.
And also I think people are trying to figure out what is it, how do we have a Science of Reading and a science of math, and what would that mean?
And I think there's interest on both sides for that as well.
Mike Petrilli
All right.
I know.
Has Virginia done much on the high-quality instructional materials in either reading or math?
Amber Northern
Oh, yes.
We have our approved list.
Mike Petrilli
Okay, you do?
Okay.
Amber Northern
And also approved lists for core, for supplemental, for intervention.
We get a lot of lists.
Mike Petrilli
Uh-huh.
And do the school divisions have to actually follow them?
Amber Northern
I mean, at the early grades for the Science of Reading, those grades, and then it becomes voluntary as the kids get older.
But yeah.
But yeah, you know, and I'm pushing on fewer, fewer, fewer options on that list because we know it's hard to get them all implemented well.
Mike Petrilli
We just talked about that.
Amber Northern
Absolutely.
Mike Petrilli
But Aaron told me, and I don't think I even knew this, that Core Knowledge Language Arts from the state of Virginia, Commonwealth of Virginia, I should say, is number one, is number one in Ohio, which is amazing.
I mean, those years of E.D. Hirsch out there in the wilderness trying to make the case for content-rich English language arts curriculum, and it's finally happening.
Amber Northern
Yeah, yeah.
I know.
Now we need to evaluate it, which is going to be the topic of YRM.
Mike Petrilli
Okay.
Well, good segue.
Let's talk about what you've got for us on the research.
Amber Northern [25:10]
All right.
We are talking about how there are so few studies on HQIM, period.
So I dug up this one.
It's a little, I think it was at the end of last year, but it didn't cross my radar, partly because I wasn't here for part of last year.
But it's from WestEd.
It's evaluating Illustrative Math, which is the big popular one.
They want to know if it improves student achievement, like all of us want to know that.
It consistently gets top marks from EdReports.
And then I dug up and tried to find out, you know, exactly, did we have good percentages around take-up?
And apparently RAND says it's used in about 7 percent of districts nationally.
And in Massachusetts, where this study takes place, it's used in 20 percent of middle school districts as their core curriculum.
So anyway, and there's just a lot of hubbub and buzz around Illustrative Math, given that some Common Core writers were involved as well in the development of this curriculum.
So my kids, I saw it, at least I think with Leandra was, I think maybe it was Nico was too early or too young or too old.
But yes, Leandra did Illustrative Math.
Mike Petrilli
I'll be curious.
But if you looked at any of his homework, it's aligned to the Common Core.
Amber Northern
It emphasizes conceptual understanding, mathematical coherence, and student-centered problem solving.
It's most popular in grades six through eight, growing adoption in K-5.
In this study, they identified 101 schools across 55 districts using Illustrative Mathematics in at least one middle school grade during the 2024-25 year.
Because districts adopted it in different years, they've got 19 schools that adopted it before COVID, and the remainder started in 2020, 2021, or later.
So they've got the staggered adoption timeline they can use.
The authors compared trends in MCAS scores before and after IM adoption in treated schools versus matched comparison schools.
What drove me a little bit crazy is we did not have information, and David always brings this up and I think it's a good point, of what the comparison schools were using.
We're told that the comparison schools also used high-quality instructional materials, HQIM, rather, but we don't know which one or ones.
And that's a little unfortunate because we're just not sure what we're comparing to here.
Anyway, the study goes back to 2016-17.
So those schools had two years of pre-adoption baseline before they started with IM.
So you've got kind of a range of schools at different places.
The schools were matched on prior achievement and key demographics.
And then the authors looked at the impact of IM in each of the first three years following adoption, and then separately for the schools that adopted before and after the pandemic.
Because, as you can imagine, the pandemic was kind of a big deal.
And they surveyed math teachers in 28 IM schools and asked them about implementation fidelity.
Okay.
Finding number one: Overall, in the aggregate, IM schools did not perform significantly differently from their matched counterparts.
Pooling across all 71 treated schools, effects were null in years one and two, and they were barely significant in year three.
Number two: Results diverged sharply depending on when schools adopted IM.
So the 19 schools that adopted before COVID, so that was 2018-19, had significant positive effects.
So 0.18 standard deviation in year one.
That grew to 0.39 by year three.
The 52 schools that adopted IM during or after the pandemic, null effects every year.
So it appears, we could talk about this, it appears to be the case that the pre-COVID schools stood out because maybe they had this one year of implementation before the pandemic.
And then when it hit, it looks like these comparison schools declined while the IM schools held steady.
And then those effects accumulated throughout COVID, not after it, because I looked into that.
And the gap grew by year three, largely because those comparison schools never recovered.
And then number three: Among the pre-COVID adopters, those ones that did show some promise, they were concentrated in grades seven and eight.
So grade eight showed effects of 0.40 standard deviation in year one.
That grew to 0.43 by year three.
And then grade seven reached 0.50 by year three.
Grade six, positive, but not as statistically significant or large as grade seven and eight.
And then for those post-COVID adopters, again, effects largely null.
But when you look in the grades seven and eight, those gains appeared to apply to a small group of schools that actually had time, again, before COVID hit to prepare and have all this PD.
And it also seemed like those schools were less economically disadvantaged than the later adopters, which had something to do with this.
And then they have this long discussion where they say, you know, the pattern reflects implementation conditions around COVID.
And they looked into this implementation survey I mentioned, and the instructors and practitioners said that that first year of adoption included a lot of deep professional development and collaborative time and all that stuff.
And the COVID-era adopters didn't get those things.
And they say, you know, it looks like there's some promising stuff happening here in middle school math, but you got to have these strong conditions in place.
And that said, it did also seem like their sample was skewed to higher-achieving, less economically disadvantaged populations because, and this is a finding in itself, Mike, when they looked at the more diverse districts and the lower-performing districts, one condition of participating was that you had to have adopted one curriculum.
And those urban areas had adopted multiple curricula, so they weren't eligible.
So I'm like, okay, why is that happening?
So anyway, that's partly why their sample skews a little bit on the more advantaged side.
But anyway, it's messy.
You know, it crosses COVID.
So some of this is hard to figure out.
But we need more of these studies.
And what I also liked about the study is that it was not funded by Illustrative Math.
Mike Petrilli
Hey, yeah.
Amber Northern
It was actually funded by Gates.
So that's even more rare because sometimes when you see these studies, they, of course, are funded by the developer.
So that's what I've got.
Mike Petrilli
All right.
Very interesting.
You know, I remember us talking way back when, maybe during COVID, about how among the many problems, and I'm sure this was 99th on the list, I mean, we had many other big problems with that, this was going to be a problem for education research, right?
That you were going to have, you know, these studies that maybe even started before COVID.
But then how do you make sense of this enormous disruption?
And yeah, so this is one where you're like, okay, this is really tough to figure this, to tease this out.
I mean, maybe now, okay, we're far enough past that we can start to see more and more studies that are completely in the post...
Amber Northern
Right.
That's right.
Mike Petrilli
So we don't have that complication.
But you're right.
We need more of these.
You know, one of the interesting conversations that has happened with the new Sold a Story by Emily Hanford, which is about, you know, capital-D, capital-I Direct Instruction.
That's right.
Engelmann was, you know, he was obsessed about curriculum design, instructional design, testing, you know, precisely how you explain things to kids and what worked and what didn't.
That's why he ended up, you know, suggesting these scripts.
It wasn't because he didn't trust teachers.
It's like he really figured out that, no, he tested it.
If you say, if you teach the letters in alphabetical order, that doesn't work as well as if you teach the letters in this different order.
You know, and so you should, you know, we figured that out.
So you should stop teaching them in alphabetical order.
So, you know, there was a sense even back then, and I'm sure rudimentary ways, no computers or early computers, I mean, to just constantly be testing, you know, what's working, what's not, what do the results say?
And so all we've been able to do for the most part is ask, well, do these new HQIMs appear to be aligned to the standards?
That's an important question, right?
Do teachers find them usable?
Okay.
But the big question is, do they work?
Amber Northern
Do they work?
Yes.
Mike Petrilli
This is certainly important.
So good for Gates.
It would be good to get the Institute of Education Sciences back online.
Amber Northern
Yes.
And that's where I was going, right?
With all of these curricula that are now digitized and all of these sort of back platforms, right?
They can do all of these experiments, right?
Like, you can send some kids one way of teaching and you can send some kids another way of teaching it.
Like, we should be running all these experiments behind the scenes with these digital curricula and actually seeing, you know, when we assign kids this task versus this task, you know, how does that impact achievement and their progress on tests?
And so I hope we see more of that because sometimes the developers like to keep those data proprietary to themselves because maybe it might not be, you know, all that welcoming to the curriculum.
But we have the possibility to do much more sophisticated randomized studies than we once did.
Mike Petrilli
And that's saying something different than I did, right?
Which is that, and therefore, Illustrative Math and CKLA and the other ones should keep evolving, right?
So rather than just be like, okay, this was Illustrative Math at one moment in time, and hopefully if they've got good data...
Amber Northern
That's rigorous.
Mike Petrilli
They can keep tweaking and improving, and why this piece worked but that piece didn't.
And so we're going to make it better and better.
Amber Northern
That's right.
Mike Petrilli
And it seems like with digital materials, that's easier.
I mean, we had all these conversations on the show and elsewhere about screens in schools and the pros and cons.
And, you know, it is tempting with AI and everything else to just go back to paper and pencil.
But it is pretty nice, the notion that, hey, you don't have to wait 10 years to put out another textbook edition.
Once you figured out that the fraction units are confusing everybody, you could actually change it right away.
Amber Northern
Right.
There are mini experiments, M-I-N-I, right?
We can do many, but we can also do mini ones to get at this.
And I mean, I won't even sort of blow your mind more, but there's all this talk about these synthetic databases that aren't real student-level data, but they mimic it pretty darn closely.
And so being able to use those also to run synthetic experiments that are better data than we've been able to run with synthetic data systems in the past is also really important.
And other people could talk about that better than I can, but my gist working at IES was that there were all of these newer synthetic data systems that were going to be able to mimic real kids much better than the data systems we'd had in the past where we could, you know, because we can't use the actual student data.
But anyway, I'm over my skis.
But all I'm saying is technology has some positive impact in terms of experimentation with curriculum.
Mike Petrilli
I'm going to nod along and pretend that I know what the heck you're talking about.
All right.
Great stuff, Amber.
Good study.
I need more of them.
That's good.
All right.
Well, that is all the time we've got for this week.
So until next week.
Amber Northern
I'm Amber Northern.
Mike Petrilli
And I'm Mike Petrilli of the Thomas B. Fordham Institute, signing off.
