Mike Miles, superintendent of Houston Independent School District, joins Mike Petrilli and David Griffith to talk about his new book, By Design: How to Transform School Systems, and how districts can use systemic thinking and accountability with support to improve schools.
Then, on the Research Minute, Brian Fitzpatrick reviews new research on how classroom context can affect teacher evaluations, and how schools can use the logic of value-added models to develop better evaluations.
Recommended content:
- By Design: How to Transform School Systems —Mike Miles
- State takeovers are back —Michael J. Petrilli, SCHOOLED
- Texas’s controversial takeover of Houston’s schools—Daniel Buck, for the Thomas B. Fordham Institute
- Classroom Composition Affects Teacher Performance Ratings —William Delgado and Lauren Sartain, EdWorkingPapers (April 2026)
Feedback Welcome: Have ideas for improving our show? We would love to hear them. Send them to [email protected]
This transcript was generated with the assistance of AI and lightly edited for clarity and readability.
Transcript
Introduction
Mike Petrilli [00:02]
Welcome to The Education Gadfly Show. I’m your host, Mike Petrilli of the Thomas B. Fordham Institute.
This week, we talk with Houston Superintendent Mike Miles about his new book, By Design: How to Transform School Systems, and how districts can use systemic thinking and accountability with support to improve schools.
Then Brian reviews new research on how classroom context can affect teacher evaluations and how schools can use the logic of value-added models to develop better evaluations.
All that and more this week on The Education Gadfly Show.
Clip [00:36]
This is The Education Gadfly Show.
Well, I think that’s a good place to start, Mike.
I mean, what do you say?
You’re both Mike.
Good grief.
What does Gadfly say?
Mike Petrilli [00:52]
Hello, this is your host, Mike Petrilli of the Thomas B. Fordham Institute, here at The Education Gadfly Show and online at FordhamInstitute.org.
Now, please welcome our special guest for this week, superintendent of Houston Independent School District, Mike Miles. Mike, welcome back to the show.
Mike Miles
Thanks, Mike.
About to get in the ring or something.
Mike Petrilli
Yeah, we need some walk-up music here for that as well.
Also joining, as always, my co-host, David Griffith.
David Griffith
Superintendent of none, right?
Mike Petrilli
Yes. Lucky you. Lucky you, David.
Well, Mike, it’s great to have you back on the show. I think we had you on just as you were beginning your work in Houston. And now you are back because you have written a book somehow while turning around one of the nation’s largest school districts.
You’ve also written a book, and it’s a very good book. I was very honored to blurb it, and we’re excited to have you here to talk about that book. It’s called By Design: How to Transform School Systems. Let’s talk about it on Ed Reform Update.
Ed Reform Update: Transforming school systems by design
Mike Petrilli [02:05]
Okay, well, Mike, I want to get right into it. Some other time, you’ll have to tell us when you were finding time to write a book. But listen, it’s such an important book, partly, of course, because we’re all fascinated by what you’re doing in Houston, which so quickly showed very promising results in student achievement and other outcomes and seemed to take so many different ideas from the education reform world.
I want to jump right into that because, I’ll admit, as a reader, I was trying to figure out what this means. You talk about system transformation and systemic reform. Many superintendents are trained, when they go to get their doctorates or go through prestigious programs, to say, “Here’s what you need to do when you come into a school system or try to turn things around: jump into fixing operations, adopt a new curriculum, fix the central office, fix the HR program, fix how teachers are evaluated.”
And you write in the book that, instead, you really need to think about changing the system itself. So can you explain that to us?
Mike Miles
Sure. And Michael, you kind of set it up the right way. I think you’ve got it.
I get lots of calls from my colleagues, and not just now, but over the last several years. People want to know how to transform quickly. They’re looking for the secret sauce. In our profession, people want the secret sauce.
And I think most people realize there’s no secret sauce, but there is the traditional playbook. So they’re looking at curriculum. What is the best curriculum to adopt? What textbook should we adopt for math? We do that every four or five years, and we change it around. We focus on that. We bring in trainers. People want to improve professional development and want to know how professional development will stick, because most of the time it’s just sit-and-get and then teachers go back to the classroom and nothing happens.
And on and on and on, through numerous reform efforts.
We’re also taught to go slow to go fast. Go slow to go fast. And do one thing well at a time. In fact, the whole last twenty or thirty years, I argue, is all about piecemeal, incremental reform and change, layering changes onto a broken system.
Imagine you have to fix your father’s Oldsmobile, and you’re just tweaking and layering improvements onto that old engine instead of coming up with a new EV motor or some other systemic reform.
So I argue in the book that you have to go beneath all of that first. And I give several examples because it is hard to do. It’s not what we usually do.
Take curriculum, for example, a textbook adoption. Right now, the profession is all into high-quality instructional materials. That’s a good thing. Science of reading materials, that’s a good thing.
But let’s say someone calls me and says, “What should we do?” And I say, “High-quality instructional materials.” They say, “That’s great.” And they go out and do that. They find the best textbook. They do what we did in the past, which is bring several textbook companies into the gymnasium, set up their stations, have the teachers and principals walk around and ask questions, and then finally adopt one.
Then I’m going to ask the superintendent, “All right, great. You adopted it. Are teachers supposed to use it? Can they opt out of it?”
That’s an important question.
And the superintendent says, “No, they can’t opt out. We spent a lot of money. They’re going to use it.”
Okay, great. How do you know they’re going to use it? And what if they don’t use it well? Do they have to use it well?
“Yes, of course they have to use it well, because we’re going to bring in these outside consultants from the textbook company and they’re going to train them.”
Okay. They’re going to train them how often? I’ll get an answer, and then I’ll ask, “All right, what if they still don’t use it well? How will you know?”
“Well, we’ll get a report from the professional development and we’ll see how the teachers feel in the survey of the professional development.”
Well, okay, that doesn’t tell me.
“Then we’re going to get principals to monitor.”
How often will principals monitor? Does it matter? What if they find out that the teacher isn’t using it well?
“Okay, well, then we’ll do more professional development.”
And I can go on and on. That’s what we do. And we should not be doing that.
So you have to go to the underlying conditions. I call them the condition-setting measures, the things that every school and every district need if they’re going to implement well.
On top of that, we are so used to the old system that the system is going to try to fight change. There is status quo bias. There are regression effects. And I talk about all of those things in the book, because superintendents should be aware of them.
So I talk a lot about condition-setting measures like accountability. What is your accountability regimen? In other words, is there accountability for outcomes? And what does that look like?
First of all, the profession has gone away from the A word. We don’t like to talk about it a lot. But when we do, it’s just thrown here or there without any notion of what real accountability is.
What are expectations, and how do you maintain them?
Imagine you aren’t clear about that. In Houston’s case, when I arrived, there were 274 autonomous schools. No clear expectation of instructional quality. No definition of what good instruction looks like.
So are you setting that out clearly? And how do you maintain that?
Expectations, accountability, autonomy. What is the level of autonomy? Who has autonomy? Who doesn’t? Does everybody have autonomy? Are schools their own domain? Are principals the captain of their ship totally, or are they in an armada with some rules?
Earned autonomy and accountability with support
Mike Petrilli [08:58]
Yeah. And I’m glad you raised the autonomy piece. This is something I’ve really been struggling with lately because, at Fordham, we for a long time bought into this notion of accountability in return for autonomy. The charter school model. Checker wrote about this decades ago.
But it turns out that it’s really hard to make that work. Back in the 90s, people were saying, look, the problem is that a lot of people in the schools may say they want autonomy, but they also need a lot of support. They need capacity. They may not make good decisions when it comes to curriculum. They may not know what instructional excellence looks like. They haven’t been trained on that. Or maybe they haven’t been lucky enough to work in a fantastic school, so they just haven’t been exposed to it.
So maybe we went too far or were naive about the kind of autonomy that can work, especially in a system like you’re talking about, in our traditional public schools, where there really isn’t autonomy because the teachers union contract has all these rules and limitations in it. Or just the culture that has developed over many decades keeps principals from having true freedom of action.
There are all these issues that you really have to get into if you’re ever going to get to the point where you do want to empower people. You don’t want total command and control, but you can’t be naive about it either.
Mike Miles
That’s right. And I’m all for autonomy, but it has to be earned. So we have an earned-autonomy framework.
And it has to be talked about ahead of time, not after the fact, especially if you’re a failing district or failing school.
When I arrived in Houston, there were 121 D- and F-rated campuses. Two hundred seventy-four autonomous schools received 121 Ds and Fs and a whole bunch of Cs. Autonomy was not working.
Let’s earn that autonomy.
Yes, let’s be realistic. We do have some capacity issues. Teachers work hard. I don’t care where they are. It’s hard to be a teacher in today’s profession, 100 percent. But that also doesn’t mean that every teacher can teach well. And I’m sorry, our students need effective teachers. We have some who are not.
We need to be honest about that instead of saying, you know, we ourselves, our profession, are to blame when people say, “Those who can’t do, teach.” I hate that, right? That just drives me crazy.
Because then try to be a teacher in HISD. I tell you what, you better be an effective teacher. Not anybody can do this job. It is tough, tough work.
But the profession has let ineffective teachers remain in the classroom for so long, and that’s one of the problems with the unions. They have made the public think, “Oh, okay, you’re going to allow ineffective teachers to be there, so anybody can do that job.”
And they can’t. It’s a tough job.
Now, you said something that’s very important, Michael, and I say this all the time also: accountability without support. You said if you’re going to have accountability, you have to have support. And I agree 100 percent.
Accountability without support breeds fear. That’s clear. But accountability with strong supports leads to a high-performance culture. And that’s what we need.
So those are three of the six condition-setting measures.
The others are that you have to understand the principles of decision-making. What are your rules for who makes which decisions when? Even at central office, there are decisions that cut across departments. Who gets to make those tough decisions when they cut across departments?
How will you monitor? Are you monitoring? If you have one executive director monitoring twenty principals, that’s a recipe for failure. Everybody needs to be monitored, including the superintendent. Everybody needs to be monitored and held accountable to their evaluation or their metrics and goals.
And then the last is probably the hardest: What do you do when there is persistent resistance to change? Do you just allow that? Several teachers say, “I don’t want to use the curriculum. I like what I’ve been doing.” What do you do?
We’ve got to get around this notion of letting teachers do what they want behind closed doors. That’s not a profession. Let’s get back to professional culture.
One of the biggest things we do, and one of the things I describe in the book, is paradigm shifts. There are twenty-five different paradigm shifts, mental models that we’ve changed in Houston. One of those is this notion of teacher buy-in.
We ask people to buy in when they join the organization. That’s your chance to buy in or buy out.
When you join the organization, we have an employee value proposition that we put to you before you sign the contract, before you say, “I want to work in HISD.” It says you will have pay for performance. Your principal will be observing your classroom a lot, in some cases daily. You only have ten PTO days, plus five if you’re a veteran teacher and have them saved up. Once you go over that, you will be removed from the district.
On the other side of this employee value proposition, we say you get the highest salaries in Texas. You get the best professional development. You’ll have your curriculum provided for you, et cetera, et cetera.
And twice in that document, it says, “Choose.” Choose the best fit for you.
Mike Petrilli [15:25]
Yeah. And of course, this is a very different value proposition than most school systems give to teachers.
David, let me get you in here. What’s on your mind?
David Griffith
Well, I think that’s a good place to start, Mike. I mean, what do you say? You’re both Mike. Good grief.
Other Mike, first Mike, Mike number one, what do you say to people who say, “I just can’t do this in my district”? There are more rules that I’m not allowed to break if I show respect for contracts and basically the rule of law, right?
Isn’t some of this just something that you were allowed to do because you were given more freedom than the typical superintendent has in a big district?
I mean, you are appointed by the state. You’re given sort of dictatorial power in a way, right? This is not what most superintendents enjoy. They have to report to an elected board. They have a union, et cetera, et cetera.
Can other districts do this?
Mike Miles [16:19]
Yes. And yes. And yes.
We in Houston were able to move faster than others because we have a board of managers who still can veto on policy issues, approve the budget, approve my contract, renew or not renew my contract, and things like that.
But you’re right. They are not an elected board. And so we can move faster.
In Texas, we don’t have master agreements, but we have unions, and unions have power at the ballot box when it goes to electing board members because we have low turnout during board elections, like most places. A union can really sway who gets elected.
So we do have some constraints.
At the same time, we did many of these reforms in Dallas, for example. In Dallas, we put in the largest pay-for-performance plan in the nation back in 2013, and it’s still going. And now Houston has the largest pay-for-performance plan in the nation.
The things about looking at the condition-setting measures: Why can’t you talk about your monitoring regimen prior to the school year starting and prior to choosing your operations? The answer is, you can.
Why can’t you talk about defined autonomy and define the levels of autonomy, even with an elected board?
Why can’t you teach at grade level, which is one of the paradigms we shifted? We teach at grade level. And that means you have to have good scaffolding and differentiation, yes. And there’s a way to do that.
You can think about pay for performance. That’s a hard one to pass, but it can be done, and so forth and so on.
So there are many things that superintendents should do anyway. Why not? Why aren’t you outlining your expectations and how you’re going to maintain those expectations?
In Houston right now, if you were to go to any school, any school, and ask any teacher, after they come back at the end of this week, give them a couple of days to retrain, and any time during the year you come, you will see that 10,000 teachers will have one answer for what the quality of instruction is.
That same answer will be the same answer if you ask any principal, the same answer if you ask any executive director of instruction, the chief of schools, the deputy chief of schools, and the superintendent. All on the same page.
That can happen with an elected board and a union. But we don’t do that in our profession because we don’t spend enough time on the condition-setting measures and the systems.
I talk a lot in the book about system regression effects, meaning every system is trying to pull you back to the way the status quo was. And you need to be clear about that. You need to be aware of that so you can identify it when it happens or before it happens, and put in measures and steps to mitigate those or prevent them.
Those are some of the things every superintendent can do.
We also talk about instructional coherence, which is our bread and butter. Systemically, what does that mean?
That means even your staffing should be tied to the quality of instruction. What does that mean? It means you should do a performance interview when you’re hiring people, tied to the quality of instruction.
When you remove people, your retention policy, who you’re going to retain and who you’re not going to retain, should be tied to the quality of instruction and how effective the instruction is.
Otherwise, you’re just hiring people to fill slots. You look at the resume, and fine. That doesn’t tell you a lot about the quality of their instruction. You see what I mean?
I can go on, but it’s this kind of systemic thinking that improves school systems, even with an elected board.
Finally, I’ll say this. More and more districts, and more and more states, should think about accountability. And they should think about moving expectations higher, not lower. They should also think about changing their governance system and the impact of the unions so that kids will improve.
Sometimes people ask me, “Should I go to a place with a strong union? If I go to a place like that, I can’t do what you do.”
And I say, well, if you really want to make an impact, choose a board or choose a district that wants change, that wants to do it. And I hope this doesn’t offend too many people, but if you go to a place like Chicago, you’re going to have little impact if you want to be transformative. So just know that, or go somewhere else.
Mike Petrilli [21:42]
No, that is good advice.
And I have to admit, look, if I had to put my money on it, I’d say some of the high-performing charter networks, which, look, not all of them do all of these things well, and they have their own struggles, but they tend to be places where there’s a clear value proposition. They know what they’re about. They’re intentional about the people they hire. They have that shared sense of mission.
I love this notion that you’ve made Houston an attractive place to work. And once that happens and you get way more applicants than you need, because you’re paying people so well, suddenly you can be selective. The mindset in so many other places is like, “Well, we can’t be selective because we’re just desperate for warm bodies, so we have to take whoever we get.”
Mike Miles
Yeah, Michael. So three years ago, the year before my team got there, the vacancy rate at the start of the school year was 544. Last year at the start of the school year, we had two teacher vacancies. And this year, I think the last count was under twenty. We have 10,000 teachers, so that’s a really small vacancy rate.
And actually, we have fifty additional teachers already this year. The vacancies are in specialty positions, like bilingual pre-K or career tech ed positions, like a welder or something like that. So we actually have more teachers than we need. But just like anywhere else, we lose teachers during the year, so we have an extra pool to fill the spots.
Mike Petrilli [23:26]
Well, Mike, we could keep going, but we’re short on time. But people, please dig into this book. There’s so much there for all of us to learn from. By Design. Check it out wherever you buy your books.
Superintendent Miles, really appreciate you coming on the show.
Mike Miles
Thanks a lot, Michael. David, thank you.
Mike Petrilli
All right. Now it’s time for everyone’s favorite, Amber’s Research Minute.
Research Minute: Classroom composition and teacher ratings
Mike Petrilli [23:54]
Brian, welcome back to the show.
Brian Fitzpatrick
Thanks so much for having me.
Mike Petrilli
Yeah. Brian Fitzpatrick filling in again for Amber Northern. Don’t worry, people. Amber will be back. Don’t think she’s gone back to the Department of Education or anything. That has not happened. She’s just at the NCSL conference this week, talking to those state legislators. So Brian filling in, which we appreciate.
David, I realize that you and I kind of didn’t overlap that much during the World Cup, so we didn’t even get to banter about the biggest event of the year, which is a real shame.
David Griffith
It’s true. I thought it was overall a very entertaining event with a somewhat unfortunate finale. I’m not going to hate on your favorite country, Argentina, too much. But you need heroes and villains, and the World Cup had both.
Mike Petrilli
Yeah. Hey, my second favorite country. Okay, come on, USA.
David Griffith
Yeah, we were both the heroes and the villains. Listen, I think it’s better to be in the hunt than to actually win. I think true fandom lies in the journey.
Mike Petrilli
No, look, I love the fact that it feels good this summer. People, I mean, it was joyful. The World Cup was joyful.
And then, you know what else seems joyful? I haven’t seen it yet, but The Odyssey. I feel like all this reaction is like, oh my God, it’s a wonderful, epic, incredible movie, old-style Hollywood. And people are happy about it. I just feel like people want to be happy about these pop culture events this summer.
Brian Fitzpatrick
A lot of joyful killing, right, Michael? A lot of joyful killing of Trojans.
Mike Petrilli
Well, my son did point out, Nico is upset that they call it The Odyssey, even though big chunks of it are actually from The Iliad.
David Griffith
Smart kid.
Mike Petrilli
I know. And you would think they would have thought this through. They could have had two movies instead of one. I don’t know why they didn’t do The Iliad and then The Odyssey.
David Griffith
Well, they kind of did with Brad Pitt.
Mike Petrilli
Oh, okay. All right.
David Griffith
Yes.
Mike Petrilli
I don’t know. It’s not a sequel, right?
David Griffith
True.
Mike Petrilli
All right. Enough on that. Brian, let’s do research. What do you have for us this week?
Brian Fitzpatrick [26:01]
So this week, we’re talking about a new working paper by William Delgado and Lauren Sartain. It’s entitled “Classroom Composition Affects Teacher Performance Ratings.”
The main finding is in the name, but they did an interesting paper where they begin from the premise of, hey, look, we’re researchers. We like value-added models for assessing teacher quality because they allow you to net out, to control for, the things that teachers can’t control, namely student composition.
But then they say, okay, despite all this, in the Race to the Top era, classroom observations really beat out value-added models in terms of the scale at which they are used.
So they say, well, let’s look at these classroom observations and see if they have this failing that we might expect them to have, of not netting out differences in student composition across classrooms.
To do that, they use data from the Chicago Public School system, CPS, and they rely on two outcomes.
The first is classroom observations, mostly done by principals and assistant principals. The second is this apparently rising phenomenon, in middle and high schools, of having students rate their teachers. Now, in Chicago, the second bit of this is low consequence. It doesn’t actually go into teacher ratings. But what’s interesting is that they use it in their analysis, and they find the exact same thing for both of these outcomes, for the AP and principal ratings and for student ratings of classrooms.
They use a teacher fixed-effects design, which it’s easy to think about as comparing teachers to themselves. So think in an elementary school: If a teacher goes from having zero kids with a history of suspensions, which is a key variable here, to having one in the next year, that can be very disruptive, as you might imagine.
The other key variable is previous test-score data. So test scores from the previous year on all these students. They’re saying, okay, what happens when you just so happen to get a classroom with a lower average previous test score?
Their main finding is unsurprising. If you just so happen to have students with lower test scores or more suspensions in their histories, you get lower classroom observation and student ranking scores, as we might expect.
Now, one minor gripe with the paper is that in the abstract, they say they don’t find any difference by student demographic characteristics, which I was about to report as this wonderfully optimistic finding. I do wonder whether they overstated it a bit.
Once you dig in, what you discover is that, in fact, being assigned more male students, more Black students, more poor students, and more students with special education status, all of that means lower average ratings for teachers. It’s just that it operates through the fact that all those student groups have lower test scores and higher suspension histories.
Then they end on this wonderfully helpful note, unlike most academic papers. They point to a policy problem, and then they provide a solution, which is that we can essentially use the logic of value-added models and apply it here and adjust these ratings based on student composition.
We don’t need to go into detail on how you do that, but they show precisely how you do it. And they provide this tool to anyone who’s interested, which I thought was a great note to end on.
So yeah, that’s the paper. What are you guys saying?
How should teacher evaluations adjust for context?
Mike Petrilli [29:55]
It’s really good, Brian. I feel like there have been a few papers like this in the past, but I don’t know if they got into the mechanisms. There was this sense that, oh, teachers in high-poverty schools are being punished, in effect, for teaching tougher groups of students and therefore getting lower ratings. Now we have some sense of what that mechanism is.
Of course, I do wonder how many places are still doing this, at least teacher evaluations that matter. D.C., I guess Houston, as we just heard from Mike Miles, and some in Texas writ large with the teacher incentive allotment. There’s some of this going on.
There are certainly places all over the place where teachers are getting feedback, and maybe it is still in the guise of an informal teacher evaluation, so this is something they should consider.
But here’s the thing: You are sometimes going to be given a tough class. If that is the challenge, then again, if the goal is to give you feedback and to improve your instruction, you can’t just control for that and wish it away. You need to be given good advice by your instructional coach, assistant principal, or somebody on how to deal with the fact that you’ve got these kids who have disruptive behavior in their history or are lower achieving.
Am I thinking about that right, David? What do you think?
David Griffith [31:22]
Yeah. I mean, I think where my head goes is, is perfection the goal? If it is, then I think we’re in trouble, because I think it’s probably unattainable.
Teachers are not alone in sometimes being dealt a difficult hand and sometimes getting a worse evaluation than they deserve. It’s funny you mentioned the World Cup earlier, Mike. The literature on coaches, for example, shows that they explain maybe a fifteenth of a team’s performance, right? Which means we’re basically guessing when we say Mauricio Pochettino did a great job or a terrible job with the U.S. men’s national team. It’s mostly about the players.
And I think when you are walking into a classroom and evaluating it, it would probably be an exaggeration to say it’s mostly about the students, but it is a nontrivial amount. I do think that we owe teachers a certain amount of grace when they have a challenging, obviously challenging hand.
I don’t think Pochettino should be fired if he doesn’t win the World Cup. Similarly, I think when a teacher is struggling with challenging students, we need to measure two or three times and cut once.
I guess the other thing I would say is we have to find our refuge in multiple measures. I’m not sure that student evaluation should be one of them. That’s one of those ideas that I liked the first time I heard it, and then a decade later, I’m like, maybe not.
I do think it makes sense to balance test scores, value-added, with administrator or master-teacher observations and just do our best.
I don’t know if that answered your question, but I think that’s where I come down.
Let me just say really quickly, the controlling-for stuff, I don’t know. I’ll be interested to see what Brian says about this. I’m not against it, but I also wonder if people knew that was going to be happening, wouldn’t it change the behavior of evaluators somehow? I just wonder if it would really, in the end, work.
Mike Petrilli [33:46]
Yeah. But again, it depends on what it’s for.
If it’s a high-stakes thing, like your salary or your retention is tied to it, that’s one thing. If it’s really about, hey, we’re trying to improve instructional practice, which is where most places are at this point, how do we regularly go into a teacher’s classroom and give feedback? I don’t know.
What do you think, Brian?
Brian Fitzpatrick [34:07]
Yeah, I think there are a couple of pieces of that.
I totally agree with the distinction between if you’re going to do a $10,000 bonus like they do in Texas, then yeah, I think you have to do everything you can to control for things outside of the control of the person being evaluated.
But insofar as worse instruction is happening in classrooms that are more disruptive because they have more disruptive students, that’s not a measurement problem. That’s accurately measuring reality.
The other piece of this, I think, is that Winston Churchill quote, that democracy is the worst form of government with the exception of all the others we’ve ever tried.
There’s a certain sort of anti-reform person who will read a study like this and say, “Look, this is just evidence why we shouldn’t be doing this whole evaluating teachers and judging them based on their quality, because they know best and we shouldn’t be doing that sort of thing.”
Well, you know, any system like this is gameable. You would come up with a way to game this new system. Get as many disruptive students as you can into your class to get the most adjustment upwards and then figure out a way to have them not show up the day of the evaluation.
Mike Petrilli
Nobody would choose that. Nobody.
Brian Fitzpatrick
Right. Extreme example. But yes, anything is gameable.
You could imagine, Mike, you were saying essentially that the observers themselves, who might currently be saying, “Oh, they had this disruptive kid they had to deal with. I’ll give them some grace for that.” Well, now, if you know an adjustment is going to happen, you’re not going to give that grace because it’s going to work on the back end.
All sorts of things could happen here, but I do think this is directionally right.
Mike Petrilli [35:51]
Yeah. And a couple more thoughts.
This is one reason why I think we should pay people more in tough schools. I know it’s different than a tough classroom, but let’s say tough schools, high-poverty schools, should just get paid more. There should be an incentive-pay thing that’s just about serving in tougher schools.
And then on top of that, maybe we do the performance-pay thing. But if you do that, at least we’re trying to recruit good people and good teachers into these tough places where they’re more likely to have tough assignments. So that at least helps with some of this problem that we’re talking about.
The other thing is, once we use AI more for these kinds of evaluations, which I assume is going to happen, it’s going to get better and better and should have less bias, right? So you wonder if maybe that will help to solve the problem a little bit. Human beings look at a classroom and think, “This isn’t going well.” Maybe the AI will not be so biased.
David Griffith [36:49]
Well, that brings up an interesting question, which is: What do you tell the AI to do with disruptive students? Because you have to tell it how to respond.
Do you tell it to say, “Okay, you need to adjust for the fact that a student was disruptive”? Or do you say, “Don’t do that”? It gets us back to this measurement issue because you have to train an AI model.
I think we’re in trouble. Honestly, I’m generally in favor of value-added. In every context that we’ve talked about it, it has improved the incentives. It has improved the information.
I just think we’re going to have a really hard time with this one because we’re simultaneously saying, this is a serious evaluation. Take this person’s opinion and feedback seriously. This matters. And then we’re also saying, and we’re going to adjust it because really, at the end of the day, we don’t trust them to be objective.
I’m not sure it is possible to deliver those messages simultaneously within the same measure. Maybe that’s the real source of my discomfort or worry here. I don’t know if we can deliver those messages simultaneously within the same measure. I just think it’s challenging.
Mike Petrilli [37:55]
All right, gang. We should end there.
This was great, Brian. Thanks for bringing it on, and thanks for doing the Research Minute twice in a row. We do appreciate it.
Brian Fitzpatrick
Happy to fill in.
Closing
Mike Petrilli [38:06]
All right. But that is all the time we’ve got for this week. So until next week...
David Griffith
I’m David Griffith.
Mike Petrilli
And I’m Mike Petrilli of the Thomas B. Fordham Institute, signing off.
The Education Gadfly Show is a production of the Thomas B. Fordham Institute, located in Washington, D.C. For more information, visit us online at FordhamInstitute.org.
