On this week’s Education Gadfly Show podcast, Joshua Dunn, Executive Director of the Institute of American Civics at the University of Tennessee, joins Mike and David to discuss how public schools will be affected by the end of the Chevron deference—the judicial doctrine in which courts defer to federal agencies’ reasonable interpretations of ambiguous statutes. Then, on the Research Minute, Amber examines a new paper criticizing the famous STAR class size study.
Recommended content:
- “Fishing for rules” —Joshua Dunn, Education Next
- “The case for the supreme court to overturn Chevron Deference” —Wall Street Journal
- “The Chevron deference is desperately needed” —David Martin, Washington Post
- Karun Adusumilli, Francesco Agostinelli, and Emilio Borghesan, “Heterogeneity and endogenous compliance: Implications for scaling class size interventions,” National Bureau of Economic Research (April 2024).
Feedback Welcome: Have ideas for improving our podcast? Send them to Daniel Buck at [email protected].
The following transcript was created using AI software.
Michael Petrilli:
Welcome to the Education Gadfly Show. I'm your host, Mike Petrilli of the Thomas B. Fordham Institute. Today, Joshua Dunn, executive director of the Institute of American Civics at the University of Tennessee joins us to discuss what the end of the Chevron doctrine could mean for schools. Then on the research minute, Amber reports on a new paper criticizing early research on class size reduction. All this on the Education Gadfly Show.
Hello. This is your host, Mike Petrilli of the Thomas B. Fordham Institute here at the Education Gadfly Show and online at fordhaminstitute.org. And now please welcome our special guest for this week, Josh Dunn. Josh, welcome to the show.
Joshua Dunn (00:55):
Great to be with you Mike.
Michael Petrilli (00:56):
Josh is Executive Director of the Institute of American Civics at the Howard h Baker School for Public Policy and Public Affairs at the University of Tennessee. My goodness, we've got to use some abbreviations or something to be able to get that all in.
Joshua Dunn (01:11):
Yeah, it's some acronyms would be helpful.
Michael Petrilli (01:13):
Right? It's exciting this, maybe we'll talk a little bit about this, Josh, I know this is a new role for you, but first let's bring in David Griffith, my co-host.
David Griffith (01:22):
Hey Mike, awlways a pleasure.
Michael Petrilli (01:24):
Yeah, great to see. So Josh, I know you best because we work together at Education Next. Josh writes the Legal Beat column and has for many years where I'm an executive editor. And yeah, you just moved from the University of Colorado, Colorado Springs to the University of Tennessee. My understanding is that this is one of now perhaps a growing number of programs focused on civics at higher education institutions.
Joshua Dunn (01:51):
That's correct. There's several of them around the country now. You have Arizona, Texas, North Carolina, Florida, Mississippi has one. Ohio is, I think five now are starting there. So yeah, there's a growing list of these civics institutes where people are concerned about lack of civic knowledge and decaying civil discourse. And I think that's the mandate for all of us to try to make improvements on both those.
Michael Petrilli (02:16):
Well, that's exciting. Very good. And I heard perhaps a rumor that our friend, Lamar Alexander May have had a role in encouraging you to come to Tennessee.
Joshua Dunn (02:24):
Yes, yes. He was involved and yeah, he did reach out and I see him fairly regularly now here. Yeah, he's a visitor to the Baker School fairly often.
Michael Petrilli (02:36):
Oh, that's great. Alright, we'll give him our best. If you see him, we are here to talk about your latest column, which was on the Chevron deference. The Supreme Court Heard had an oral hearing on this a little while ago. They're going to come out with a decision in June. Sounds wonky. Could have a huge impact though actually on the US Department of Education and all the other federal agencies. So let's talk about it on Ed reform update. Okay, Josh, so the actual case is about fishing. The old doctrine is about Chevron as in the oil company. What the heck does this have to do with education? Give us the brief tutorial. What is the Chevron deference?
Joshua Dunn (03:20):
So Chevron deference, emergent case from 1984 where the Supreme Court said that courts should defer to the judgments of administrative agencies when they were interpreting ambiguous federal statutes. And the idea was that well, generalist judges, they don't have the expertise to go and second guess the decisions of these expert administrators who have all sorts of background knowledge about the particular issues that they have to work out in applying and interpreting these federal statutes. And so there's been some criticism of Chevron deference for, well, really since the beginning, but it's been growing in recent years. And so now there are a couple of cases that came before the court where there's a good chance of the court overturned this Chevron deference. They think that actually undermines the rule of law leads to uncertainty with the law. And yeah, this really affects really any federal agency. It's obviously not just the ones before the court right now, in this case, I think you could make an argument that the Department of Education might be the most affected if the court overturns Chevron.
Michael Petrilli (04:33):
Yeah, I mean, conservatives have argued for a long time that this is a way of basically empowering the administrative state and that these regulatory agencies then end up making law, and that's supposed to be the job of Congress right now. Of course, Congress has not exactly been showing itself to be a highly effective institution recently. And so the notion that Congress is going to have the capacity to get into the weeds on some of these issues, it would need to, in a complicated policy issue, the environment or education, anything else, or be able to make quick changes if they've made a decision that's not working out. Whereas an administrative agency, at least maybe through rulemaking and the like, could be quicker to make adjustments and have the expertise to go deeper. And then there's a role of judges, like you said. So is this about giving power back to Congress saying, no, I'm sorry, department of Education or any other department, you can't make the laws? Or is it about giving power back to the judges who get to decide what the law means?
Joshua Dunn (05:39):
Exactly. So that's one of the reasons why at least conservatives initially supported Chevron was because they thought it built in political accountability. Because the idea was that presidents could then come in and influence agency rulemaking in these areas. And so you actually would have some control of the process. But I think over time, what we've seen is that often, as you mentioned, the agencies or the concern has been the agencies then just kind of go off on their own and implement their own rules. And so you might not even get a whole lot of political oversight from the executive branch precisely because these agencies are often insulated from direct control by presidents. I mean, presidents can exercise some, but you have career officials in these agencies and they can kind of outlast them and work through them. And so there's some dissatisfaction because of that. So then the question is, yes, what's going to happen here? Is this going to transfer authority to the courts or will courts maybe when you do have disputes over whether or not a statute is ambiguous, do they just kick it back to Congress and say, look, you aren't even entitled to regulate in this area. Congress has to, Congress has to be clear if it wants you to do something in this area,
Michael Petrilli (06:53):
Let's take an example that is fresh from the headlines, right? We're just like Law and Order here and it's Title ix, the Department of Education just put out rulemaking. So proposed regulations around Title ix. We've seen this go back and forth under different administrations, right? And one of the big questions is when Title IX says, I think the words are something like you shall not discriminate on the basis of sex, or if you do, you can lose federal funding. Does that mean that by discriminating on the basis of sex, does that protect kids that are L-G-B-T-Q or is that a different set of issues and is that going too far? Is the Department of Education stretching the law when it's interpreting it that way or not?
Joshua Dunn (07:40):
Yeah. So obviously the Department of Education now in the Office for Civil Rights, they're arguing that the Supreme Court's decision in Bostock versus Clayton County gives them some authority to do this. That involved Title seven. But then there's still this issue about whether or not that applies to Title ix. Does the meaning of sex have the same meaning in those two? The court didn't officially decide that in Bostock versus Clayton County. So that's what the Department of Education is saying. Previously though, that didn't get in the way of them back in the Obama administration, they kind jumped headlong into this and said that it does even before Bostock versus Clayton County. And they just issued a dear colleague letter this time around with a Title IX regulations. They at least went through a kind of an abbreviated rulemaking process where they opened up notice and comment.
(08:29):
Procedures were not as long as you would normally have, but I think it was a month or so and they still got what, around 250,000 public comments on the issue. And I actually think that that was an indication that the Department of Education recognized that some of their past behavior might be coming under suspicion by the courts where they just issue these dear colleague letters and run with it as if they'd gone through a regular rulemaking process. But so they did kind of go through this rulemaking process this time, and now as you mentioned, they've just now issued new regulations that should go into effect on August 1st.
Michael Petrilli (09:06):
Right. Alright. But this is, again, if the court comes out and said, okay, we're changing the Chevron deference, we are now going to go back to a time when if it's ambiguous, the judges are going to have to decide it or they're going to have to punt it back to Congress. This could be the kind of thing on Title IX or maybe on some of the discipline policies that we've seen where there's a sense that the regulatory agencies are stretching the meaning of the law to fit their policy preferences, that you may have courts snapping back more aggressively than they did in the past.
Joshua Dunn (09:41):
I think there's no doubt about that. And I think that really, given the current composition of the Supreme Court, it's unlikely to see the Supreme Court say, okay, judges go in and fill in the blanks. Now, since it's not appropriate for agencies to do this, you can now substitute your own judgment about what's best. I don't think we're going to see that. I think it is going to be much more likely for the Supreme Court to say, yes, this is ambiguous, and you have to go to Congress for authorization to engage in rulemaking in this area. So I think that's what's going to happen. Then the complaint will be, well, Congress is kind of busy or they can't get their act together. And I think for the Supreme Court though, given the people who are on it now, they're going to say, well, that's Congress's problem. Yeah, it's not our job to make up for their deficiencies as an institution. So
Michael Petrilli (10:31):
Again, to the Title IX point, you'd say, well, so if we believe that the law should protect L-G-B-T-Q kids, for example in schools, that is the job of Congress to pass a law that makes it clear that we're going to now change the language in Title ix.
Joshua Dunn (10:46):
I think so. Although again, it depends on whether or not they're willing to apply this reasoning from that Bostock decision to Title ix. I think that's unlikely though, because you do have this issue of athletics that's looming out there. Even though the Biden administration, they decided not to address athletics, and particularly with transgender students because that was going to be political dynamite going into November. And so with its latest round of proposed regulations, they punted on it. But I think the Supreme Court is smart enough to know that if they actually did apply the reasoning of Bostock to Title ix, that immediately the Department of Education would go in and start issuing those kinds of regulations and saying that schools actually are mandated to allow students to participate on the team that matches their gender identity.
Michael Petrilli (11:38):
So David, how are you feeling about this? We're going to empower re-empower the first branch, right?
David Griffith (11:44):
Yeah, I'm not wild about that. Basically, given the status of the first branch right now, maybe if we could wind back, turn the clock back a few decades, I'd feel differently. But I've never even experienced a functional legislature. I mostly have questions, to be honest. I'm struggling to wrap my heads around the implications. And it all starts with this word ambiguous in my lifetime. I've seen almost every clause of the Constitution debated. It strikes me that they're all kind of ambiguous. Your average law maybe is a little more clearly written, but I assume that they don't come with this label that says, I am ambiguous on them. So doesn't it all just kind of boil down to judge's discretion when the
Joshua Dunn (12:27):
Well some of that. But there can be, you can look at statutes and see where there's a clear authorization to engage in rulemaking in a particular area, and then perhaps others where it's not. And you've seen the court do that with what they've called this major questions doctrine. They say, well, look, this statute, this involves such an enormous area. If Congress actually intended for you to have this authority, they would've been explicit about it. And I'm sympathetic to the idea that, yeah, Congress hasn't been terribly functional, but perhaps part of the reason they haven't been functional is that they've been able to pass off their responsibilities to agencies in courts. And so maybe this will be the thing that forces them to actually do their job.
Michael Petrilli (13:13):
And look, we saw this week with Ukraine and the Israel, sometimes when their back is up against the wall, they do act perhaps much later than they should. But that is one way to look at it. Hey, that is all the time we've got on this discussion. I'm sure we could go much longer. And Josh, it's always a pleasure. I find this legal stuff fascinating, and I appreciate you making it clear for us.
Joshua Dunn (13:39):
Oh, thanks. I hope it clarified some. We can be certain though. There'll be more litigation to help clarify it for us.
Michael Petrilli (13:48):
Alright, well check out Josh's column on this at Education Next. We will link to it in the show notes. Again, Josh Dunn, executive director at the Institute for American Civics at the University of Tennessee. Thanks so much for joining us, Josh.
Joshua Dunn (14:01):
Thanks, Mike.
Michael Petrilli (14:03):
Alright, now it's time for everyone's favorite Amber's Research Minute. Amber, welcome back to the show.
Amber Northern (14:15):
Thank you, Mike
Michael Petrilli (14:16):
Just had a great conversation there about legal stuff. And I have to say, in an alternative universe, a different timeline for my life, I think I would've gone to law school.
Amber Northern (14:26):
Really? Wow. I've never heard you say that.
Michael Petrilli (14:28):
I really find this stuff fascinating, especially the constitutional law.
David Griffith (14:35):
You're an armchair economist, Oh, excuse me, lawyer, arm chair, lawyer. You're also an arm unsure economist though, Mike.
Michael Petrilli (14:41):
Yes. I do also wish I'd been an economist. I know. Or maybe both. The problem with the law school thing is I don't think you are allowed to go to law school and say, okay, I want to go to law school if someday I get to either be on or argue in front of the Supreme Court.
Amber Northern (14:56):
Wow, that's a high goal.
Michael Petrilli (14:59):
I guess that's not how it works. Right?
Amber Northern (15:02):
Don't sell yourself short, Mike.
Michael Petrilli (15:03):
Yeah. Alright. And maybe it's not too late.
Amber Northern (15:05):
We have these lawyers, by the way, that come and adjudicate these teacher cases on the state board. And one we affectionately call Perry Mason because he acts like he is in front of the Supreme Court, but it's just us. But anyway, he has those same ambitions, Mike, I do believe.
Michael Petrilli (15:24):
Well, what you got for us this week?
Amber Northern (15:25):
Oh man, I'm going to have to get technical on you, at least try not to. But man, there was this NB ER study that came out recently. Again, super technical, but it calls into question probably one of the most well-known studies in education ever. Which, what's that?
Michael Petrilli (15:44):
The STAR class size study
Amber Northern (15:46):
The STAR Study on star class size.
David Griffith (15:49):
Oh no.
Amber Northern (15:50):
It had serious methodological concerns. Again, I had read some of this but not seen it so well articulated. So just a reminder, if you're under a Rock Tennessee star, use an experimental design to assess the impact of class size reduction interventions. This current study breaks down the problems with that original research. It's conducted by three economists at UPEN and Princeton, and they provided in this NBER paper a better way to analyze the results from STAR reliably to gain new and useful information about class size interventions. So hey, I thought that was a good way to frame it. Star again, conducted between 1985 and 1989, randomized grade kindergarten students at participating public schools and do one of three class types, small, regular, and regular with a teacher's aid evaluations of STAR showed significantly higher scores for students attending the small class type leading to states. If you recall, Tennessee and Colorado, California adopted these policies, reducing class sizes didn't live up to the hype.
(17:05):
It's partly because California especially hired a lot of unqualified teachers to meet the new demands. What else could they do? So these economists go into great detail about the issues with the original methodology. I'm going to try to simplify it, which is not easy. This thing is long. I think it boiled down to this two stage lease squares model was poorly aligned with the experimental design because it treated the class types as the same as class size and class type and class size are not the same things because the school principals had discretion in choosing the target class sizes for each type. So there were substantial differences in compliance with different schools focusing on both heterogeneous sizes of classes, meaning that the dose of the intervention differed and hetero heterogeneous class size reductions. So the amount of the reduction also differed between the treatment and the control arms.
(18:10):
So their model here they go identified a weighted average of school specific effects, but those weights depended on each school's compliance with the demands of the experimental design, which, and they didn't follow that, which I'm going to tell you about that in a minute. So all that was endogenous with the class size effectiveness model that they were using because certain types of schools were more or less compliant with the experimental design, which first we're hearing about that, or at least first I'm hearing about it. So what they did instead was they dug into the details of the program design and implementation at each school, and they found that schools actually under complied with the intended implementation of STAR by having class size reductions that were smaller. So the experimental design called for 13 to 17 student range for the treatment and 22 to 25 kids for the control.
(19:14):
So the average reduction in class size between the control and treatment was seven students. But again, it varied a lot across the schools. They had between zero and 12. So okay, that's a pretty big range and doesn't exactly meet the qualifications of the design. And then strangely enough, the schools that had the largest impact, the effects, they could be positive or negative, they showed the lowest compliance with reducing class sizes. So they didn't have the bigger reductions. Alright. Their analysis, I hope we can get through this, they did something called group random effects approach. So they're trying to fix all this stuff. So they're allowing, this allows 'em to model the class size levels for both the treatment and the control class types because again, there was a lot of discretion and they found that nearly all of the gains from reducing class size in the tar star study were driven by 29% of the schools in the sample.
(20:19):
In fact, if the 29% of those highly sensitive schools had been omitted from the experiment, their model would've failed to detect any causal effect of class size on test scores. So then they dig into this set of schools, their 23. So these 23 schools, which they call group two, their mean test score increase for those kids caused by reducing class size by one student equated to an improvement of 0.09 standard deviation. But when you average all across the groups and students, that same figure is 0.02 standard deviation. So it gets, it's a lot higher when you're in this higher impact group. A little more about group two, they had larger under compliance, which means they didn't, in my mind, they didn't reduce class sizes as much as attended. So they had bigger classes in general. And the students in group two also attended school between four and five days fewer.
(21:27):
They had not as long as the school year compared to the other two groups. And then they close, they've got this long discussion on, we really need to think about how we scale these interventions that, and we really need to look into the effects by subgroups of schools and think about targeting our resources and policies where they're most likely to have the largest effects. Because monitoring implementation is a hard thing to do across so many schools and so on and so forth. So another scaling problem, but also some important information I think on this study.
Michael Petrilli (22:08):
Wow. Isn't it the case that there were other studies that used the same data to answer other questions, not just class size. I feel like maybe about teacher effectiveness or other things and makes me just wonder about how many other findings now do we need to question whether they might not be in fact true.
Amber Northern (22:30):
Yeah. And I'm sure they probably got into that in this very long background section.
Michael Petrilli (22:38):
Well, and look, and this is not about ancient history here. I mean, we know New York City is going through this right now because unfortunately they were able to get this class size policy through recently that's going to force them to go through and lower class sizes in every school, including in the affluent schools. And a lot of us are worried that we're going to see, again, that's going to lead teachers to transfer from the high poverty schools to the more affluent ones. The high poverty schools are going to have lower quality teachers, and then we're not going to get the results that we want. So I thought you were going to just tell us that they were able to tease out sort of what happens when you scale up. And of course there's variation, but it seems like it's even more fundamental than that, that the actual study had some serious problems and that we need to be much more realistic about what class size reduction can achieve based on that, even if scaled up effectively, even without all of the problems with say, lower quality teachers.
Amber Northern (23:38):
And other folks have pointed out, it was just one in five schools that agreed to participate. And so you would think if you agreed to participate, then you would play by the rules. But there was just a lot of flexibility in how these interventions were carried out.
David Griffith (23:59):
Yeah, it's troubling. I mean, it's not that troubling if you're skeptical of class size reductions, but it is troubling. I guess the fundamental problem, as I understood it, was that the schools that implemented it most diligently saw essentially no effect. And the schools that implemented it sort of halfheartedly saw bigger effects. And that was sort of inexplicable to the researchers.
Amber Northern (24:27):
That's right. Yeah. I mean, if they had larger under compliance, I'd have think that through, right? That means they had the bigger class sizes and they had the bigger impact. Right. Am I thinking about that right, David?
Michael Petrilli (24:42):
Or the difference again, the difference. They may be all, it could have been that they had small class sizes throughout right? Or big class sizes throughout. It was just that the differences weren't that large.
David Griffith (24:53):
Yeah. As I said, what you were saying, it was that the differences were between the two were smaller in the places where there were bigger effects. Was that what you were saying?
Amber Northern (25:02):
The differences between the Right. The group two had the bigger class sizes because they under complied more. And the other schools that I guess didn't have as high level of under compliance didn't have the effects. It can kind of make your head hurt when you think about it, but I think I'm thinking about it, right?
Michael Petrilli (25:26):
It's the kind of logic puzzle you might find on the lsat. So again, back to law school idea,
Amber Northern (25:33):
There's some problems with the study people, and we probably need not put it on such a pedestal. I think that's my takeaway and that, hey, it gets down to a very plain fact about how hard it is to scale anything, even if it was a good intervention.
Michael Petrilli (25:53):
Yep. No, I think that's right. I think that as research has improved, it does mean that it's a higher bar for some of those old studies back in the day. Alright, well we'll need to let that be that, because that is all the time we've got for this week. So until next week,
Amber Northern (26:13):
I'm Amber Northern.
Michael Petrilli (26:14):
And I'm Mike Petrilli at the Thomas B. Fordham Institute signing off.
