The Education Gadfly Weekly

I’m a decades-long supporter of school choice in nearly all its forms and likely to remain that way so long as traditional, district-operated public schools ill-serve so many kids, produce such widespread mediocrity by way of achievement, give parents so little say in so many matters, and cater to the interests and priorities of the adults in the system more than the needs and interests of children, taxpayers, and the general public.

Whew! Yes, and I mean it, and few will be surprised by that paragraph. That said, however, I’m getting seriously unnerved by how the country is coming apart, by how many people are putting pure self-interest ahead of anything smacking of the public interest, by mounting intolerance of those who are different or who disagree, and by diminishing confidence in the shared values, institutions, principles, and traditions that have held us together as a nation, most of the time anyway, for the better part of three centuries.

Which forces me to wonder whether putting all our education hopes in markets, self-interest, competition, and “invisible hands” just might be contributing to—at least moving in tandem with—other fissiparous forces that are weakening the valuable shared assets that we inherited from earlier generations. Recent surveys certainly suggest that mounting public support for school choice is coinciding with diminishing confidence in shared institutions and public values of all kinds, including patriotism itself.

Figure 1. Percentage of respondents to WSJ/NORC poll who said these values are “very important” to them, 1998–2023

It’s not just markets and conservatives, of course, that may be pushing—I attribute no intent here—against those shared institutions and values. Identity politics and the overreach of DEI and innumerable forms of intolerance and cancellation on the left are, to me, a lot more damaging to that inheritance. So, too, are many of the core tenets and practices of progressive education, such as children learning—on their own—that which interests them rather than participating in shared lessons, readings, and interactions.

But maybe we who yearn for more and better schooling options for America’s kids should try to do our part. Maybe we should pause for three seconds and ask whether there are ways of furthering choice while also helping to sustain, even strengthen, the shared inheritance.

We’ve known—I’ve surely known—for years now that pure market forces in K–12 (and higher) education do not reliably yield more effective schools and better-educated children. Sorry, Milton F and Corey D and a host of other living colleagues. Too many things go awry in that marketplace, from parents who make bad (if understandable) choices to greedy school operators who don’t care about outcomes, not to mention kids who lack competent adult guides. So, for years, I’ve urged a “regulated marketplace” that requires prospective school operators to demonstrate their bona fides and holds all schools to account for their educational results, thereby constraining the market to choices that (one would hope) include a sufficient array of quality options. Sadly, though, we know that the constrained market often fails to yield enough good schools accessible to enough needy kids, so we continue to struggle with the resulting tensions and tradeoffs. (Who else remembers the films “Waiting for Superman” and “The Lottery”?) We know, too, that the established adult interests of public education adore constraining the marketplace and do their utmost to limit options all over the place, even while themselves failing to deliver a quality product to millions of kids.

Which says to me that regulation of the school-choice market is a mixed blessing—a necessary evil, if you will—but also one that in no way deals with the problem of our weakened inheritance and our coming apart.

It also has to be noted—this really hurts—that we’re seeing mounting evidence that increases in public-sector school choices, charters especially, are bad for Catholic parochial schools and perhaps for other “traditional” private schools that, on the whole, have striven to maintain some of that inheritance via faith, values, morals, and example. Will the spread of ESAs send more kids back into those private schools? Or will more choice result in more coming apart?

Yes, I want it both ways. I want a plethora of quality school options for families, but I also want our “education system” in its variegated forms to strengthen rather than weaken our shared inheritance and pull us more together than apart.

Putting some reasonable limits on fissiparous alternatives to that system could help a bit, such as booting more kids off social media for larger portions of their lives, maybe nudging them back to Little League, Cub Scouts, and story hour at the library.

Back in school, there could be hope in the realms of curriculum and perhaps graduation requirements, at least if these were more widely agreed upon and shared across the various K–12 delivery arrangements. That’s more or less how most other modern industrial democracies manage to balance school choice on the one hand with national unity on the other: The schools may be independently operated and diverse in various ways, but they generally adhere to a common core curriculum.

I understand that curricular diversity is a core tenet of U.S.-style school choice, and I’m well aware of the fracas over Common Core and such, but I’ve also been pointing out the latent (if limited) consensus around most of the country in realms such as civics and history curricula. If we worked at it, we could find something similar in the (arguably) less contentious realms of math, science, and even English. Which is to say, it should be possible to develop a framework of shared curricula spanning big chunks of the main K–12 subjects, curricula that would be acceptable to the vast majority of Americans and could be taught in the vast majority of schools of all sorts. Schools would naturally add and embellish, and in time, perhaps two-thirds of the curriculum might be “common” across almost all elementary and middle schools, maybe half in high school. If it pains you to think of commonality across state lines, we’d get somewhere by pulling it off within them.

That won’t unite a divided land. But it might slow the coming apart.

What else? A regulated marketplace and partially-shared curriculum can’t be the whole story. I’m not so bold as to forecast a truce in the culture wars. But what else can we devise that might better balance our hunger for school choice and diversity with America’s need to preserve the best of its inheritance?

In 2022, seventeen states mandated that schools hold back students who aren’t meeting reading standards by the end of third grade, and eight others allowed it. Since then, however, Michigan has repealed their requirement—joining Nevada as the only state to do so—and others are considering legislation to gut their policies. Such efforts are driven by worries about holding back growing numbers of students harmed by pandemic learning loss. But these repeals wouldn’t help struggling students catch back up. On the contrary, they’d only sweep the problem under the rug.

In 2012, Ohio lawmakers enacted the Third Grade Reading Guarantee, a significant early literacy reform package. Under the initiative, schools must administer diagnostic reading assessments to students in grades K–3. If students are identified as off track, schools must notify their parents and create improvement plans. The policy also requires schools to retain students who, based on state assessments or state-approved alternative exams, aren’t meeting reading standards by the end of third grade. To improve their reading skills, schools must provide such students with intensive interventions such as summer reading programs or tutoring.

In the ten years since the Guarantee became law, there have been multiple attempts to water down or eliminate the retention provision. The latest effort, House Bill 117, aims to repeal the requirement by prohibiting districts from holding students back based on their scores on state assessments. Backers of the bill argue that the state should be emphasizing literacy instead of standardized tests, and claim that retention has a negative impact on students.

There are a few issues with these arguments. First, state leaders are emphasizing literacy. On top of the policy’s existing focus on improvement efforts from kindergarten through the end of third grade, Governor DeWine recently unveiled a plan to significantly strengthen the state’s early literacy efforts, and recommended investing $174 million in additional state spending to do so. The governor’s plan would require schools to use curriculum and materials that are aligned with the science of reading, cover the cost of professional development for teachers, and provide literacy coaches to schools and districts with the lowest reading proficiency scores.

Second, implying that standardized tests and early literacy are mutually exclusive is wrong. Assessments are a crucial part of any early literacy effort. Without them, it’s impossible to gauge how well students can read. Local tests—including those that are teacher-designed and -administered—are important, as they offer immediate feedback to help teachers plan lessons and identify which students need extra help. But state assessments are equally important because they serve a different purpose. Unlike classroom assessments, state tests track growth (or lack thereof) over time. They’re comparable across districts and regions, which makes it possible to identify and measure achievement gaps. They also objectively measure student performance against state standards—benchmarks that are designed to gauge whether students are reading on grade level and will be ready to succeed in the upper grades.

Third, critics are ignoring a trove of data that refutes their claims that retention hurts students. In fact, research conducted in several states indicates that retention, in combination with intensive supports, has a positive impact on students’ short-term and long-term success. Let’s take a look at a few examples from around the nation.

Florida

It’s imperative to talk about Florida when talking about early literacy. Like Ohio, the Sunshine State requires schools to identify K–3 students who exhibit a “substantial deficiency” in reading, notify their parents, and provide intensive interventions like a minimum of ninety minutes of daily reading instruction and summer reading camps. Florida has also retained students who score below a certain threshold (with some exceptions) on the state reading test since 2002. In the two decades since, research studies have found positive impacts from retention. Consider the following:

• A 2015 study, completed by Guido Schwerdt, Martin West, and Marcus Winters, and updated in 2017, found that retention in third grade increased students’ high school GPAs, led them to take fewer remedial courses, reduced retention probabilities in future years, and had no negative impact on graduation.

• A 2018 report on the costs and benefits of test-based promotion found that the threat of retention led to “statistically significant and substantial” improvements in math and reading performance within third grade prior to retention. It also led to significant and substantial gains in eighth grade math and reading, and increased the probability that students would earn a diploma.

• A 2019 study focusing on English language learners who were held back in third grade found significant academic gains for those who were retained, as well as reductions in later remedial course-taking.

Mississippi

Mississippi passed its early literacy policy in 2013. Much like Ohio’s, it requires students who don’t reach a minimum score threshold on state tests to be retained and receive additional support and interventions. In the decade since, the Magnolia State’s impressive academic progress has been dubbed a “learning miracle.” Improvement on the National Assessment of Educational Progress (NAEP) has been particularly noteworthy; from 2011 to 2022, Mississippi ranked first among states in fourth grade reading gains.

According to a working paper from the Wheelock Educational Policy Center at Boston University, retention played a big part in these overall improvements. The results show that students who were held back via the state’s retention policy scored more than 1 standard deviation higher relative to their barely promoted peers by the end of sixth grade. In other words, students who were retained scored, on average, around the 62nd percentile in English language arts when they were in sixth grade. Meanwhile, comparable students who weren’t held back scored, on average, in the 20th percentile.

A recently published analysis identifies two reasons why the state’s “purposeful retention” policy has been crucial to its success. First, it’s much more than simply repeating a grade. By law, students receive plenty of extra support and interventions, including a minimum of ninety minutes in reading instruction based on the science of reading. Second, it acts as “an accountability tool for the system’s adults, designed to support them in changing their practice.” Mississippi didn’t just retain struggling readers. It also heavily invested in the adults who are responsible for helping them by adopting high-quality curricula and materials, offering professional development in the science of reading, providing literacy coaches, and changing teacher preparation programs. In short, retention played a crucial role in “aligning the system’s adults—teachers, parents, and administrators—around meeting the needs of students.”

Indiana

Ohio’s neighboring state of Indiana has had a retention policy since the 2011–12 school year. Like Mississippi, struggling receive additional support that’s based on the science of reading. And like Mississippi and Florida, Indiana offers evidence that retention can have a positive impact. A report published by the Annenberg Institute at Brown University used a regression discontinuity design to compare retained students to those who barely passed the state’s promotional threshold. They found that, in the fourth grade, retained students earned much higher state test scores than their peers who weren’t held back in both reading and math. These gains persisted through seventh grade, though the magnitude faded somewhat over time. There were no significant impacts from retention on disciplinary or attendance outcomes.

—

After seeing all these data, one might wonder why Ohio—which has had a retention policy for over a decade—hasn’t seen similar progress. Although there were noticeable improvements in state assessment data prior to the pandemic, 40 percent of Ohio third graders currently aren’t proficient in reading. NAEP scores have also been largely flat.

The lack of progress is likely due to Ohio’s long history of backing off on policies that set high standards for students and schools, including in early literacy. Annual attempts to water down and eliminate the Third Grade Reading Guarantee are influencing implementation efforts, which in turn impacts student outcomes. It’s also important to recognize that Ohio’s current early literacy policy isn’t as comprehensive as those in other states. Mississippi, for example, didn’t just identify struggling readers and intervene. State leaders went all in on the science of reading and flooded teachers and schools with support. Ohio, on the other hand, hasn’t championed the science of reading or made any effort to ensure that schools are following the research.

Governor DeWine’s recently proposed early literacy plan would cover many of the areas that the Guarantee did not address. Its defining characteristics—adopting high-quality curricula and offering teacher professional development aligned with the science of reading, as well as providing literacy coaches—were instrumental to Mississippi’s success. But so, too, was retention. If Ohio lawmakers go through with their latest attempt to eliminate the Guarantee’s retention provision, they won’t just be ignoring research and positive results from other states. They’ll be undercutting Ohio’s promising push on early literacy. For the sake of Ohio’s students, let’s hope they change their minds.

I read with interest Daniel Buck’s recent piece, “The agonizing individualism of progressive education.” In his view, progressive schools fail to uphold communitarian values by overemphasizing individualism. He worries that they lack direct instruction, rich group discussions, and clear behavioral expectations. On each of these charges, I think he is wrong.

I have a unique vantage point from which to make this claim. I am a public school graduate, a progressive-high-school alumnus, and current board member of one of the best-known progressive schools in the country. I have spent my career working to increase the number of high-performing public schools serving kids from low-income families in cities around the country. I have a deep appreciation for both urban charter schools and progressive education leaders like John Dewey and my own college thesis advisor, the late Theodore Sizer.

In his critique of progressive education, Mr. Buck paints a picture that bears no resemblance to progressive schools I’ve attended or visited.

He contends that progressive schools “do not impose a curriculum or knowledge worth knowing. Rather, let [students] discover their way into whatever interests them most.... Never do they discuss together, read out loud together, practice problems together, or sit silently struggling through the same task together.... Never mind an expectation that students master...any common body of knowledge to bind us closer together.”

This was not my experience. I attended my neighborhood public elementary school, but my family moved when my stepfather lost his job. Because we relocated to southern Vermont, my sister and I ended up going to The Putney School on scholarship. Putney is one of the nation’s oldest progressive independent high schools.

Was Putney an educational lord of the flies with students following their individual interests without guidance from adults or adherence to group norms? Nope. We had a core curriculum in math, science, English, history, and the arts. We had rich group discussions of canonical books. I’ll never forget lugging around the massive, hardcover volumes containing the complete works of William Shakespeare from which we read and discussed one play every three days in my senior English class.

We had direct instruction in math and science, Socratic discussions in history, and a student-led, faculty-advised Standards Committee that adjudicated student discipline cases on behalf of the community.

But we also had project week instead of final exams. During project week, students had to design and execute capstone projects in both the academics and the arts. Each project took a minimum of twenty-four hours to complete. One year, I wrote and performed a musical score for the winter theater production of As You Like It. One of my fellow students retrofitted a farm tractor to run on bio-diesel. (That student went on to found and sell a successful farming technology company fifteen years later.)

Granted, there is immense privilege in attending an independent school like Putney. But many of the progressive public schools that I’ve visited—whether they be Expeditionary Learning, Deeper Learning, Montessori, or members of the Coalition of Essential Schools—share common commitments to the instructional, pedagogical, and communitarian values that Mr. Buck practices in his own teaching.

I agree that “our public schools are one of the few unifying institutions that we have left.” But I was left flummoxed by the next sentence: “If we allow progressive educational principles to continue to individualize and atomize the classroom, we shouldn’t be surprised if our culture and political climate follow suit.”

Rarely have I had such whiplash when reading two adjacent sentences. Progressive educational principles are not driving the culture wars. Legitimate (and manufactured) flashpoints over Covid policies, partisanship, race, gender ideology, and guns are the real drivers. If you’re looking for threats to public schools, don’t point your finger at John Dewey; look at the polemicists in the cynical political class who use fear to splinter us into warring tribes.

Mr. Buck ends his piece with a description of a wonderful closing activity that he does in his classroom. It is an example of the kind of classroom ritual I experienced many times in my progressive high school, and have seen in many visits to progressive schools over the years. While I think Mr. Buck is deploying and assaulting straw men in his critique of progressive schools, I think he’d love teaching at one.

Despite the expansion of computer-based testing in schools over the last decade—and ongoing concerns about negative impacts on student outcomes—research on the topic is scarcer than one might expect. One of the more recent rigorous studies (2018) covered Massachusetts’s administration the PARCC exam and found that students taking the test online tended to score lower.

Is that still the case? A newer study from a trio of economists at the University of South Carolina examines the rollout of computer-based testing (CBT) in the Palmetto State starting in 2015. While some of the methods are similar to the Massachusetts study, the researchers here use a better poverty measure (direct certification versus school-lunch status) and are able to track effects for three years versus two.

In South Carolina, elementary and middle school students began taking social studies and science tests online in 2015—ahead of a legislative mandate requiring them in 2016—while online test administration in math and English language arts (ELA) began in 2016. However, not all schools were able to meet the deadline, and many applied for waivers due to insufficient security, bandwidth issues, and so on. CBT was nearly universal in the state as of 2018 but required several years of phased rollouts, which were leveraged in the study’s design.

The research team used data including the share of online testing for each grade level for each school for each of four tests (science, ELA, math, and social studies), administrative school-level data, and information on the level of technology per school, such as broadband access and devices in use per teacher. In the first analysis, they observe pre- and post-transition patterns for all subjects but have the most data for science and social studies tests, which equates to four periods prior to the modality change and three after. Their second analysis controls for various school and student characteristics and allows them to examine impacts by subgroup.

Their key finding shows negative effects from CBT on average scores, ranging from a 0.02 standard deviation (SD) reduction in math to a 0.09 reduction in ELA in the first year of CBT adoption. Perhaps more important, the negative effects do not fade away in either of the subsequent years of the study. When looking at grade levels, however, the researchers find a negative effect for ELA and science in both elementary and middle grades, while math and social studies is negative only at the middle school level. Additional analyses continue to show that the impact on ELA is the most negative, and that middle school students in general experience more significant decreases in test scores than do elementary kids.

Next they look at differences by poverty status—defined as students in poor households receiving food stamps or other monetary benefits. Results reveal that CBT is apt to lower test scores for all subjects for students in poor households, especially in ELA. Among non-poor households, CBT has no significant effects on social studies and math, but negative impacts on science and ELA, although these effects are much smaller than the negative impacts on students in poor homes.

Finally, they find smaller negative effects of CBT in schools where technology is more readily available (measured by tech devices in use per teacher). Poor students still show negative effects from CBT even in schools with better technology, but those effects are more muted.

Obviously, the persistence of negative effects in this study is worrying, as one would hope the adoption of any new technology or procedure would get smoother over time. Analysts speculate that, because many students switched from elementary to middle school buildings between test administrations, they could easily have experienced test-mode whiplash, going from pencil to computer (or vice versa) more than once.

Still, other studies say that it is not the mode of test administration per se that’s the problem. It has to do with things like computer practice. To wit, it is more difficult for some young children to type than to handwrite their answers, especially if they have limited exposure to computer technology.

No matter what’s at play, the mechanics of staggered rollouts and pencil-and-paper alternatives to CBT in 2018 read like nostalgia in 2023. As many as forty-eight states now have computer-based tests (with some still also offering paper, too). Moreover, the pandemic forced students and teachers to rely on electronic teaching and learning. We know that this forced experiment led to dismal academic results and misery. But did some determined youngsters forge through the adversity and pick up more familiarity and facility with computers that perhaps stood them in good stead on state exams? One can hope.

SOURCE: John Gordanier, Orgul Ozturk, and Crystal Zhan, “Pencils Down? Computerized Testing and Student Achievement,” Education Finance and Policy (October 2022).

As school accountability systems reset following pandemic disruptions, an opportunity arises to improve their accuracy and make sure the intended responses to data resulting from them are properly tuned. A new study from the U.S. Department of Education’s Institute of Education Sciences looks at a way in which academic proficiency rate calculations for small schools and small subgroups of students might be sharpened and improved.

School-level data for each year from 2015–16 through 2018–19 come from the Pennsylvania Department of Education (PDE), and comprise all of the elementary, middle, and high schools in the state that were included in ESSA accountability calculations in those years. Which means we’re looking at schools examined for possible inclusion in Targeted Support and Improvement (TSI) and Additional Targeted Support and Improvement (ATSI) under state-adopted subgroup performance criteria. Many schools, however, had subgroup sizes below twenty. Those subgroups had traditionally been excluded from the TSI and ATSI calculations due to concerns over unreliability (termed “instability” by researchers) of data derived from such a small sample. The researchers theorized that there was a way to improve the proficiency rate calculation of smaller subgroups in these schools using Bayesian hierarchical modeling, a statistical method used to make inferences about a population based on a number of individual observations. (Lots of information on the model can be found here and here. It is used in fields as diverse as astronomy and marketing.) The eight subgroups examined in each school were Asian students, Black students, Hispanic students, White students, multiracial students, economically disadvantaged students, students with disabilities, and English learners. (Despite targeting this research on improving the measurement of very small subgroups, Native American/Alaska Native and Hawaiian/Pacific Islander student groups were simply too tiny to be included.)

Analysts looked at the percentage of students scoring at or above the state’s threshold for academic proficiency in each school-subgroup combination and the number of tested students in English language arts and math for each school-subgroup combination in each year, looking to stabilize the proficiency rate data for each. They found that their stabilized proficiency rates showed more consistent variation across subgroup sizes, indicating that they are more reliable than unstabilized rates. Thus, use of Bayesian stabilization could allow for inclusion of smaller subgroup sizes with similar statistical reliability as larger samples.

They then reran PDE’s data using a minimum subgroup size of ten to produce a new list of schools with low-performing student subgroups as determined by state criteria. The new calculations moved one subgroup above the proficiency cutoff for ATSI designation in nine out of the 193 schools originally identified. Specifically, the subgroup of White students in six schools and economically disadvantaged students in three other schools all were raised above ATSI designation. Their conclusion: The schools were misidentified and funding to support students in those subgroups in those schools was not necessary, diverting money from schools and students who needed it more. Small potatoes in one state, perhaps, but potentially much more important writ large.

The researchers note two limitations of their study. The first is a lack of student-level data, which they surmise could allow for even sharper stabilization of proficiency rates. Second is that most of the subgroups whose status changed were very close to the ATSI proficiency cutoff. PDE has discretion on borderline cases like these, and while the researchers set inflexible cutoffs, they assume that state officials would have simply kept the classifications as originally calculated, making their work moot in the real world.

Despite these limitations, the Pennsylvania Department of Education was encouraged by the results of this experiment and partnered with the researchers to incorporate data stabilization into the state’s actual ATSI calculations for 2022. No outcomes have been announced yet, but the researchers believe that not only will this correct for measurement errors in the past, but will also help smooth out data collection gaps experienced in 2020 and 2021 due to pandemic-induced testing disruptions. Surely many eyes will be trained on the Keystone State whenever results are released.

SOURCE: Lauren Forrow, Jennifer Starling, and Brian Gill, “Stabilizing Subgroup Proficiency Results to Improve the Identification of Low-Performing Schools,” Institute of Education Sciences, Regional Educational Laboratory Mid-Atlantic (February 2023).

On this week’s Education Gadfly Show podcast, Carole Basile and Brent Maddin from Arizona State University join Mike Petrilli and David Griffith to discuss how a new team-based staffing model could be good for students and teachers. Then, on the Research Minute, Adam Tyner discusses how selective high school admissions could change if schools looked at more than just test scores.

Recommended content:

• Carole and Brent’s team-based teaching project at Arizona State University: Next Education Workforce
• The Next Education Workforce: Team-based staffing models can make schools work better for both learners and educators —Carole Basile and Brent Maddin
• “To improve student outcomes, ask teachers to do fewer things better” —Robert Pondiscio and Jessica Schurz
• “In one giant classroom, four teachers manage 135 kids—and love it” —Hechinger Report
• The study that Adam reviewed on the Research Minute: Marco Pariguana and Maria Elena Ortega-Hesles, “School Choice, Mismatch, and Graduation,” University of Western Ontario (November 2022)

Feedback Welcome:

Have ideas for improving our podcast? Send them to our producer Nathaniel Grossman at [email protected].

Cheers

• “We are in the midst of another national argument about schools. But, as usual, the squabbles do little good for children who need to learn.” —Jay Mathews

Jeers

• The San Francisco Unified School District hinders disadvantaged students by denying Algebra I to all eighth graders, regardless of their preparation and motivation. —San Francisco Examiner
• “Schools kick students out but call it a ‘transfer.’” —Hechinger Report
• Some teachers unions are opposing legislation that mandates schools use science-of-reading- aligned curricula, claiming it’s a “slap in the face to educators.” —Education Week

• A majority of Americans don’t believe a college degree is worth the cost. Economist Bryan Caplan agrees and thinks college is worthwhile for only “A” students in high school (and “B” students if they pursue certain majors). —Wall Street Journal
• “Can apprenticeships work in the U.S.? Employers seeking new talent pipelines take note.” —Havard Business School
• NCIEA president and executive director Scott Marion discusses the advantages and challenges of through-year assessments. —National Center for the Improvement of Educational Assessment
• Lawmakers across the nation are cracking down on student misconduct in a sign that discipline “reform” fever is breaking. —Chalkbeat
• “New campaign tries to convince parents that their kids have fallen behind.” —Education Week
• The co-founder of Rocketship Public Schools reflects on the charter network’s first fifteen years. —The 74