No Bites on Friday (to help you avoid indigestion). Back Monday after Thanksgiving. Let’s hope there are some clips worth snarking upon when we return.

1. Nonprofit organization HOPE Toledo recently received a $300,000 grant to continue its work providing full ride college scholarships and additional supports to graduates of Toledo City Schools’ Scott High School. The piece doesn’t say where that new seed money came from, but it seems like folks are grateful for all the additional green no matter what. Longtime Gadfly Bites subscribers (Happy Turkey Day to all six of you!) will see what I did there. (WTOL-TV, Toledo, 11/17/21)
    
2. In Cincinnati, a new all-boys Catholic grade school is planned for the Bond Hill neighborhood in fall of 2023. It will be the first in the low-income neighborhood and the donors responsible for getting the ball rolling are hopeful that the new option will prove attractive to local families. New boys’ school coming. (Local 12 News, Cincinnati, 11/18/21) Between now and then, a number of changes and consolidations will be coming to the various parishes—and schools—in the Diocese of Cincinnati in response to a shortage of priests. (Spectrum News 1, 11/22/21)
   
   

3. Meanwhile, in the Cincinnati City School District, the president of the elected school board seems defiantly determined to win back the love of SORTA, the local transit agency, despite the rather rancorous relations reported in the news over the last several months. Her stated desire to rekindle the decades-old partnership for transporting district students again next year also runs directly in opposition to her own board’s vote to look for other partners, a situation which made it impossible by law for SORTA to work with the district—a fact which SORTA officials are having to awkwardly repeat in the press. Ouch. Love can be so complicated sometimes, can’t it? (Local 12 News, Cincinnati, 11/22/21)
    
4. I don’t want to call it a Thanksgiving miracle, but somebody out there seems to like really like Ohio’s Third Grade Reading Guarantee (both the letter and the spirit of the thing) according to this opinion piece published yesterday. And that someone is an elementary school principal! (Daily Advocate, 11/23/21)
   
   

5. And finally on holiday eve: What more needs to be said about an amazing-sounding afterschool program in Dayton that has been running for more than a decade, is entirely volunteer-staffed, teaches coding as well as career and life skills, has done more to fire up interest in computer science among local youth than Dayton City Schools, and is run by a 71-year-old retired Army major? Only “more of this, please!”. (Dayton Daily News, 11/21/21)

Did you know you can have every edition of Gadfly Bites sent directly to your Inbox? Subscribe by clicking here.

• Though surveys showed that many teachers considered leaving the classroom in recent years, and some predicted nationwide teacher shortages, the data show that teachers are sticking it out. —FiveThirtyEight
• “Can public money go to religious schools? A divisive supreme court case awaits.” —Education Week
• Mystic Valley Regional Charter School in Massachusetts is under pressure from antiracist advocates for its view that America is a melting pot. —Time
• Habi Zhang, a Chinese doctoral student, worries that the lack of rigor in U.S. schools is putting her third grader—and the next generation of Americans—at a disadvantage compared to students in China. —Wall Street Journal
• “Democrats’ child care plan could help millions—or it could be a big mess.” —Vox

Cheers

• An exam-based high school in San Francisco can continue using merit-based admissions policies after a court overturned a shift to a lottery system. —SFGate
• Indianapolis Public Schools approved a plan to share $27.5 million in referendum funds with charter schools, the first district in the state to do so. —WFYI Indianapolis

Jeers

• The University of California system has announced an official end to SAT and ACT admission requirements—a move that, despite being done in the name of “equity,” could hurt disadvantaged students most. —EdSource

Education leaders—principals, superintendents, state chiefs, philanthropy heads—make lots of decisions, and we exhort them to “use evidence” when they do. But we should stop doing that for at least four reasons. First, K–12 evidence itself is chock full of problems, as many have pointed out on these pages. Second, pestering isn’t a good strategy, as we’ve seen with pushes for Covid-19 vaccination, or I’ve learned while try to get my eighth grader to turn off his bedroom light when he leaves the room. Third, there’s a larger movement—often for good reason—to stop “trusting the experts.” And fourth, even when there’s “good evidence” for an education practice, service, or product, K–12 actors often find it irresistible to do a cheapo version of that thing.

For example, there is good evidence for high-dosage tutoring delivered by carefully picked tutors. So what happens? Leaders, flush in 2021 with Covid federal aid, launched scores of low-dosage tutoring programs without careful tutor selection. I predict most will find, years later, that their programs had no effect on kids—like Minnesota Math Corps.

One problem with education evidence is that the “flagship” intervention gets enormous attention to detail. The scaled versions get much less. Rationally, our preferences should be in this order:

1. Best: A scaled, cheap version of X that actually works.
2. OK: An expensive, high-quality version of X that actually works—to be used judiciously, given limited resources.
3. Worst: A scaled, cheap, less carefully managed version of X that doesn’t work.

So why do so many education studies find that examples of #3 are failures? Why are decision makers so overconfident at the moment they choose a “cheap version” of an “evidence-backed” program, rather than the actual evidence-backed program?

It’s because nobody warns leaders when they’re about to do so.

Jim Savage at Schmidt Futures pointed me to a possible solution to this evidence problem: prediction markets. I’ve been thinking about it ever since. It’s a way of getting past the cognitive biases and organizational behavior of leaders and their underlings, and instead involving outsiders in a particular way and getting them to make predictions.

From Vox:

Just like you can bet on sports, you can bet on, well, anything you want, from “fewer than 10 million Americans will be uninsured in 2025 if this bill passes” to “migration from Central America will increase” to “this new cancer study will fail to replicate.” The point is that you can make money by making good bets. And that incentive is what can make prediction markets a pretty good forecaster of what’s to come.

...Why would you expect a betting market to get anything right? Well, because if a betting market is getting something wrong, then a person can make money by betting against the consensus.

One challenge of a forecasting market, however, is that you need lots of bets to make it work. Millions of people want to bet on sports or stocks. Tens of thousands want to bet on politics. How many people, however, want to bet on whether, say, Nashville’s tutoring program will raise achievement for students? Not many!

Wharton professor Philip Tetlock wondered whether there was a “hack” to this puzzle. Do we need a whole forecasting market to see into the future? Or could surveying people in a structured way give us most of that insight? We could call this a forecasting tournament.

Tetlock and his colleagues found these tournaments worked pretty well if you structure it right. You can get the “wisdom of crowds” with a pretty small crowd. You might need just twenty to sixty people—not thousands. Not only do you get a “wisdom of crowds” effect, you also figure out which individuals are good at predicting.

So who would you invite to participate in a K–12 forecasting tournament? First, education researchers. That is something Drew Bailey of University of California Irvine has been thinking about. He told me:

Forecasting has the potential to get a group of educational researchers—maybe one with a very diverse set of prior beliefs and values, one at risk for talking past each other in an insufficiently structured context—to focus on a specific set of questions of interest to the potential funder. Further, forecasts can benefit from this diversity: Forecasters will pay attention to different features of proposed educational interventions, and learning about when they succeed and fail will help us learn about the key features we should be attending to when forecasting the success of an educational intervention.

Moreover, a forecasting tournament provides useful information to participating researchers themselves. Making a forecast before knowing how well a program works will increase the chances that a researcher is surprised when they learn about the results—and receiving surprising feedback helps people change their minds. It may also promote intellectual humility.

Second, reporters and insiders can sometimes predict well, particularly when they’re free to dig around. For example, Nate Anderson of Hindenburg Research predicts stock prices. He’s a short seller, so his goal is to find overvalued stocks. He was curious about Facedrive. It was newly public company valued at $1.4 billion. Think of an eco-friendly version of Uber, with electric or hybrid cars instead of regular ones. Facedrive already had 13,000 drivers. Anderson dug in. He found the true number of “active” drivers was closer to 600. Also he wrote:

Rather than focusing on tackling just one resource-intensive highly competitive market like ridesharing, Facedrive recently entered a second—food delivery. We found Facedrive’s platform has a total of seventeen restaurants compared to Uber Eats’ 400,000 and GrubHub’s 300,000.

We called several of the “most popular” restaurants on the Facedrive Foods page. One didn’t seem to have a working phone number, and two said they don’t use Facedrive anymore.

Anderson found a ton of other problems, shorted the stock, and published his findings. Later, the stock fell 96 percent from its high.

Can that short-selling model work for an education prediction tournament? For example, I can name big city tutoring programs right now that are like Facedrive. One I know of has almost no kids showing up for tutoring after school. Another is so short of tutors that their tutor “group sizes” are huge—which is not really tutoring, but more like a class. I believe—maybe I’m wrong—that a future evaluation of either program will show no positive effect for kids. I would “short” these tutoring programs if I could. I wish there was a way to get my intel to someone who mattered or cared, who would try to fix these problems, or at least stop subjecting kids to low-quality tutoring.

Finally, the third group that likes to forecast are the amateurs. They tend to be quant-oriented people with no domain background. They just like running their own numbers. Last year, for example, Youyang Gu was a twenty-seven-year-old MIT grad working on a sports analytics startup when Covid-19 hit. Gu googled “epidemiology” and noticed that death predictions were very far apart. One model said 2 million dead in the United States by summertime. Another predicted 60,000. Gu wondered if he could do better. Per the MIT Technology Review:

Within a week, he’d built a machine-learning model and launched his COVID-19 Projections website. He ran the model every day—it only took one hour on his laptop—and posted Covid-19 death projections for fifty US states, thirty-four counties, and seventy-one countries

By the end of April 2020, he was attracting attention—ultimately, millions checked his website daily. Carl Bergstrom, a professor of biology at the University of Washington, took notice and commented on Twitter that Gu’s model was “making predictions that seem as good as any I’ve seen.”

Gu’s track record only kept improving. Over the course of the pandemic, he outperformed the credentialed experts.

—

Imagine a forecasting tournament of education researchers like Drew Bailey, plus “insiders or reporters” like Nate Anderson, who can work the phones or show up and observe, plus amateur outside quants like Youyang Gu.

Bailey told me:

If it turns out the Forecasting Tournament has predictive power, forecasts could be considered during the process in which educational interventions are designed or selected for evaluation. This would allow funders to choose more promising interventions. It would also incentivize applicants to attend more to the likely effectiveness of their interventions. Finally, it could help identify individuals with a knack for forecasting.

Over time, funders might attempt to recruit reviewers who have proven to be successful at forecasting impacts in previous work, or even favor applicants with strong forecasting records. I think it would make sense to favor a proposal for an RCT [randomized controlled trial] of an intervention that forecasts a large impact from a research team with a track record of making accurate forecasts over an identical proposal from a team that is consistently over-optimistic.

So an entity like the Bill & Melinda Gates Foundation, the Institute of Educational Sciences, or a big city superintendent could ask a forecasting tournament to predict the effects of ten different large grants they were considering, and make better decisions of what to approve.

Here is an example: The Chancellor of the New York City Department of Education considers a summer school initiative. “Researcher” forecaster 1 predicts a large gain because the summer school model is a replication of a previously successful effort. “Digger” forecaster 2 calls a friend who is a lifer NYC principal who can think of five failed summer school rollouts in her twenty-nine-year career, so forecaster 2 bets against the effort. And “amateur” forecaster 3 hypothesizes that improved air conditioning will change summer school for the better and bets on a small gain.

Eventually, thirty forecasters make their picks, and the tournament prediction is sent to the chancellor. If the consensus is negative, maybe the chancellor does something before the summer school: cancels it, spends more on it, or changes it.

Leaders could also make use of the narrative justifications for these forecasts. That’s where predictors explain their reasoning. The hope is to improve a program before it even begins by understanding the “why” of a prediction, and then trying to gather the implementation team to correct deficiencies before they manifest. In addition, these justifications may make it clear to the funder what parts of grant proposals deserve more or less attention during the review process. For example, perhaps narrative justifications will drive funders and authors to attend more to specific impediments to scaling up effectively, or to more systematic reporting of the impacts of previously funded work by the authors, or to require a more thorough implementation plan.

So imagine two types of forecasting tournaments. In the first, the predictions are “quiet.” Someone creates an intervention and pre-registers a randomized controlled trial evaluation. The tournament gathers people to bet on what the result will be in five years. That’s how we validate who, in fact, are good at predicting. No leaders are involved.

In the second type, leaders ask the forecasting tournament to make predictions, and then they, the leaders, read the narrative justifications. Maybe one sees a lot of “fail” bets and reads that an intervention will fail because summer programs tend to have very low attendance rates, no matter how they’re designed, no matter what the incentives are for students. So rather than try to climb a mountain that is impassable, the leader changes the intervention to focus on willing kids, those who want to show up, instead of chasing resisting kids. The tournament would then discard the original predictions and allow everyone to revote based on the new program design.

Could this scheme work? Could a forecasting tournament provide a new tool that would nudge leaders to “act on evidence”? If so, would the result actually be better K–12 decisions, helping more students?

I want to see the details before I make my prediction.

In the wake of the November elections, hundreds of new school board members will soon take their seats overseeing Ohio’s 600 plus districts. From media accounts, the winners appear to be a mix of school establishment candidates and grassroots contenders running on culture-war issues. While this doesn’t necessarily indicate a widespread mandate for change, there is likely more momentum than usual for challenging the status quo at the local level. Yet beyond addressing touchy issues around critical race theory and masking, what might reform-minded board members tackle? Here are a couple ideas.

Build choice-friendly communities

It’s no secret that most local school boards—and their national and state affiliates—have long opposed educational alternatives that are not under district control. They have traditionally viewed public charter and private school options as the enemy, and seem to understand their role to be defenders of districts’ institutional interests.

Unfortunately, that’s a cramped, territorial view of educational leadership in a community. Let us first note that school board members are not bound by oath to protecting adult jobs or district budgets. Instead, they are elected by the public at large to work in the best interest of students and to represent the broader community. Board members can work towards both of those purposes by supporting school choice. Studies show that expanding choice often improves achievement among students attending both traditional public schools and other alternatives. Moreover, supporting choice might win over more constituents; choice does poll well, and yes, parents of non-district students vote, too. And a choice-friendly posture could make levies just a little bit easier to pass and even prevent culture war clashes from dominating board meetings, as Neal McCluskey at the Cato Institute has argued.

One might counter that there’s little a board member could do to advance choice. It’s true that district boards in Ohio do not oversee the operations of charter and private schools, nor are they responsible for their outcomes. Yet districts can and do impact the choice environment in ways that either encourage or stifle these options. Consider the following ways that board members can help build choice-friendly communities:

• Provide reliable yellow-bus transportation to children attending non-district schools. Despite their legal obligations, districts have been known to create transportation headaches for charter and private school students.

• Include charter schools in levy requests, as permitted under state law. Except for Cleveland, no district in Ohio has chosen to share property tax revenues with charters, leaving them with less overall funding than their district counterparts.

• Offer unused school buildings for sale at reasonable costs to public charter or STEM schools in adherence to state law. Unfortunately, a few districts have evaded this responsibility by nominally operating buildings, demolishing them, or offering them at unaffordable rates.

• Lease unused classroom space to a charter or private school seeking extra square footage. This could be a win-win, as districts receive additional funds while helping to meet the needs of another educational entity.

• Work with charter and private schools to create a universal enrollment system. Much like the common app that allows students to apply to multiple colleges using one form, such a system would make applying for K–12 schools simpler and fairer for parents.

• Participate in interdistrict open enrollment. While most Ohio districts allow open enrollment, dozens of school boards continue to deny entry to non-resident students, limiting public school opportunities for nearby families.

• Ensure that administrators are properly reviewing parental notifications for homeschooling and not demanding more than is required under state regulations. As permitted under state policies, district boards could also allow homeschoolers to take one or two courses at their schools—for instance, an AP or science lab course. Last, they could also make sure that the district welcomes homeschooling students for sports and other extracurricular activities.

Ensure district schools follow the “science of reading”

While choice issues are more akin to foreign policy, board members must also pay attention to domestic matters—and that means improving district performance. There are many ways to work towards this goal, such as overhauling archaic teacher compensation structures, encouraging more advanced coursework in middle and high schools, or evaluating superintendents based on objective measures of academic improvement. But one particularly ripe area for board advocacy is early literacy, most notably insisting that district elementary schools follow the science of reading. This refers to a large body of evidence finding that children learn foundational reading skills best through explicit, systematic phonics instruction as opposed to “whole language” or “balanced literacy” type approaches.

School boards are legally responsible for prescribing the curriculum used by district schools, but how exactly can board members ensure their schools adhere to this science and are helping all children read well? Here are a few ideas:

• Check their elementary schools’ curriculum alignment with ratings from EdReports, a non-profit organization that reviews curriculum and textbooks. If materials are poorly rated—e.g., the Fountas & Pinnell curriculum and Lucy Calkins’ Units of Study receive low marks—they should transition to ones that adhere more closely to the science of reading. Among the top-rated curricula include the Core Knowledge K–2 curricula and McGraw Hill’s Wonders.

• Set aside portions of the budget to ensure dollars are available for curriculum and high-quality professional development needed to implement new programs and practices.

• Encourage administrators to hire teachers and principals with strong backgrounds in the science of reading. That might include hiring graduates from elementary school prep programs that are highly rated by NCTQ, such as the University of Akron and Marietta College. (Though Ohio requires coursework in phonics and literacy, NCTQ goes further by evaluating programs based on whether they dedicate sufficient time to the science of reading and hold students accountable for learning such content.)

• Ensure that struggling readers receive the help needed to read proficiently by the end of third grade. To this end, board members could make sure that elementary schools promptly notify parents of reading deficiencies, as required under state law, and partner with them to implement strategies for improvement. They could allocate dollars for summer and afterschool reading programs and engage external partners, such as local libraries or colleges, that add extra capacity for literacy efforts. Such a focus on early literacy, including being willing to retain students who need extra help, is especially critical given the learning losses of the past year.

• Raise awareness in the community about the importance of early literacy and insist that all children will be able to read when properly instructed and given the right supports.

***

School boards have significant powers to shape K–12 education in local communities. By working towards a more choice-friendly community culture and pushing for effective reading practices, board members can use those powers in ways that support parents, benefit students, and unite communities. 

We’re back after a short break. Looks like we missed a couple of humdingers. Let’s get to it!

1. Like the academic data contained on them, analysis of Ohio’s school report cards was limited this year. Thus, Aaron’s detailed blog published several weeks ago is linked and quoted as part of this piece on them. The remainder of the piece focuses on the Ohio Department of Education’s presentation of “sobering” report card data to the state board of education, which occurred at their meeting last week. (Gongwer Ohio, 11/19/21) Additionally, Aaron published an op-ed at Vindy.com over the weekend in which he opined for higher expectations for Youngstown City Schools’ students…and the adults who are responsible for serving and teaching those students. (Vindy.com, 11/21/21)
    
2. It is so rare that I get to announce a new independent STEM school option is coming (this time in Xenia), and it makes me so happy to do so, that I won’t even bother to whine about the “STEAM” designation in its name. I know the STEM model upon which this new school will be built; I know that families in this region will be clamoring to get in (y’all might THINK families won’t come from Dayton or Cincinnati for this, but I assure you you’re ultimately going to be proven wrong about that); and I know that if it is fully, faithfully, and properly executed, thousands of students will find their proper fit and their academic futures will open wider than ever. So call it whatever you want and put as much arts curricula in there as you feel necessary. If you run a proper STEM school, it’s all good. Woohoo! (Xenia Daily Gazette, 11/18/21)
   
   

3. Speaking of school choice, here are details on a new charter school coming to the Cleveland suburbs even sooner: January of 2022. New Story School is designed to serve students with autism, emotional needs, and various behavior issues from ages 5 to 21. Awesome news! (Cleveland.com, 11/21/21)
    
4. Staying on the topic of school choice for one more clip, and adding a twist of Mountain State irony for good measure: West Virginia’s Professional Charter Board ultimately approved five of the six applications that came before it last week—two virtual charters and three brick-and-mortar ones. It is worth noting, because I have followed this story for a while and because I am something of a brat, that if the Monongalia County School Board had approved the one application before them last year, there would be one brick-and-mortar school in the entire state and the school choice folks probably would have been OK with that. However, because they rejected that one for no rational reason, there will now be five times as many charters, two of which are statewide, with more likely to come just because they can. Life comes at you fast, it seems. (West Virginia Public Broadcasting, 11/17/21)
   
   

5. But all is not rosy in school choice land. You have no doubt heard that a number of school districts across Ohio and across the country are adding random days to their Thanksgiving breaks under banners such as staff mental health needs and staff appreciation. Here’s one such story from Stark County last week in which it is specifically noted that private schools in the area (and charters, too, probably, if anyone in the media cared to check) were also being forced to close—whether they wanted to or not—because bus service would not be provided by the district whose job it is to do so. I don’t know about any of y’all, but if I had found out on Friday that my kids’ school was going to have to be closed the following Monday without clear reason (or without wanting to) and with minimal notice, I would feel the opposite of appreciated. (Canton Repository, 11/17/21) Speaking of the opposite of appreciation, Worthington City Schools’ efforts to cease busing service mid-year for 43 resident students to a handful of private schools outside the district is drawing fire. Their reasoning is stated clearly enough: they want to prioritize “their own students” in light of staffing struggles and boosts in fuel prices. Unfortunately, they have decided to do this—and to express their reasoning for it a bit too plainly—in the wake of some changes in state law (and a sternly-worded reminder of same from ODE) that make it clear that such prioritization against school choice families is not kosher in Ohio. It could be that plentiful money available to districts these days will smooth things over in the end, but not before some more public crankiness I reckon. (Columbus Dispatch, 11/20/21)
    
6. And in Cincinnati, the bad blood between the school district and the local public transit authority—initiated in the weeks just before school started, as I’m sure my longtime Gadfly Bites subscribers will recall—has continued unabated since August. However, it seems from this piece that an end is in sight, by which I mean that Metro will cease providing any student transportation on the district’s dime when the current contract between the two entities expires in July. In the meantime, I’m sure you can expect the same amount of recriminatory rhetoric, demands for refunds without receipts, and students being used as pawns to remain constant. Or get worse. (Cincinnati Enquirer, 11/18/21)

Did you know you can have every edition of Gadfly Bites sent directly to your Inbox? Subscribe by clicking here.