Editor’s note: This essay is an entry in Fordham’s 2026 Wonkathon, which asked contributors to answer this question: “What rules should govern the uses of digital technology in schools—and who should set them?” Learn more.
Author’s note: These are my own views, not an official position of the Center for Educational Progress, where I’m Director of Operations. In the interest of disclosure: CEP is currently the plaintiff in a lawsuit against Curriculum Associates, the maker of i-Ready, challenging certain of i-Ready’s marketing claims. Nothing here should be read as commentary on that case, on any legislation CEP is involved in, or on any other product or vendor.
1. Frictionless surfaces
Much talk about how ed tech can or should be used in the schools reminds me of a particular type of high school physics problem. They’re the ones that begin with “Assuming a frictionless surface…,” and they often describe a scenario where an object’s velocity stays constant because it isn’t encountering any resisting forces. Despite describing conditions that do not exist in the real world, these models still work—i.e., produce close-enough answers—because the effects of the friction that they assume away are usually small, and so can be factored in afterward.
Popular arguments and programs in favor of ed tech adoption frequently make similar assumptions. The problem is that friction in the classroom, the home, and the schools is of a different kind than the forces assumed away in those basic physics problems. An ed tech product might show clear gains in a controlled setting: maybe it’s a supervised pilot, or a particularly motivated school or classroom adopting a device for a clear, limited purpose. Those results then get touted at trade shows or advertised to an Assistant Superintendent for Curriculum & Instruction, who usually exists in a procurement regime that incentivizes constantly adopting new technologies—mostly just software now—because the future of learning always had to be digital.
Yet all of these new software packages are getting loaded onto one-to-one laptops and tablets that students are now using in class and are expected to use at home. And neither in the classroom nor at home are those frictional forces as minimal as they are in the physics problems. Distractions, multitasking, and the glowing pull of machines that are literally built to do everything are the main forces acting on the object. They are the new learning environment we built.
2. Abstracting and idealizing
All reasoning abstracts: it leaves things out, and research (especially education research) has to as well. One way it can go awry, though, is through idealization: reasoning about agents by attributing capacities that real humans lack, and then treating those capacities as the standard.[1] Ed tech’s evidentiary habits do both. They (and we) frequently reason about a kind of student that doesn’t really exist and then appeal to implementation problems to explain why new learning software hasn’t worked. But the flaws are in the way we model ed tech generally.
What education policymakers need is a revolution in the old sense of the word: a turning back. Instructional software should have to prove it beats analog materials under conditions students actually learn in. And the decision whether it has or has not must sit with individual districts; not with vendors, nor outsourced to some state agency list of products keyed to ESSA evidence tiers that don’t have enough to say about real classroom conditions. What states should do is establish a clear standard—like “Any instructional software purchase must carry the burden of proof against an analog comparator under deployment conditions”—and then ensure that district-level adoption decisions follow it.
3. Software and noise
Why the special standard for ed tech, i.e., new educational software, when hardly any educational product, analog or digital, has been subject to such a regime?
Because laptops and tablets are different from textbooks and worksheets. The latter are designed to do one thing, but laptops and tablets are designed to do…everything.[2] The nature of the digital device is to not specialize for any particular task, but to maximize use cases across domains. So they’re good for answering emails, watching short-form videos, playing video games, browsing the web, and so on. Moreover, digital devices and the apps on them are designed to maximize how much each individual uses them. The essence of their design is to capture our attention, and so the frictions are latent in the products themselves. Attentional derailment devices, of a sort.
The field of education already struggles to get clean signals. Multiple large meta-analyses have shown that education research, broadly speaking, suffers from serious epistemological problems. This ranges from the methods it uses to produce knowledge to the standards it holds peer-reviewed research to. The latter is part of a larger replication crisis in the social sciences, and the former also directly impedes getting better, more efficient practices to teachers and schools.
Now, more and more learning is being done through broader and more “personalized” learning software packages. Whatever that will ultimately mean for education research, I suspect the end result will be more noise and more opaque signals. When we consider current research into ed tech, however, the idealized student-user implicit in much of it frequently adds a layer of bias to the results. Where it might be possible to cut through noise with enough data, a bias resulting from an idealized model user will keep distorting results no matter how large the sample. All of this, to me, suggests the more reasonable attitude to take toward ed tech adoption is a default skepticism of any software’s claims to improve learning.
4. District decisions, state standards
So what would carrying the burden of proof look like? State education agencies could supply a common protocol for carrying out on-site pilots under real conditions: some classrooms could use the new software program, while comparable classrooms would use the analog materials that the ed tech claims to effectively replace. Every purchase request would have to name the analog comparator being replaced alongside the evidence that emerged from the classroom deployment trial. Lacking either, the request would be denied. A similar test could be required for any software contract renewal, too, which would prevent schools getting locked in to products that frequently get worse or bloated over time.
Districts could also start with weaker, more feasible versions of the analog standard by conducting the test during summer learning programs, or in supplemental instructional periods with students who are lagging or struggling more than their peers. (They could even start by comparing new ed tech products to ones they already use.) But because the standard would only apply to student-facing ed tech, it would not prevent teachers and administrators from using cutting-edge technologies to improve everything happening around and behind instructional time. AI-powered software obviously has immense potential to streamline administrative processes, and teacher-facing diagnostics and planning tools are already starting to arrive. The risk will be that these products get bundled with less-effective, flashy student-facing ones, locking schools into entire “systems” that end up eating budgets and employee time with few benefits to show. Districts should thus require student-facing components to be priced and bundled separately.
5. Slow progress is good progress
Maybe the country is already too far down the ed tech rabbit hole to make a strict “analog comparator standard” feasible. But there are nearly 14,000 districts that are at least able to try some version of it. While the standard won’t undo years of one-to-one rollouts overnight, perhaps it does not need to. All that’s needed is to get the ball rolling in the right direction again, making sure to factor in the friction this time. That will be the first step toward restoring a healthy technological environment in the schools.
[1] O’Neill, “Abstraction, Idealization and Ideology in Ethics,” in J.D.G. Evans (ed.), Moral Philosophy and Contemporary Problems (Royal Institute of Philosophy Lecture Series 22, Cambridge University Press, 1987), pp. 55–69.
[2] Thanks to Laura Mazer for making this point clear to me.
