Curriculum Associates CEO Kelly Sia recently defended her company’s flagship i-Ready platform to Fordham readers. The program, used by millions of students, has come under fire from teachers, parents, and students as part of an ed-tech backlash sweeping the nation.
To set the record straight, Sia stated that Curriculum Associates “welcome[s] honest scrutiny of our research.” She then pointed to a large correlational study out of Georgia State University that evaluated i-Ready use in one large southeastern metro district during the post-pandemic recovery as proof their approach “works.”
She’s right to build her case from it. This appears to be the only independent, peer-reviewed journal article directly evaluating i-Ready Personalized Instruction outcomes. So let’s take her up on the offer and scrutinize it. When you read the fine print, a very different picture emerges than the one she’s selling to us.
Start with the number she would like us to focus on. To show her product lifts the achievement of traditionally underserved students, Sia quotes the study directly, and the quote sounds impressive: Students who were “Black, male, free and reduced-price meals–eligible, or with intellectual disabilities experience(d)…up to 14 weeks of instructional growth.”
But look at what she leaves out. Read the actual study and you learn that the 14 weeks applied only to the intellectual disability group, only in reading, drawn from a sample of around 900 students. For every other named group, the effect was 2.2 to 3.5 weeks. Four groups were bundled together, and the biggest result was associated with all of them.
This is why we can’t trust a for-profit ed-tech company to interpret research on their own platform or write their own review. It’s in their interest to cherry-pick the quote that sells the product.
And it’s important to understand what 2.2 to 3.5 weeks actually means, because converting achievement into “months of learning” can be misleading and mathematically messy. A cleaner approach is to look at percentile growth. Translate the effect sizes and you get a bump of 1–2 percentile points over an entire school year. In plain terms: A student at the 50th percentile moves to the 51st or 52nd by June. The result is statistically significant, yes. But a result can be statistically significant and still be practically minuscule.
There are other facts in the study that Sia omits and that any administrator considering this program deserves to know.
The most important one: The study found that the top third of students in prior achievement derived virtually no statistically significant benefit from the platform. They spent hours on i-Ready’s software, and their math achievement didn’t budge, showing only a tiny, borderline blip in reading. And consider this: The district deliberately prioritized the platform for its lowest-performing schools. So this sample’s “top third” aren’t necessarily all high achievers. Many are likely closer to the national median. This means in a typical district mandating many hours of screen time, the group experiencing no meaningful effect could in fact be much greater than one third. As a father of two high-achieving children who are mandated to use this program every day, I watch this “zero effect size” play out with my own children. It’s frustrating to know they are spending valuable instructional time in front of a screen receiving no measurable benefit.
Then there’s the finding about teacher involvement. Sia notes that “outcomes improved further when teachers assigned lessons in addition to computer-assigned ones.” That’s true. Researchers found a strong positive correlation when teachers manually assigned lessons. These teachers did so 16 to 19 percent of the time, which translates to nearly one in every five lessons in math and one in six in reading. As a Title I teacher who has used i-Ready for seven years, I can tell you that’s not a trivial amount of work. It requires paying close, sustained attention to the students in front of you.
Sia wants you to read this as proof the program works and works even better with a teacher’s help.
But this is a correlational study, and this explanation is simply her interpretation of the results. Here’s an equally plausible explanation: The “added benefit” has nothing to do with the software. Instead it is showing us the impact on student achievement from the kind of teacher who watches student data closely and is likely a more effective instructor overall. Was the benefit from the teacher or the program? With a correlational study, you can’t tell. It’s messy and open to myriad interpretations.
Even the researcher whose framework Curriculum Associates uses to market the program’s effectiveness, Matthew Kraft of Brown University, says that correlational estimates should be viewed with heavy skepticism. He cautions that “correlational relationships are, on average, substantially larger than causal effects” and that “effect sizes from studies based on correlations or conditional associations do not represent credible causal estimates.” This means, by the company’s own chosen measuring stick, the bump of 1–2 percentile points is likely an overestimate of the true effect. Even taken at face value, it falls below the 0.10 threshold the company’s own efficacy page calls meaningful.
The researchers were candid about this in a manner Sia was not. She says the study shows i-Ready “works.” The authors say: “Our analyses are descriptive versus causal as we cannot fully account for endogenous variation.” They directly admit that they are “unable to collect data on or account for other recovery interventions occurring during the same period,” and are “unable to fully understand or account for selection into treatment.”
In other words, they couldn’t separate i-Ready from everything else happening during the chaos of post-pandemic recovery. We can’t tell if the software was driving the results or was just along for the ride.
None of this proves the program doesn’t work. It proves we don’t actually know. To make a strong causal claim here, Curriculum Associates needs a randomized controlled trial or a much stronger causal design. That’s the kind of evidence Curriculum Associates has avoided across the product’s 15-year history.
If Curriculum Associates truly believes in the efficacy of their product, they should prove it: Fund an independent, pre-registered RCT and commit to publishing the results no matter what they show. As one of the largest and best-funded ed-tech companies on the planet, they are well positioned and have the resources to do it.
None of this is trivial. While the program license is cheap at roughly $30 per student per year, and on paper that looks like a strong return on investment, the true cost is paid with instructional time. Just consider how running i-Ready with fidelity eats up classroom minutes before a single online lesson begins. There’s the logistical process of distributing devices, managing logins, untangling headphones, and navigating websites, only to reverse this process minutes later. Even with a very conservative estimate of five minutes for this entire transition, daily use would add at least 25 minutes of lost instructional time per week. And that’s assuming kids stay logged in, focused on their work, and ignore the temptation of the entire internet that is only one tab away. Add that to the 30 to 49 minutes the company recommends, and you are realistically looking at a minimum of 55 to 74 minutes per week per subject. That is the equivalent of an entire core instructional block lost to an algorithm every week.
All that for a likely inflated boost of 1–2 percentile points for the neediest students and no measurable benefit for a large fraction of the kids who are mandated to complete this daily screen time ritual. Such a mandate should require a heavy burden of proof.
To be clear, this isn’t an argument against all educational technology. If a tool can prove with robust independent evidence that it’s a better use of students’ time than alternative practices, I’m all ears. i-Ready has yet to provide this proof.
So is the juice worth the squeeze?
Sia says she welcomes honest scrutiny. Good. Fund an independent RCT and publish the results, whatever they are. Until then, the honest answer is the one the fine print already gives us: not enough for the squeeze they’re asking schools to accept.
