There are two aspects of standardized testing to which opponents tend to object: The testing itself and how the results are used. A recent study out of Denmark looks at the latter, attempting to determine if critical feedback delivered to parents about their child’s performance on a math test adversely affects students along academic and social-emotional dimensions. But in fact it finds the opposite: Negative feedback appears to spur improvement.
Denmark has a nationwide system that requires testing in subjects including reading, math, geography, and the sciences in grades two through eight. Analysts matched four cohorts of third grade students in 2010 through 2013 to their sixth grades scores in 2013 through 2016, respectively. They were able to match nearly 95 percent of all the third grade cohorts, totaling almost 192,600 students. Danish schools are mandated to report the results of the tests to parents via a letter that is automatically generated within their testing system and may be distributed by postal mail or through the school’s online portal. The letter does not include the child’s raw score, but rather a label that converts the score into one of five categories: considerably below average, below average, average, above average, or considerably above average. The scale score that produces the rating is held in strict confidence by the teachers (in fact, the school principal, school board, and municipal board only have access to the average test results by school year and cohort). Analysts accessed the raw test score data and used a regression discontinuity design to compare the outcomes of students scoring just below each category’s threshold—and thus receiving more negative feedback via a corresponding negative label—to the outcomes of students scoring just above the threshold. They produced estimates based on the cutoffs for each of the five performance levels. They also combined student-level scores with student level survey data about multiple aspects of well-being.
The key finding is that negative feedback—specifically, a more negative-sounding label—received in third grade math generates significant increases in math achievement in sixth grade by 6 percent of a standard deviation (SD), compared to students who received the relatively better label or “signal.” This is true regardless of the child’s ability level, meaning analysts observed a similar pattern of improvement all along the performance distribution. Still, students who fell below the lowest threshold—labeled as considerably below average—appeared to benefit even more from having the test information (i.e., the larger impacts amount to 0.12 of a SD for the lowest performers versus 0.06 of a SD for the average student).
It is unclear which group is driving the observed pattern. It could be that those falling just below the cut off get more motivated and try harder, but it could also be that those above the cut off now feel comfortable and reduce their effort going forward. That question aside, analysts are able to determine that the effect seems driven by boys, who perhaps, based on prior research, often overestimate their math abilities compared to girls. Maybe a negative label serves as a reality check for them to step it up.
Analysts also looked at Danish national survey data on student well being conducted in 2015 and 2016, which asked kids about a variety of topics such as school satisfaction, whether they have been bullied, and their intrinsic motivation. These are exploratory analyses because they only have data for two cohorts. Still, they see no systematic negative relationships on students’ self-reported well-being. In other words, information communicated to parents about their child’s relatively poor performance does not decrease that child’s well-being. If anything, there’s some evidence that academic confidence increases for kids in the lower part of the distribution. In short, low performers are no worse off in social-emotional terms after receiving negative feedback, at least insofar as it is defined in this study.
The report concludes that information on test performance in elementary school improves future academic performance in math. Moreover, rather than being viewed as negative or harmful to students, the categorical labels appear to communicate valuable information to parents upon which they can act. The study adds to a growing literature that shows that personalized feedback and informational “nudges” have the potential to motivate action in K–12 education. That said, upwards of 90 percent of American parents continue to believe that their child is performing at or above grade level, despite what standardized test results say. Maybe that’s because the labels that states tend to use to describe performance in general fail to connote negativity at all (and some are inscrutable!).
We’ve got to do better. Let’s not deny families the power of transparent and user-friendly data to help them to help their kids.
SOURCE: Louise Beuchert, Tine Louise Mundbjerg Eriksen, and Morten Visby Krægpøth, “The impact of standardized test feedback in math: Exploiting a natural experiment in 3rd grade,” Economics of Education Review (June 20, 2020).
