When the University of California began phasing-out college admissions test scores as part of a recent legal settlement, the rationale was “equity.” Lawyers for the students who brought the lawsuit said that “SAT and ACT scores are largely a proxy for a student’s socioeconomic background and race,” rather than measures of academic merit. The years-long backlash to standardized tests has often been couched in this way. To many, the test scores gaps among different groups of students is evidence that they are part of an institutionally classist or racist system.
But if the pro-equity solution is to eliminate the tests, what are we left with when we decide who gets admission to selective colleges? And would the remaining factors really enable admissions officers to evaluate student aptitude from a perspective that is less associated with race and class?
A new study from researchers at Stanford University and Mount Holyoke College speaks to this issue and suggests that eliminating tests would not foster greater equity. Specifically, the study finds that the content of college admissions essays are even more correlated with a student’s socioeconomic class than SAT and ACT scores are.
In this groundbreaking study, the researchers analyzed about a quarter million college admissions essays from the 2016 admissions cycle of the University of California system. Content analysis software identified different topics that the students wrote about (e.g., “winning competitions,” “medical experiences,” or something about the student’s specific major), as well as identifying information about the form and vocabulary used in the essays. The results of the content analysis were then compared with students’ SAT scores and the reported incomes of the students’ families.
In line with many other studies, the researchers find that family income and SAT scores are indeed correlated.
What should give anti-testing advocates pause, though, is that the researchers found an even stronger statistical link between family income and the form and content of the students’ essays. In other words, if an admissions officer evaluated candidates based only on their essays, and remained blind to their SAT scores, they might pick students in a way that disadvantages low-income students more than they are already disadvantaged. (The researchers did not investigate differences by race. But because race and class are themselves correlated, it is likely that some of these findings would hold for racial gaps, too.)
This is the first study to examine essay content in this quantitative way, and some questions remain regarding the extent to which the algorithms evaluate the essays in ways similar to human admissions officers. Still, the findings should shake up the emerging conventional wisdom that standardized testing is a barrier to equity.
For some, this news may come as a surprise. But in reference to this paper, policy blogger Matthew Yglesias spoke for the testing hawks when he noted (in a tweet that has since self-destructed) that the “Only possible way to predict that downweighting tests would further privilege the privileged would have been to think about it for 10 minutes.”
SOURCE: AJ Alvero et al., “Essay Content is Strongly Related to Household Income and SAT Scores: Evidence from 60,000 Undergraduate Applications,” Science Advances (October 2021).
