Many complain, legitimately, that the ed-reform world has been overly focused on math and reading scores, to the detriment of other important—but not as easily assessed—student outcomes. This working paper out of the University of Arkansas aims to address this issue by exploring a potential new measure of non-cognitive ability: survey-item response rates. The authors use data from the 1997 National Longitudinal Survey of Youth that tracks a nationally representative sample of young adults; respondents are born between 1980 and 1984 (making them between the ages of twenty-nine and thirty-three now) and are surveyed annually on issues like employment, assets, and wages. When the analysts compared information collected in 1997 to the respondents’ highest educational outcomes as reported in 2010 or earlier, they find that the number of items either left blank or answered “I don’t know” is a significant predictor of educational attainment, even after controlling for many factors, including cognitive ability. The fewer the number of questions left unanswered, the greater the likelihood overall that the respondent had enrolled in college. (For example, a one-standard-deviation increase in response rates increased the amount of education received by .31 years, or 11 percent of a standard deviation.) The authors posit that failure to respond to these questions could mean a loss of interest or lack of effort, which they contend is a valid measure of conscientiousness. While one might dispute this assertion—and while the inclusion of “don’t-know” responses in their count could be an issue, given that the respondent could legitimately not know the answer to that particular question—this is an intriguing, creative, and promising exploratory study.
SOURCE: Colin Hitt and Julie R. Trivitt, “Don’t Know? Or Don’t Care? Predicting Educational Attainment Using Survey Item Response Rates and Coding Speed Tests as Measures of Conscientiousness,” EDRE Working Paper No. 2013-05 (August 2013).
