There is a growing body of research that evaluates the effectiveness of supports for college students in helping them persist in school and complete their degrees. Anything from propitiously timed financial incentives to in-depth support systems can make the difference between continued attendance and dropping out, especially for first-generation, low-income, and other underrepresented students. A new NBER paper adds to that literature, examining the role that professors could play in supporting at-risk students.
Specifically, this study—a huge project including both a pilot and a scale-up intervention at a single large university—looks at the impact of personalized and strategically timed e-mails from professors to students as a means of increasing student engagement, performance, and persistence. These are large lecture courses, so this level of personal outreach is not typical. The first e-mail welcomes the students to the class and provides tips on how to succeed in the course in general; the second and third e-mails provide targeted performance feedback timed midway through the course and just before the final exam. In short, the professor e-mailed the treatment students individually to give them customized input on their latest assignments.
The three–e-mail intervention is implemented in two waves. First, in spring 2016, analysts randomly select one half of students in fourteen large undergraduate classes as the treatment group. Then, in the fall 2017, to assess whether spillovers might be biasing their estimates, they draw the treatment in two ways: in eight large classes, they randomly select a third of students into treatment and also randomly select the entire class to receive treatment in ten classes where the same professor taught two sections of the identical course. Because professors volunteered to participate, researchers also looked at differences between them, finding that, although their Rate My Professors ratings were not significantly different, there were demographic differences between them relative to gender and race. It is unclear what impacts those differences may have had.
On average, the e-mails had no discernible effect on student performance in the course, including by course grades, percentage of points earned, passing the course, and earning an A or B. However, student survey data show that, on average, treatment students responded that the professor was more approachable, available, and caring and that she or he kept them informed. When parsing heterogeneous effects, they find that first-year students particularly benefit, especially those who are less prepared according to their high school GPA (they outperform control students by around a third of a letter grade). Also, freshmen men of color—black and Latino—are six percentage points less likely to drop the course.
Researchers also find a positive and sizable “spillover effect” for additional nontreated courses (no professor e-mails) for black and Latino students who are first years, male, and have low high school GPAs. For instance, the treatment effects for black and Latino freshmen is 0.31 grade points for treated courses and 0.15 grade points for nontreated courses. In summary, this light-touch intervention significantly improved students’ perceptions of their professors and courses across the board and improved the course performance of underrepresented students in their first year of college in particular.
There is one significant unanswered question regarding the scalability and sustainability of the treatment protocol. Although student survey data show that personalized feedback on assignments likely enhanced outcomes—and this outreach is something that would, in a best-case scenario, routinely be part of a professor’s job—the time commitment in large lecture classes would likely preclude such feedback on a regular basis. Qualitative survey data from the professors underscore that reality. Still, any institution interested in supporting its most vulnerable students could do worse than learn how to build capacity for personalized faculty engagement with this light-touch intervention.
SOURCE: Scott E. Carrell and Michal Kurlaender, “My Professor Cares: Experimental Evidence on the Role of Faculty Engagement,” NBER Working Paper #27312 (June 2020).
