Depending on whom you ask, teacher evaluation is a vital part of helping educators grow and improve, good for some situations but not for others, or a punitive exercise which serves to demean the profession. Whatever one’s bent, in-person observations are a large part of the teacher evaluation landscape, and they come with significant time costs for both observers and teachers. Administrators can take between ten to thirty hours to evaluate each educator, including observing, writing up, and discussing feedback. A recent study conducted by Tom Kane, David Blazar, and colleagues asks if there’s a better way. They examine the impact of substituting videotaped lessons for in-person principal observations for purposes of evaluating teachers.
Kane and his team conducted a randomized field trial in four sites in Delaware, Georgia, Colorado, and California, involving 433 teachers and 134 administrators. Schools were randomly assigned to the control group, which had traditional in-person observations (typically two or three per teacher per year) or the treatment group of video observations, which allowed teachers to record themselves (approximately twice per month) and to submit three videos of their choosing for evaluation. Observers would then time-stamp the video with comments and discuss with the teacher afterwards using the same rubric for all teachers in both evaluation types. Fifty-two schools were randomly assigned to treatment, and fifty-five to control. Researchers also collected a variety of achievement and survey data from teachers and students.
Relative to the control group, the study found no difference in terms of the amount of time devoted to teacher observations as reported by administrators. Both groups reported spending about forty-one minutes per week addressing various aspects of the observation process for a randomly selected teacher. However, though it did not save time in the aggregate, administrators in the treatment group shifted their observation work to times of the day or week when classes were not being held, such as before and after school, during lunch, in the evenings, and on weekends or holidays. In fact, the timestamps on the video comments showed that about two-thirds of evaluation work occurred during non-instructional hours.
Treatment teachers reported perceiving the observation process to be less adversarial and more fair and supportive than did teachers in the control group. They were also more likely than control group teachers to identify a specific change in their practice and to be more critical of their teaching. Treatment principals were also were more likely to report that teachers were rarely defensive during the post-observation meeting; however, they were also 20 percentage points less likely than their control group peers to report that teachers had a better understanding of student learning and classroom challenges in their school as result of the classroom observation process.
In terms of retention, the video intervention had significant positive impacts on whether teachers remained in their same grade, school, and district the following year. But when it came to student responses, researchers found no difference between the groups in terms of how students viewed their classroom experiences—engagement, classroom management, and so on. And arguably the most important finding: Video evaluations had no impact on student achievement.
The mixed bag of findings—a win for amity and convenience, a wash for self-reflection and student achievement—defies easy explanation. Kane and his team suggest that having teachers select videos could conceal particular instructional weaknesses, making the process less informative than a drop-in, in-person observation. (In fact, principals in the treatment group reported their own reservations about ditching the in-person evaluation.) It could be that videos would work better as a coaching tool, with video feedback including specific instructional suggestions that teachers practice and then resubmit to their supervisors. That way, the video becomes an invitation to submit your worst lesson, not your best. Learning from one’s mistakes is not only a lesson that teachers should preach to students, but should also practice themselves.
SOURCE: Thomas J. Kane et al., “Can Video Technology Improve Teacher Evaluations? An Experimental Study,” Education Finance and Policy (April 2019).
