First-year teachers—especially those who begin on the lower end of the performance scale—tend to improve over time if they remain on the job, according to rigorous research studies. These findings are the same whether the analyses use student test scores or rubric-based classroom observation scores. But little is yet known about what exactly drives their improvement. A new study using data from Tennessee aims to shed some light, combining both types of scoring.
Beginning in 2011–12, Tennessee implemented a new evaluation system that required rubric-based classroom observations for all teachers every year, conducted by a principal or assistant principal. The number of annual observations range from 1 to 4, with newer teachers typically evaluated more often than their veteran peers. Grade level, subject matter, and prior evaluation results also play a part in the number of observations. Teachers are rated from 1 to 5 on each of nineteen rubric items, and evaluators also select one item as an “area of refinement” (AOR) for the teacher. For this study, analysts focused on AORs as a ripe area of comparison to observe teacher performance changes over time.
The choice of AOR is up to each evaluator, but the state offers these guidelines: (a) Which areas on the rubric received the lowest scores? (b) Which of these areas would have the greatest impact on student achievement or other areas of the observation rubric? (c) In which area will the teacher have the most potential for growth? School leaders are then supposed to help the teacher improve in whatever area is chosen.
The research team looked at results from 650,000 classroom observations (including those for about 17,000 novice teachers) conducted from 2012–13 through 2018–19, comprising all rated rubric items and all AOR selections. Students’ value-added scores for teachers over this time period are also analyzed. Empirically, they found that novice teachers and veterans show sizeable differences in which rubric items are chosen by raters as the AOR, with novices substantially more likely to receive an AOR relating to classroom management and the presentation of instructional content—basic teaching skills—whereas veterans are more likely to receive an AOR relating to higher-order aspects of the job, such as activities that teach students problem-solving skills. The lowest-performing novice teachers (based on value-added scores) were also more likely to receive these basic-skill AORs than their high-performing peers. Higher-performing novices and veterans are remarkably similar across most rubric item scores and chosen AORs.
To determine whether identification of an AOR leads to improved practice, the researchers constructed a balanced panel of teachers for whom they could observe the first five years on the job. They mainly focused on the two skills—presenting instructional content and managing student behavior—that showed the biggest novice-veteran gaps. They observe that the lowest-performing quartile of novices become relatively more effective in these areas, judging both from rubric scores and by the decreasing probability that these items are chosen again as the AOR as the novices gain experience. Additionally, there are increases in the probability that higher-order skills will begin to be chosen as the AOR for these teachers over time. However, novices who were higher performing at the outset show far less improvement over their first five years, indicating that they have likely already mastered the basics but still require time to start building the higher-order skills that showed up as their AORs.
In discussion, the researchers assert that their findings show improvement in overall effectiveness of novice teachers in the first few years on the job, driven specifically by improvements in core teaching skills among the lowest initial performers. Evaluators are seeing and noting these basic skills as needing refinement and are likely, as the Tennessee evaluation system expects, working with the novices to improve these specific practices. While the same process is happening with higher-performing novices, the skills they need to improve appear less likely to rise in the first five years, despite whatever assistance is being provided to them.
Overall, these are promising findings, as they provide a tentative roadmap for how school leaders and policymakers can best support new teachers. However, this analysis includes only individuals who stayed in the classroom for at least five years; nearly half of new teachers head for the exits before then. That could lead to some biasing of results. For example, it’s possible that low-performing teachers who showed little or no improvement were nudged out the door—or exited on their own—and thus did not get included in the study. Additionally, we have no data on what specific supports or development efforts school leaders provided in response to any AOR selections, which somewhat blunts the wider applications of these findings.
SOURCE: Brendan Bartanen, Courtney Bell, Jessalynn James, and James Wyckoff, “‘Refining’ Our Understanding of Early Career Teacher Skill Development: Evidence From Classroom Observations,” Educational Evaluation and Policy Analysis (January 2025).
