The concept of better teacher evaluations once held such promise, but lost its luster rather quickly. Well before COVID-19 hit, many states had backed away from only recently adopted teacher evaluation policies. With COVID-19 now dominating the landscape, more schools and districts are opting to forgo teacher evaluations.

A new study by Thomas Dee (Stanford University), Jessalyn James (Brown University), and James Wyckoff (University of Virginia) considers the durability of evaluation systems, albeit in a pre-pandemic context. They examine what is arguably the best known of these systems, District of Columbia Public Schools' IMPACT system, 10 years into implementation.

Dee et al. assert that after the initial burst of enthusiasm, it would be reasonable to expect that IMPACT's initially positive outcomes would dim, but in fact, they argue that it remains an effective system.

Their study is fascinating. They examine what happens to teachers who score just under the threshold needed to qualify for a higher rating category, such as a teacher found to be "Minimally Effective" (though just a few points shy of "Developing," comparing her to a teacher who did squeak passed the line to earn "Developing"). Though presumably quite similar, the responses to their different evaluation ratings were quite different, likely because each faced different consequences. While teachers earning top ratings can receive substantial bonuses and those in the lowest rating category are immediately subject to dismissal, teachers earning a "Minimally Effective" rating two years in a row are also subject to dismissal.

Teachers with a "Minimally Effective" rating were 11 percentage points more likely to leave the District the following year than teachers scoring just a few points higher in the "Developing" category. Teachers who didn't leave after receiving a rating of "Minimally Effective" made real progress in their scores the next year, much more so than teachers earning a few more points.

Past studies suggest the import of these trends: As less effective teachers leave or become more effective, student learning measurably improves.

Notably, these effects are quite similar to what past studies of DCPS's IMPACT found. Through ongoing revisions (modifying the incentive structure to emphasize effective teaching in high-need schools, adding a rating category, and reducing the weight of value-added scores), IMPACT continues to have an impact. What hasn't changed is holding true to the key policy goals, maintaining principal buy-in, and creating strong supports for teacher improvement. Other districts need to take note.