There is a lot of confusion over the accuracy of using valued-added measures (VAMs) to judge teacher performance. Is one year's data ever enough to judge performance? Is VAM useful for estimating the performance of all teachers or just those at the extreme ends, i.e., the very best and very worst?

Researchers, Peter Schochet and Hanley Chiang of Mathematica Policy Research mine the cavernous depths of VAM data now available from districts all over the country and conduct simulations to try to address these questions. What they show is not new, but it is interesting and useful.

For example, if a district were to target teachers who were at or below the 18th percentile of performance for some negative consequence (e.g. probation, dismissal, denial of tenure) and used three years of data to calculate that estimate, the VAM estimates would be "wrong" for one out of four teachers (meaning that those teachers were actually stronger performers than the three year calculation showed). If a district relied on only one year of data, the VAM estimates would be "wrong" for one out of every three teachers. However, if a district were to try and identify fewer teachers at the bottom end, say, at or below the 8th percentile, and used three years of data, the accuracy rate would be much improved. Then the VAM estimate would only be wrong for only one out of six teachers.

You might be thinking: wait, if teachers teach the students who take the tests that are used to calculate the VAM, how can they be misclassified? In short, teachers are not the only contributors to student performance. Some classrooms may have more resources, fewer disruptions and/or less needy students, all of which will impact student and therefore teacher performance. This variation in classrooms must be controlled for to assure valid calculations and teacher-to-teacher comparisons. Moreover, some schools may have more resources, better leadership, more capable teachers and/or less needy students than others, which will systematically impact school performance. Again, VAM calculations must remove these resource differences to assure proper comparisons.

Even with these caveats, the authors note that VAMs have distinct advantages to other measures of teacher quality: they are better predictors of subsequent-year outcomes in classrooms than credentials, education, experience or even observational measures.