I’ve posted a lot about the shortcomings of using Value-Added Measurements (VAM) and, in general, any kind of student test scores in standardized tests in teacher evaluation (see The Best Resources For Learning About The “Value-Added” Approach Towards Teacher Evaluation).
Another study has come out confirming those shortcomings, but using a different “lens” to look at it…
Non-Cognitive Ability, Test Scores, and Teacher Quality: Evidence from 9th Grade Teachers in North Carolina has just been released by C. Kirabo Jackson at Northwestern University.
His study considers the widespread research that non-cognitive skills like self-control and perseverance have been shown to have as great an impact, if not a greater one, on long-term success than cognitive skills. Given those findings, he looks at how teachers who have generated high and low student test results have done in generating positive student outcomes in those non-cognitive skills (which he measures by “student absences, suspensions, grades, and grade progression”).
Though slogging through academic papers like these often lead me to thoughts of shooting myself, I think this paragraph sums it up and is worth a read:
In sum, the results indicate that a teacher’s effect on test scores and other non-cognitive outcomes are largely orthogonal such that teachers who tend to improve test scores are no more or less likely to improve non-test score outcomes…. It is clear that a teacher’s effect on non-cognitive skills is essentially missed by her effect on test scores.
This implies that roughly half of teachers classified as above average at improving test score will be below average at improving non-cognitive ability and roughly 25 percent of teachers in the top 25 percent of improving test scores will be in the bottom 25 percent at improving non-cognitive ability. Because unexplained variability in outcomes associated with individual teachers is not just noise, but is systematically associated with their ability to improve unmeasured noncognitive skills, classifying teachers based on their test score value-added will likely lead to large shares of excellent teachers being deemed poor and vice versa
…. Another implication is that if teachers must expend less effort improving non-cognitive ability in order to improve cognitive ability, regimes that increase the external rewards for test scores (such as paying teachers for test score performance or test-based accountability) may undermine the creation of students’ non-cognitive skills (Holmstrom & Milgrom, 1991). In light of the large estimated benefits to higher noncognitive skills (particularly for students at the lower end of the earnings distribution) in Table 2, this may be cause for concern.
A cause for concern, indeed….