A few days ago, I wrote a somewhat popular post titled Measurement Matters….Maybe Not So Much. It was about the new paper written by Angela Duckworth (of “grit” fame) and David Scott Yeager (a researcher of “growth mindsets”). The paper is titled “Measurement Matters:Assessing Personal Qualities Other Than Cognitive Ability for Educational Purposes.”
Emily Hanford shared a video the two of them made talking about the paper, which I’ve embedded below. There’s nothing particularly new about it if you’ve read their paper, but it was somewhat interesting to hear them talk about it.
I hadn’t seen videos like this before where researchers talk about a paper that has just been published. Maybe it’s a common practice but, if it isn’t, it seems like it would be a nice way to help laypeople gain a better understanding of research.
You may have heard about the article released today that was written by Angela Duckworth (of “grit” fame) and David Scott Yeager (a researcher of “growth mindsets.” It’s titled “Measurement Matters:Assessing Personal Qualities Other Than Cognitive Ability for Educational Purposes.”
I thought I’d share about the good and the bad in it.
Here’s my summary and “take” on the article and its topics:
They basically say that it is important to develop a common and better term for the range of Social Emotional Learning skills like self-control, grit, growth mindset, etc. — and to particularly reconsider the term “non-cognitive skills.” In addition, they say that the present instruments being used to measure these qualities are imperfect and should not be used for high-stakes decision-making related to students, teachers or schools, and that research should be done to determine better forms of measurement.
It’s great news that they are saying that SEL measurements should not be used for any kind of high-stakes decision-making. I wonder if that means that Ms. Duckworth is stepping away from her research last year suggesting that Districts could determine grit scores from the resumes of teacher candidates and using that information in their hiring decisions (see This Has Me Concerned: “Study Links Teacher ‘Grit’ with Effectiveness, Retention”). I’ve sent that question into her institute and hope to have a response to share soon (Professor Duckworth graciously responded within hours. You can see it at the end of this post).
It’s not so great news that they seem to say what’s really needed in order to start using SEL skills in high-stakes decision-making is just a better way to measure those SEL skills.
I think that entire column is applicable to the issues raised in today’s article, and here’s just one excerpt (there’s some irony in my sharing it here because Duckworth and Yaeger also use the same quote in their paper):
“Not everything that counts can be counted, and not everything that can be counted counts.”
The quote is sometimes (erroneously, it seems) attributed to Albert Einstein. And that’s probably because we tend to think of Einstein as a sensible fellow who understood the limits of quantification. I think we might add a corollary to Cameron’s observation: Going through tortuous gyrations in order to count something doesn’t mean it should be counted.
Maybe I’m naive or unsophisticated (and I’m happy to hear comments suggesting both), but I just don’t see the need to officially measure everything that students do or don’t do.
What are your thoughts on the new Duckworth/Yaeger article and the issues it raises, as well as on my commentary about it?
Response From Angela Duckworth:
I did write a paper that encouraged scrutinizing an applicant’s “track record” of grit (e.g., in college sports or public service or anything else where they could show up, or not, over multiple years and really apply themselves). The thing I like about this measurement approach to grit is that it is, in theory, less fakeable than a self-report questionnaire and also not as susceptible to reference bias, because you do not have to ask the applicant to rate themselves. However, it is like any other measurement approach limited and imperfect. Here is a paragraph from the paper:
“We suggest that school administrators consider grit as one factor— among many—in identifying promising new teachers. While no single factor in isolation should determine a hiring decision, the method for quantifying grit from biographical data developed for this investigation represents a practical tool for predicting success in the first few years in teaching. Despite its predictive validity, policymakers should proceed cautiously when using this measure of grit during the screening process and continue to consider a wide range of variables, not just those that are easy to measure, when making hiring decisions. In addition, before using these measures for high-stakes purposes, districts should conduct their own internal validation studies to ensure grit is predictive of valued teaching outcomes in their sample of teachers.”
So, I hope the message that comes through is that all measures are imperfect, each measure is imperfect in its own way, and we need to recognize the limitations and advantages of any measure when we think about specific purposes to which these measures would be applied.
As regular readers know, I’m a big believer in helping students understand and develop a growth mindset, and have recently published two very popular posts about it.
However, I also think it’s possible to have too much of a good thing.
Carol Dweck, whom I have praised repeatedly, and her colleagues just published a paper on self-control/willpower and a growth mindset. Jonah Lehrer has written the best explanation of it that I’ve seen. Basically, their study suggests that contrary to previous research that has found self-control to be a resource that can be depleted and then needs to be replenished through food, beverage, or other means, in fact, having a growth mindset about your capacity for self-control is really the best way to keep your willpower at a high-level (I believe that’s an accurate summary, but am more than willing to be told otherwise).
I’m not convinced that pushing our students to develop a growth mindset should be the primary strategy we educators use to help our students deal with all the challenges they face, including self-control and stress. Though I think a growth mindset is a critical perspective we want our students to develop, I think also acknowledging that we all have some limitations, and learning strategies to effectively cope with them is an equally important concept and skill to learn (here are strategies around self-control and stress I teach students).
I am not trying to put words into the mouths of Professor Dweck (and Heidi Grant Halvorson) and her colleagues by suggesting that they believe that having a growth mindset is the answer to all these challenges, though I think that pushing these ever-expanding claims about the power of a growth mindset can sometimes leave that impression.
Though there’s nothing in it that regular readers of this blog wouldn’t already know, it nevertheless provides what might be the best readable compilation of important strategies around Social Emotional Learning Skills, assessment, and classroom management that can be found anywhere.
Each section first states the psychological concept, then explains it, and finishes with an analysis of why it’s important to teacher and include practice ways to implement it in the classroom.
If you’d like to learn more details about the concepts they cover, you might be interested in:
I’m a big proponent of Carol Dweck’s research on a growth mindset (see The Best Resources On Helping Our Students Develop A “Growth Mindset”). I use it with my students and, in fact, it’s a concept we push heavily school-wide in our Social Emotional Learning initiative. I’ve seen a number of students positively affected by it, and it’s provided me with a positive tool to improve my classroom’s environment.
I had hoped that some of the 275 comments following the critique might provide me with some clarity, but I was amazed at how few of the comments actually related to the study itself. Most commented on topics as wide-ranging as global warming, El Nino, and speaker fees for academics. Reading those 275 comments is time I’ll never get back .
I’m hoping that readers with far more knowledge of research data analysis might be able to enlighten me about these dueling claims, and will also be requesting the assistance of people who I know have greater knowledge in this arena.
The more I learn, the more I discover I don’t know.
Today, he unveiled a new study that, as usual, has received lavish media attention. This one is very intriguing though, of course, his past work makes me a little wary of his conclusions in this one. It is difficult to be wary,, though, of such common sense results. Basically, he says that low-income children moving from poor neighborhoods to middle-income neighborhoods results in better life outcomes for the kids when they are adults.
Duh, you might say. Of course, living in areas with better-supported schools, less crime, and better community services would lead to more success for kids. Agreed. However, a previous study of much of the same data a few years ago did not find that to be the case. Chetty says his results are different because more years have passed and the positive outcomes took longer to become apparent.
I’m wondering if the study’s conclusions, if accurate, might specifically apply to education policy discussions in two ways:
One, though I doubt it will, it would be great it would quiet those who push Social Emotional Learning as a “Let Them Eat Character” approach to responding to poverty instead of seeing SEL for what it is — if done well, a useful supplement to classroom instruction. Interestingly enough, the most public proponent of that mistaken and damaging perspective is New York Times columnist David Brooks, who pushed it again in a widely ridiculed weekend column. You can read more about this topic at my Washington Post column, The manipulation of Social Emotional Learning.
Despite what Brooks and other might say, escaping poverty is not just a problem of “psychology.”
Plenty of studies have shown that students facing more challenges benefit more being in a mixed-ability classroom than in a lower-tracked one. The counter argument has been that some research shows that advanced learners do not gain similar benefits and are even hurt. However, as Carol Tomlinson has discussed, those studies showing a disadvantage for advanced learners have not been done in classrooms where teachers have been trained in differentiating up (she calls it a “plus-one” environment), as well as differentiating down.
As she writes:
The studies most cited in terms of benefits of homogeneous instruction for bright learners examined two conditions: heterogeneous classrooms in which little or nothing was done to provide plus-one learning for advanced learners, and homogeneous classrooms in which teachers regularly planned for plus-one learning.
In the two decades since those studies, I’ve observed and studied schools in which the entire faculty focused on providing a third condition: differentiation in mixed-ability classrooms where regular planning for a full spectrum of learners—including advanced learners—was a given.
It seems to me that a middle-class neighborhood with low-income families integrated within it is, in many ways, similar to the kind of classroom Tomlinson has observed — where, by force of numbers alone, a “plus-one” environment naturally occurs.
And, according to the Chetty study, what are the results of this kind of mixed-ability community for the young people who are in a more “advanced” position when families with more challenges move in?
(Thanks to Derek Thompson for bringing attention to that section of the Chetty study)
So, what do you think – am I as guilty of exaggerating the implications of this study as I have accused Chetty of being about his previous research?
Well, Slate has just published a piece from The Hechinger Report headlined, Pay Attention, Robot, which is about students teaching…robots.
It sounds like ed tech run amok, and it even suggests these teachable robots will be commonplace in the classroom within ten years. It cites some studies that supposedly highlight their effectiveness, but they don’t seem to compare the effectiveness of students teaching robots with, I don’t know, let’s say…students teaching their classmates? And they don’t seem to consider other benefits of people teaching people — like the development of intrinsic motivation (relatedness is a key element of that kind of motivation flourishing) and the refining of “people” skills.
But, disregarding the ridiculousness of these robots in all of our classrooms, the article does an excellent job describing the benefits that students gain by teaching others, and that’s why I’m taking the time to write this post. And that’s the reason I’m adding this post to my previously-mentioned “Best” list.