Larry Ferlazzo’s Websites of the Day…

…For Teaching ELL, ESL, & EFL

Measurement Matters….Maybe Not So Much

| 1 Comment

You may have heard about the article released today that was written by Angela Duckworth (of “grit” fame) and David Scott Yeager (a researcher of “growth mindsets.” It’s titled “Measurement Matters:Assessing Personal Qualities Other Than Cognitive Ability for Educational Purposes.”

I thought I’d share about the good and the bad in it.

Here’s their official summary.

You can also read articles about it at The Washington Post and in Education Week.

Here’s my summary and “take” on the article and its topics:

MY SUMMARY:

They basically say that it is important to develop a common and better term for the range of Social Emotional Learning skills like self-control, grit, growth mindset, etc. — and to particularly reconsider the term “non-cognitive skills.” In addition, they say that the present instruments being used to measure these qualities are imperfect and should not be used for high-stakes decision-making related to students, teachers or schools, and that research should be done to determine better forms of measurement.

MY COMMENTARY

It’s great news that they are saying that SEL measurements should not be used for any kind of high-stakes decision-making. I wonder if that means that Ms. Duckworth is stepping away from her research last year suggesting that Districts could determine grit scores from the resumes of teacher candidates and using that information in their hiring decisions (see This Has Me Concerned: “Study Links Teacher ‘Grit’ with Effectiveness, Retention”). I’ve sent that question into her institute and hope to have a response to share soon (Professor Duckworth graciously responded within hours.  You can see it at the end of this post).

It’s not so great news that they seem to say what’s really needed in order to start using SEL skills in high-stakes decision-making is just a better way to measure those SEL skills.

I made my case about why teachers shouldn’t be measuring individual students SEL skills in a Washington Post column titled Why schools should not grade character traits.

I think that entire column is applicable to the issues raised in today’s article, and here’s just one excerpt (there’s some irony in my sharing it here because Duckworth and Yaeger also use the same quote in their paper):

“Not everything that counts can be counted, and not everything that can be counted counts.”

The quote is sometimes (erroneously, it seems) attributed to Albert Einstein. And that’s probably because we tend to think of Einstein as a sensible fellow who understood the limits of quantification. I think we might add a corollary to Cameron’s observation: Going through tortuous gyrations in order to count something doesn’t mean it should be counted.

Maybe I’m naive or unsophisticated (and I’m happy to hear comments suggesting both), but I just don’t see the need to officially measure everything that students do or don’t do.

It appears to me that there is sufficient research out there, and enough direct teacher experience validating it, to demonstrate that there is value to teaching SEL skills (though, as I’ve regularly said, they have their place, but also have to be kept in their place).

And other researchers have used what appears to me appropriate indicators of how successful schools are being in helping students develop them, including through measuring “student absences, suspensions, grades, and grade progression.”

Of course, because of factors beyond teachers’ control (and there are many of them), we need to be careful that even these kinds of numbers are used to ensure that we are using them to be data-informed and not data-driven.

I think a much more important issue for public discussion around issues of grit, self-control and growth mindset relate to The Manipulation Of Social Emotional Learning by those who seek to use it as a “Let Them Eat Character” replacement for needed policy solutions.

What are your thoughts on the new Duckworth/Yaeger article and the issues it raises, as well as on my commentary about it?

ADDENDUM

Response From Angela Duckworth:

I did write a paper that encouraged scrutinizing an applicant’s “track record” of grit (e.g., in college sports or public service or anything else where they could show up, or not, over multiple years and really apply themselves). The thing I like about this measurement approach to grit is that it is, in theory, less fakeable than a self-report questionnaire and also not as susceptible to reference bias, because you do not have to ask the applicant to rate themselves. However, it is like any other measurement approach limited and imperfect. Here is a paragraph from the paper:

“We suggest that school administrators consider grit as one factor—
among many—in identifying promising new teachers. While no single factor in isolation should determine a hiring decision, the method for quantifying grit from biographical data developed for this investigation represents a practical tool for predicting success in the first few years in teaching. Despite its predictive validity, policymakers should proceed cautiously when using this measure of grit during the screening process and continue to consider a wide range of variables, not just those that are easy to measure, when making hiring decisions. In addition, before using these measures for high-stakes purposes, districts should conduct their own internal validation studies to ensure grit is predictive of valued teaching outcomes in their sample of teachers.”

And here is the paper in its entirety

So, I hope the message that comes through is that all measures are imperfect, each measure is imperfect in its own way, and we need to recognize the limitations and advantages of any measure when we think about specific purposes to which these measures would be applied.

Author: Larry Ferlazzo

I'm a high school teacher in Sacramento, CA.

One Comment

  1. Dr. Duckworth’s rational sounding response might best be read through the filter of educators’ own experiential knowledge of how cautionary language is heard (or more often ignored) in daily applications of “data.” One thinks of those prescription drug ads on TV. The fast, low voice is saying the data shows that this drug might kill and maim some people but the only thing many hear is: “This could make my life easier.”

Leave a Reply

Required fields are marked *.


Skip to toolbar