I have posted numerous times about how I use student evaluations in my class, their results, and why I think it’s a terrible idea to connect them to the teacher evaluation process (see The Best Posts On Students Evaluating Classes (And Teachers) ).

And, of course, The Gates Foundation has come out squarely in support of doing just that in their Final MET “Effective Teaching” Report.

Now, however, their recommendations have gone from being bad towards approaching just plain weirdness.

In a piece titled Ask the Students, Thomas Kane, the director of the Gates project, is suggesting that now it’s important to start “aligning the language of the surveys with the language of the teaching standards.”

Yup, this practice, in line with the particularly useless directive that some teachers are given to write the standards that are supposed to be covered that day on the front whiteboard, is really going to give teachers helpful information about their craft. How about this question:

Was your teacher effective in helping you learn to interpret words and phrases as they are used in a text, including determining technical, connotative, and figurative meanings, and analyze how specific word choices shape meaning or tone?

Or how about:

Was your teacher effective in teaching you to integrate and evaluate content presented in diverse media and formats, including visually and quantitatively, as well as in words?

You can see my previous posts about the questions that I think are truly helpful — both to us teachers and to our students….

The Gates Foundation Measures Of Effective Teaching has just released its final report, Measures of Effective Teaching Project Releases Final Research Report. In addition, they have issued two other reports, Ensuring Fair and Reliable Measures of Effective Teaching: Culminating Findings from the MET Project’s Three-Year Study and Feedback for Better Teaching: Nine Principles for Using Measures of Effective Teaching — both available at the same link.

I started going through the report during my break at school today, but it’s not the sort of thing you can just glance through…

However, here are a couple of tweets from respected researcher Bruce Baker:

Shocking revelation: Gates study finds VAM is best predictor of itself! metproject.org/downloads/MET_… (therefore, must use now!)

— Bruce Baker (@SchlFinance101) January 8, 2013

Brilliant inferences: Gates study says teaching too complex to eval w/any 1 measure. BUT, all measures can be validated against VAM!

— Bruce Baker (@SchlFinance101) January 8, 2013

You might also be interested in The Best Posts On The Gates’ Funded Measures Of Effective Teaching Report, which is a collection of commentaries on their report from a year ago, including my own.

The Best Posts On Students Evaluating Classes (And Teachers) (look near the end of that post) also includes some posts specifically related to the report’s emphasis on using student surveys in the formal teacher evaluation process.

I’m sure I’ll be developing a similar collection for this one over the coming days….

Speaking of which, Ed Week has just come out with an article on the report.

The Washington Post also has an article…

Here’s coverage from Reuters.

The Wall Street Journal’s take on it.

The Gates Foundation is set to release the final report of its three-year project on teacher effectiveness on Tuesday.

You can read about the concerns that I, and many others, had about it’s mid-term report a year ago at The Best Posts On The Gates’ Funded Measures Of Effective Teaching Report.

You can find additional concerns specifically about its perspective on using student surveys in a formal teacher evaluation process at
The Best Posts On Students Evaluating Classes (And Teachers) (look near the end of that post).

And The National Education Writers Association published a decent introductory piece on it today.

I’m sure I’ll be posting more about the report once it comes out…

I’ve written a fair amount sharing my perspectives on the role of private foundations. I’ve had a lot of experience directly relating to them during my community organizing career, and some dealing with their role in education. You can see these resources in The Best Resources For Learning About The Role Of Private Foundations In Education Policy.

Felix Salmon, a columnist for Reuters, had already written what I think is one of the best pieces on the use of student surveys for teacher evaluation, and that piece can be found at The Best Articles, Videos & Posts On Education Policy In 2012 — Part Two.

Today, thanks to Ken Libby, I found that he has written another one that should be on that list –
Philanthropy: You’re doing it wrong.

It’s a must-read. Here’s how it ends:

Philanthropy has always been self-serving in large part, and that’s never going to end. But there’s no good reason why you should be part of the problem.

Thanks to Alexander Russo, I learned about this recent radio interview (if you’re reading this on an RSS Readers, I think you’ll have to click through to see the embedded audio player) with Ron Ferguson and Thomas Kane from the The Gates’ funded Measures Of Effective Teaching project (see The Best Posts On The Gates’ Funded Measures Of Effective Teaching Report).

As regular readers know, there are probably few teachers out there who have been as outspoken as I have been in advocating the use of student surveys (see The Best Posts On Students Evaluating Classes (And Teachers)).

And I have been equally strong in my criticism of the Gates’ effort to use them in formal teacher evaluations (you can see links to a number of critiques in that same “The Best…” list). Nothing I’ve written or, probably, anyone else has written, has put the case against their use in this way more eloquently than Felix Salmon at Forbes (see What education reformers did with student surveys).

In this interview, Kane and Ferguson appear to be living in a bubble immune from any of these legitimate concerns. To me, at least, it’s truly scary and make me even more worried about what the final results of the MET project are going to say. They appear to have little sense of how their research might practically play-out in schools across the country.

It reminds me of the dialogue I had with education researchers in my critique of a recent “loss aversion” study where some seemed to suggest that researchers should not have to necessarily be responsible for the how their research might be applied in the world.

If you can, take a few minutes to listen to the interview and let me know what you think.

Perhaps I’m over-reacting….

(Editor’s Note: As regular readers know, I’ve written a great deal about how I use student evaluations to improve my classes and my instruction (see The Best Posts On Students Evaluating Classes (And Teachers). I’ve invited Dr. John Thompson to add his thoughts on the topic. )

Guest Post by Dr. John Thompson

Education consultant Craig Jerald deserves at least two cheers for his analysis of the Gates Foundation’s recent reports on the use of student survey data to improve instruction. Jerald explained that Gates researchers do not anticipate students giving a summative grade for their teacher. “Instead, they are asked a series of carefully worded questions about their classroom experiences that measure specific kinds of instructional practices and classroom conditions that are conducive to student learning.”

Jerald challenged the loose wording of Republican presidential candidate Mitt Romney, who said, “I would love to have the students grade the teachers at the end of the year as opposed to just the other way around so that teachers get feedback” He then praised Amanda Ripley’s “engaging and informative” Atlantic article, “Why Kids Should Grade Teachers,” while adding, “Just skip the headline.”

Jerald argued that “nobody anywhere is really asking students to ‘grade teachers,’ and when journalists, pundits, and presidential candidates call it that, they risk undermining the very tool they seek to champion.” He worried that serious misperceptions about student surveys “could translate into a very serious problem when it comes time to ask even more teachers to buy into the process.”

The problem is that many of those misperceptions belong to persons pushing tougher evaluations and/or persons who will be conducting those evaluations. Even though the increased use of student perceptions is one of the best things about the Gates Measuring Effective Teaching (MET) process, it would still be a part of a risky scheme for evaluating teachers.  It would be nice if the student survey component (like the use of videotaping for professional development, as opposed to high-stakes purposes) could be a corrective to the inherently dangerous aspects of the MET.  It is just as likely, however, that the flaws in the rest of the MET’s policies will undermine the benefits of student surveys (just as videotaping for evaluations would likely compromise the constructive use of videotaping to improve instruction.)

In other words, “to a man with a hammer, every problem looks like a nail.” Similarly, during an era of teacher-bashing school “reform,” every piece of data can look like a grade.

It is easy to see why the Gates Foundation sought to add teeth to student survey data by making it a part of teachers’ evaluations. The Atlantic’s Ripley began with the story of a senior,Nubia Baptiste, who “could have revealed things about her school that no adult could have known.”  Ripley observed, “She knew which security guards to befriend and where to hide out to skip class (try the bleachers). She knew which teachers stayed late to write college recommendation letters for students; she knew which ones patrolled the halls like guards in a prison yard, barking at kids to disperse.” Even so, Nubia never had a chance to express her judgments about school until she filled out a survey during her senior year.”

Ripley also cited the disappointing experience of Ronald Ferguson in persuading teachers to pay attention to survey results.  Over a decade, “only a third of teachers even clicked on the link sent to their e-mail inboxes to see the results.”

I understand some sorts of recalcitrance on our part.  I would never criticize teachers for turning off their brains and sleep-walking through the normative professional development quick fixes that are dumped on us.  But, when our students speak, we must listen.  And, if we and policy wonks were to heed the wisdom of students, we would finally be held to standards that make sense.  For instance, the following are types of patterns that correlate with increased student performance.  They are fundamentally different than the narrowed test prep and scripted instruction that has been imposed by “reformers.”

1. Students in this class treat the teacher with respect.

2. My classmates behave the way my teacher wants them to.

3. Our class stays busy and doesn’t waste time.

4. In this class, we learn a lot almost every day.

5. In this class, we learn to correct our mistakes.

When offered a chance to build learning environments based the above principles, we should take “yes” for an answer. If given a chance to create school cultures based on those goals, should we not do whatever it takes to build on them?

We teachers have the right to resent the predisposition to apply stakes to all sorts of data, while acknowledging that we are not blameless.  If for no other reason than for respect for our students, teachers should commit to a counter-proposal.  We should embrace Ripley’s analysis of college student surveys.  “Decades of research,” she concluded, “indicate that the surveys are only as valuable as the questions they include, the care with which they are administered—and the professors’ reactions to them.”

In return for transforming student survey results into diagnostic data for school improvement, we should commit to making student surveys a prime metric for building respectful learning environments. After all, Ripley closed with the wisdom of a principal who benefited from surveys in a pilot program where the data was not linked to teachers’ names, but “he still found the information more useful than what standardized tests provided.” The principal said, “’It’s very, very precious data for me.’”

At the same time, the Gates Foundation should consider its own evidence and earn three hearty cheers.  The Foundation deserves one cheer for using value-added models, videotapes, and student surveys to identify effective teaching. It also deserves praise for using each of those measures to make the others more reliable.

The MET should take a bow for using multiple measures to make each of its efforts more valid.  Then, there would be no shame in admitting that it had discovered that those measures are not valid enough for high-stakes purposes.  The best service that the Gates Foundation could do for students would be to acknowledge that their evidence did not support their hypothesis that multiple measures should transform teacher evaluations, incentives, and professional development along the lines that it originally anticipated. On the other hand, the Foundation would remain committed to using each measure to improve teaching and learning.  Especially when studied along with videotapes of instruction, the use of study survey data could inform discussions that would be truly transformative.

John Thompson taught for 18 years in the inner city.  He blogs regularly at This Week in Education, Anthony Cody’s Living in Dialogue, the Huffington Post and Schools Matter.  He is completing a book, Getting Schooled, on his experiences in the Oklahoma City Public School System.

As regular readers know, I’ve been very outspoken in my support and use of student evaluations of teachers as formative assessments and outspoken in my criticism of efforts by the Gates Foundation to incorporate them in formal summative evaluations of teachers (you can see many of my posts on this topic at The Best Posts On Students Evaluating Classes (And Teachers) ).

Today in The Atlantic, Amanda Ripley wrote a feature titled Why Kids Should Grade Teachers, which parrots the typical school reformer line.

Also today, Felix Salmon at Reuters wrote a devastating critique of her article and the whole idea in What education reformers did with student surveys.

It’s clearly a candidate for best educational policy post of the year. Here are some excerpts, but the whole piece is a “must-read”:

….along comes the Gates Foundation with a 36-question survey, severely chopped from a much longer one developed by Ronald Ferguson. Since there are 36 questions, the survey essentially measures teachers along 36 different axes, all of which are aligned with each other to differing degrees. In and of itself, that’s more useful than just measuring test scores, which are much less teacher-specific and which only provide one axis of educational quality.

But then what do the reformers do? They regress the survey answers against test scores, look at which survey questions align most closely with that test-score axis, and declare that those axes — the ones which test scores, by definition, are already measuring — must be the “most important”. Did you think that caring about kids was of paramount importance? Silly you! It turns out that caring about kids isn’t as correlated with test-score results as, say, whether the class learns to correct its mistakes. And therefore, we shouldn’t be worrying as much about whether teachers care about their kids; we should be worrying more about other things, instead. That’s what the test-score regressions tell us, so it must be true!…..

No! Stop! Do none of these people get it? What everybody wants, here, is better teachers. These surveys could be instrumental in helping to improve teaching. Teachers would be able to see where they score well and where they score badly, and ask themselves how to improve their scores in areas where they are weak. Principals could see which teachers were good on which axes, and set classes up so that students ended up with a balanced range of teachers. And generally, everybody could treat this data as an interesting and very rich way of improving educational outcomes.

Instead, reformers are rushing to use this data as a quantitative performance-review tool, something which can get you a raise or which can even get you fired. And by so doing, they’re turning it from something potentially extremely useful, into a bone of contention between teachers and managers, and a metric to be gamed and maximized.

There’s a big article in The Atlantic this week that has been getting a fair amount of attention. It’s titled Why Kids Should Grade Teachers (it includes a relatively lengthy description of the process in a kindergarten classroom).

I’ve written a whole lot on this topic (see The Best Posts On Students Evaluating Classes (And Teachers) ).

There are few teachers who have advocated more for the use of student evaluations than me — but NOT for use in formal teacher evaluations.

I’ve written about my objections to this hijacking of a great tool by “school reformers” at I’ve Got A Bad Feeling About This: “Next Up in Teacher Evaluations: Student Surveys.”

Feedback is welcome….