Be sure to check-out The Shanker Blog’s post on this same topic
In listening to the trio of Gingrich, Sharpton and Duncan on Meet The Press today, one of the things that struck me was this videotaped piece from Bruce Stewart, formerly the head of the private Sidwell Friends School in Washington, D.C.:
MR. BRUCE STEWART: When I began teaching in the ’60s, we had that population of people. And since then, because greater opportunities have opened up for young women and for minorities, there’s been a great brain drain from American schools. I think we want to get those people back. If you look at Singapore, look at Finland, the reason they consistently are testing their population of students in the top levels of international exams, it’s the quality of their teaching force. They all come from the top third of their colleges, universities. In the United States, our tendency today is to have that pool of teachers coming from the bottom third of college and universities and from the bottom third of those classes. That’s something we need to reverse and to change.
I’ve heard this kind of statistic about teachers coming from the bottom third of something or other before (though never about the bottom third of classes — I don’t know where he got that bizarre statistic from), and just ignored it. But hearing it on Meet The Press, from the director of a private school, got “my dander up” and I decided to look into where those numbers came from and how valid and reliable they were. It was quite a ride on a Sunday afternoon….
Here is what I found…
Tons of people use a McKinsey report as the reference for the statistic of teachers coming from the bottom third of colleges. That report just uses a quote saying that:
“We are now recruiting our teachers from the bottom third of high school students going to college…” (p. 19)
It uses as its citation “Tough Choices Or Tough Times” , a report issued by The New Commission On The Skills Of The American Workforce in 2007.
So I went there. The link in the preceding paragraph only leads to a downloadable summary, which just stated the same statistic with no citation of a source. So, I went to Amazon, downloaded a Kindle Reader for my PC, and purchased the whole report.
That report uses as its source a “Report From The Department Of Education, National Center For Education Statistics, The Condition Of Education 2002.” It quotes the report as saying:
“A report by the National Council on Teacher Quality in 2004 said that the profession attracts a ‘disproportionately high number of candidates from the lower end of the distribution of academic ability.’ And, college graduates whose SAT or ACT scores were in the bottom quartile were more than twice as likely as those in the top quartile to have majored in education.”
Well, I couldn’t find that exact quote (but admittedly, I was getting a bit punch drunk by that time and might just have missed it) in the Condition of Education 2002, though page 91 has a lot of mathematical discussions of this topic, little of which I could understand (perhaps a math teacher can take a look?). I also found it interesting that I couldn’t find any other Condition of Education reports (they’re issued every year) that examine that topic.
However, I did find information on the National Council on Teacher Quality report that was quoted (which also based its critique on SAT and ACT scores), including criticism of its methodology — it apparently only included a portion of people who were going to be teachers. In fact, it excluded that portion who typically score the highest on SAT (the link takes to you a NY Times article about it that gives details). Also, ironically, in the same year, the same National Council on Teacher Quality came out with another report basically dismissing SAT scores as a valid and reliable predictor of teacher effectiveness, saying:
“…measurable teacher attributes like SAT scores…account for only a small portion of why some teachers are more effective than others.” (p. 10)
So, after all that, what are my conclusions?
First, I’d love to find out where the Sidwell guy came-up with his numbers, since they seem to be flat-out wrong.
Second, I’d love for a math person to examine the numbers on page 91 of the report on the Condition of Education 2002 to tell me what it really says in plain English.
Third, based on what I read of the criticism of the National Council of Teacher Quality report, this “bottom one-third” number also appears to be flat-out wrong.
And fourth, even if their numbers were right (which they don’t appear to be), it’s all much ado about nothing because they themselves say it’s not a reliable predictor of teacher effectiveness.
In other words, this bottom-third thing does seem to me to be a bunch of baloney.
What do you think? Let me know if I’m right or wrong, please!
Larry,
I think you’re right. What I am wondering is did the reports you’re citing account only for teachers who went directly from college to the classroom or did it account for those who are teaching as a second or subsequent career?
Richard
Richard,
From what I saw, they only accounted for people going directly to the classroom. That’s a good point, and another reason why the research isn’t valid or reliable.
Larry
I do not think that the people who write about “Who the teachers are and where they come from”, really have any credibility due to the fact that they have probably never been in the classroom standing in front of 120 different faces day in and day out. Good teachers have many different attributes and scoring high on the SATs is definitely not one that I would rate high on my list of predictors for making a good teacher. In 1976 I scored less than 1000 and I know that I am an innovative science teacher who motivates, collaborates with, and challenges the middle school students that I see every year. Good teachers are innovative, think outside of the box, know the how to learn from failure, connect with people, and are continuous learners. Good teachers look to the future, instigate conversations, and are willing to let their students guide them when appropriate. Until we ask teachers and students in the classroom what makes a good teacher we should not be concerned about the statistics. Plus I want to say kudos to you. Because I read your blog I have been able to recognize that even though I am alone in my district as as 21st century educator at least I know that I am on the right track and have been so since 1981. I am even going to teach a class at the U of O next term to pre-service teachers on technology integration in the classroom. I cannot wait to show them one of my favorite must reads everyday.
I agree with what Marilyn said about the credibility issue. If they have never been inside a classroom as a teacher, they really don’t know what good teachers do every single day. I realize that standardized test scores are the measure due to money issues, but I think that all policy makers should spend time in classrooms. Then maybe they will see that they cannot judge the effectiveness of teachers and success of students on one or two test scores. They should look at what happens in classrooms daily.
Thank you Larry – this is a nice piece of work, and deserves to be shared.
I think there is still a “those who can’t, teach” mentality in education. Your research helps refute that notion.
I’m linking to this story, thank you!
I agree with the credibility issue – similar to all these educational consultants who come in to schools to tell us what we are doing wrong, yet they only spent a year or two in the classroom.
I also agree that there is NO actual data or proof to support their statements – I could not find anything, anywhere to support those claims.
There are a lot of teachers in my district, myself included, who are second career teachers and went to top notch schools and were top of their classes.
Many of my colleagues who went right into teaching from college also went to good schools and were in the top end of their classes.
A state school nearby, Southern Connecticut State University, is a huge producer of teachers. It stared out as a teacher prep college. It is not easy. My wife is finishing her Biology Education degree there and it is tough. Many of the professors teach at or have taught at Ivy league schools, including Yale, and they say it is as difficult at Southern as anywhere else. You have to have a high GPA in your major just to get accepted into the school of education. You have to maintain a certain GPA to get approved for student teaching. You have to pass your Praxis II (teacher certification exams) before you can even student teach. That doesn’t sound like an easy college or low level students.
I always take any “report” on education with a grain of salt. They never seem to have real numbers or data, are written by non-educators, and the sample sizes are small. In science, we would call that a very poor experiment with useless data.
I thought it might be helpful if I cleared up some of the confusion over statements attributed to NCTQ in this blog.
Just as a practical matter, NCTQ is not the originator of any research on teacher quality. We just pass along information from research when we get it so when we might present research that reaches different conclusions. We don’t avoid reporting on research that doesn’t jive with our view on teaching–our job is to present a fair and balanced view on what the research says (and doesn’t say). That’s also why it is important not just to cite one study as the reason to change a policy, but to look at a body of research and see what its overall conclusions are.
The 2004 NCTQ report that you cite is a compendium of research on the attributes of teachers having an impact on student achievement. It reviews all of the research to date on teacher attributes and helps policymakers identify the trends/patterns on what matters.
So what do we know about SAT/ACT scores? They do matter somewhat, more than other measures like certification or whether a teacher holds a master’s degree. That doesn’t mean that a teacher who didn’t do all that well on the SAT or ACT couldn’t be a great teacher. The findings just look at patterns. There would always be false negatives and false positives. All it means is that if you are a teacher recruiter, you would want to pay attention to SAT/ACT scores but that doesn’t mean you can’t hire someone because other factors balance out a low score.
To the extent that SAT/ACTs measure intelligence, being smart is generally a benefit, a value-added. We know that schools that are high performing tend to have teachers with higher SAT/ACT scores. We know that National Board teachers have higher SAT/ACT score on average than teachers who aren’t National Board. However, are SAT/ACT scores what matter the most? No. Things we can’t measure about teachers are what matter the most.
I hope this is helpful.
Kate Walsh
President
National Council on Teacher Quality
Kate,
Thanks for taking the time to contribute to this discussion.
Admittedly, I’m not a trained researcher. However, after reviewing the study, and reading the criticism of its methodology in the New York Times article, I still remain a bit of skeptic about the use of SAT scores as any kind of reliable indicator of teacher performance.
I would be interested, though, to hear your comments on the quote from that report that is being relatively widely used:
That teaching attracts a ‘disproportionately high number of candidates from the lower end of the distribution of academic ability.’
Larry
Hi, Larry –
I am a math tutal in a 3-5 elementary school, so I thought I’d peek at the report and see if I could discern anything from the data discussion.
The first thing that struck me is the phrase “Using SAT and ACT scores as a proxy for academic caliber…” This indicates to me that the authors knew this is kind of a first cut on how to objectively rank teachers’ academic skill. As Kate W. indicates, I don’t think they meant it to be a definitive judgement on teachers’ intelligence.
The problem has been what the greater audience has done with that information. They’ve turned it into a definitive “Teacher’s Intelligence”, completely negating the fact that teachers’ skill is not about succeeding on the SAT or ACT. I’ve had very intelligent, terrible teachers and inspiring teachers who were not the sharpest knife in the block, as I’m sure most of us have.
That said, they do make general statements about how SAT scores are more likely to appear – elementary v. secondary, free lunch v. private. They don’t make any judgement on how significant that is to teaching ability.
Finally, that last paragraph of data jargon has to be an over-boiled account of what was first written. It’s too condensed to make meaningful sense, as far as I could tell. They seem to say that secondary school teachers in the top half of SAT scores mirror the general college graduate population, but that doesn’t make sense. If they are in the top half, they can’t mirror the whole population.
I do think it’s OK to work through the concept of where our teachers fall academically, and does that have an impact on student learning. It’s just a shame that they looked at one numerical indicator to make these judgements, especially considering the populations out there willing to skewer the teachers.
I had a family member who used to complain about the “liberal school environment and all those liberal teachers”. My jibe back was, “Well, [Nameless], what self-respecting, conservative capitalist do you know that would be willing to work in those rougher schools for so little money? Maybe if we paid teachers more, we’d have more Republicans and you’d have nothing to complain about.” He’d just laugh and we’d go on to other discussions.
It saddens me that instead of focusing time and attention on improving schools, for the benefit of our students, we slam the teachers. Apparently all educational ills can now be explained with “well, the teachers aren’t very bright.” It’s offensive and counter-productive.
I wish that instead of spending time chatting about what is wrong with schools, those with voice would spend time improving the schools. Visiting struggling schools as observers, not as guests, might give them more insight into how to amend our educational system.
Larry,
I’ll add an old post of mine to the discussion:
http://www.dangerouslyirrelevant.org/2008/05/low-ability-tea.html
I’d also like to caution folks against making ad hominem critiques of larger-scale statistical findings. We always can find individual exceptions. But we usually have to make policy from more general, overall trends…
Scott,
Thanks for sharing the link to your post on this topic.
After reading it, I still wonder about a couple of things:
Do the studies you cite omit the same large number of teachers who took those tests that the NCTQ report did?
Is it accurate to say that scoring high on those tests equals being “smart”?
Larry
Larry,
Re: question 1, I don’t know without digging them up and reading through them again.
Re: question 2, you tell me… =) Lots of people and organizations think so, including colleges. That’s a discussion in and of itself, isn’t it, about whether or not folks agree with typical definitions of ‘smart,’ whether there are ‘multiple intelligences,’ etc.
This is how I read Page 91, I hope it helps: It compares members of the college graduating class of 1993. (Using the “Baccalaureate and Beyond Longitudinal Study” – Note 8 elaborates on this study but doesn’t indicate sample size or describe how the sample was generated.)
Specifically it compares graduates with the top and bottom quartiles of SAT or ACT scores. Sorry if I’m being needlessly pedantic, but quartile basically means “quarter of the population.” If they ranked the SAT/ACT scores of 4000 graduates, 1000 are in the top quartile and 1000 are in the bottom quartile.
Looking only at the top quartile students, 10% taught at some point before 1997. 68% of “those who became teachers” were still working as teachers in 1997. (But it’s not clear if they mean 68% of the 10% who had done some teaching – I don’t think that’s precisely what they mean. I think they mean 68% of however many considered themselves teachers when they interviewed them some time shortly after graduation.)
Looking only at the bottom quartile students, 14% taught at some point before 1997. 84% of “those who became teachers” were still working as teachers in 1997.
Well I went to school at Berkeley for my undergraduate degree in History. Then I worked for a while. I went to graduate school later at a small school in the Bay Area called Holy Names University. They had a program of classes at night that was flexible for working adults and a philosophy about education specifically urban education that I liked.
Berkeley is obviously not in the bottom 3rd of schools, but it is possible that Holy Names University is, I actually have no idea nor do I care. For the purposes of these kinds of reports, would I be classified as someone who came from the bottom 3rd of the schools because my education degree came from a bottom 3rd school?
I taught history when I began so would they consider that since my Berkeley degree was in history and I taught history I would be classified as from not a bottom 3rd school. Sorry to be so obtuse but I don’t see how anyone can be classified like this.
I taught in the US for 10 years and most people I taught with had complicated circumstances like this. I’m not sure I know anyone who went to university and then went right into teaching. I’m sure some people still do this but many DO NOT do this anymore.
For career changers we looked for an education school that had convenient times for classes if we were already working not the reputation of the school. All teachers will tell you, when you enter the classroom the first time you have to relearn everything anyway so the education school too has a diminishing role in terms of importance. So many of us who went to excellent schools at first, maybe chose less “reputable” schools just to get our certifications and basic training. There are even districts running their own certification programs in conjunction with universities.
For me what made my life early on was the support I received from specified mentor teachers or informal mentoring. This means that just about anyone, no matter what school they went to or what their scores on the SAT were, can potentially improve and become an excellent teacher, if the district has some form of mentoring in place. Ahem, there is my pet peeve.
I am very smart and academically rigorous and I only consider myself to have been a good teacher. There were others far less “intelligent” who went to lesser schools, who in my opinion were excellent teachers, much better than me. If I saw it only once or twice I would say it is anecdotal, but since I saw it often and other honest teachers often admitted the same, there has to be a lot of validity to it. There are some teacher qualities that are unmeasurable by tests but that are EXTREMELY important.
I have the unique experience now to be in a Masters program in France, IT Management. All my classes are taught by researchers. They are subject matter experts and all studied at excellent universities and work in the research institutue located within the university. According to this way of thinking they should be excellent teachers, but they are not. They are by and large the worst teachers I have ever had. They have no concept of pedagogical concerns. I think they believe that by reading their powerpoint slides they are teaching. There is no interaction with the students and no time for questions. I would take a teacher any day who didn’t go to the best school and wasn’t the most intelligent person in their field, but who knew how to break down learning into digestible chunks, how to monitor student learning to speed up or slow down as needed, posed questions, stimulated discussions. These teachers need to stay in their research labs, I’m sure they are excellent at what they do, but it is clear they are not “teachers”.
Sorry for the rant but I just came home from a very frustrating class and I have been thinking of this exact topic. It made sense in my head, not sure what spilled out onto the paper does though.
I really got to put together a good post about all this….
I’m encouraged by this! Really!
It does NOT take a smart person as our “ruler” measure and declare. This is one of the most fossilized and archaic myths there be.
What it takes to be a good teacher IS that one see’s knowledge/knowing/understanding as important. Damn the rest. Fact is, most who see this as important, see the futility and meaninglessness of most college/university courses.
We can even go further, with recent studies concluding that it actual/statistically hurts a teacher to have gone through teacher’s college!
Stay tuned – I’m going to write a paper about this one…. in praise of the teacher as a “regular joe”!
David
http://eflclassroom.com
PS. I’ll just add the ending jab of – “what have all those smart grads brought us anyways? Atomic bombs, financial meltdown, foreign military “adventurism”, a dying planet…. God – let Forrest Gump be my teacher!
This has been a fascinating discussion to follow! What I find most interesting is that few have commented on the fact that there are reasons that people don’t choose to teach, these “findings” being a case in point.
What intelligent person would choose to go into a field where such little regard is given to the time, effort, sweat, and tears that goes into daily teaching? Obviously more than anyone can imagine after reading your intellectual dialogue.
I have been at this for 33 years and choose to remain because I know I have a lot to offer and I love my students, but if I wasn’t totally convinced of my calling, being beaten up by every newspaper, politician, and parent would certainly give me good reason to run from teaching at the first opportunity. This doesn’t even take into account that we are not paid well. Yet it isn’t about the money, but it is about lack of respect by those who have NO idea what it is like to be in a classroom with children who all need individual attention and deserve the best opportunities we have to offer. Until the very same people who sit in the chairs on Sunday morning spend a week in my classroom with my schedule, my students, my curriculum, my planning time, and my paycheck, they need to stop blaming teachers and making sweeping assumptions about the quality of educators in our schools.
I saw this episode of Meet the Press and was so worked up I changed an entire grad class paper to address it. In my view it falls in line with all the other elitist memes for privatization of schools with teachers coming only from the Ivy League Teach for America corps. It also puts at odds their case for alternative paths to teaching. Surely some of the second career teachers didn’t have such hot SAT/ACT scores or even great grade point averages. Yet through the power of a life well lived and varied and interesting world experiences they are the prized teacher candidate, as they should be. We might as well all leave the classrooms now. How can I continue to encourage the high school and middle school students who, for whatever reason, have let their grades tank yet are capable, creative, and often brilliant students in unconventional ways? How can I teach them there is time for redemption with hard work? What about the students whose learning differences made school that much more difficult and the perfect grade and high SAT/ACT scores elusive yet have persevered against many obstacles to graduate from high school and go on to get their college degree? Many of these students are intuitive teachers and having worked closely with amazing and excellent teachers chose to give back and become teachers – dedicated, brilliant teachers. What quoting this statistic really means is that we don’t want to spend too much time looking at the whole student, the whole experience they might bring. It is easier to look at pedigree and scores. I would like our teacher corps to reflect the diversity and experiences in our student populations and that will include very high performing individuals as well as those who know intimately the struggle many of our students are facing. To quote the statistic is really a reflection of not having been in a classroom in a public school for any meaningful amount of time or experience. SAT/ACT scores are not a measure of “intelligence”; maybe of “smart”. How does this “smart” measure up to the actual “knowledge” your students have and need to know to navigate in their world?
I’ve always been doubtful of statistics!! Torture numbers, and they’ll confess to anything. Keep working and don’t bother about numbers!!!!
Page 91 of the report appears, by all accounts, to be a standard data exploration which has been heavily simplified for a lay audience. I’m no statistical expert myself — I only ever got to the second semester of it in college — but even I’m a little horrified that you’d characterize it as “a lot of mathematical discussions of this topic, little of which I could understand.”
1.) Sidwell’s Mr. Stewart’s claim probably comes across too strong, but *is* consistent with the COE 2002 report.
2.) The COE report (your “page 91”) is already heavily simplified, but let me try further: it explains essentially that students from the bottom 25% of college students (as measured by SAT scores) are somewhat more likely to become teachers than college students from the top 25%. The difference is noticeable, but not staggeringly so.
3.) The NY Times analysis you link to is discussing another part of the NCTQ report and *definitely does not apply* to the COE analysis on the report’s page 91 (page 115 of the *.pdf). I can explain more if you like.
4.) The fact that something isn’t a *perfect* predictor of teacher effectiveness doesn’t mean it’s not useful. In general, people who play in the NBA tend to be taller than those who don’t. Similarly, in general, students with higher SAT scores tend to be better teachers than students with low SAT scores.
Are there exceptions? Definitely. There are many excellent teachers with low SAT scores, and many poor ones with high SAT scores.
But the fact that we’re pulling teachers disproportionately from the bottom 25% means that, on balance, we’re doing worse than we would if we drew them from the top 25%.
You’re definitely right about the fourth point, which might be the most important. Even the McKinsey report that argues for teachers coming from the top third suggests there’s nothing to back that up.
I had a really long conversation about this last week with someone. This person, who was a public defender in the Bronx for 10 years before getting into education, couldn’t believe this, because it flew in the face of everything she had ever we experienced. And while she was willing to work through the cognitive dissidence she experienced, many cannot or will not do this. It’s why so many current myths (class size doesn’t matter, testing tells you something of value, competition is good, merit pay works) in education are so powerful: they appeal to the relatively small percentage of people who benefitted from traditional education structures and capitalist economic structures.
I always heard from my colleagues in my field that “those who cannot do, teach.” This was in response to a question someone asked as to why many professionals in some high paying fields choose instead to teach the skills in some college classroom rather than use them.