Gallup just came out with their annual Gallup Student Poll and, at first glance, it looks pretty depressing. But, that’s only at first glance:
Their public announcement it says, “Gallup research strongly suggests that the longer students stay in school, the less engaged they become.”
Their public announcement, however, does not include what they put at the end of their official report:
Gallup Student Poll Methodology
The annual Gallup Student Poll is offered at no cost to public schools and districts in the United States. The online poll is completed by a convenience sample of schools and districts each fall. Schools participating in the annual Gallup Student Poll are not randomly selected and are neither charged nor given any incentives beyond receipt of school-specific data.
Participation rates vary by school. The poll is conducted during a designated survey period and available during school hours Tuesday through Friday only. The Gallup Student Poll is administered to students in grades 5 through 12. The primary application of the Gallup Student Poll is as a measure of non-cognitive metrics that predicts student success in academic and other youth development settings.
Limitations of Convenience Sample Polling
The overall data from the annual administration of the Gallup Student Poll does not reflect responses from a nationally representative sample of students, and the overall data are not statistically weighted to reflect the U.S. student population; thereby, overall data and scorecards cannot be used by local schools and districts as a fit data comparison. School and district data and scorecards provide meaningful data for local comparisons and may inform strategic initiatives and programming, though the results are not generalizable beyond the universe of the participating school or district.
Here are the questions they used to determine engagement:
8. I have a best friend at school.
9. I feel safe in this school.
10. My teachers make me feel my schoolwork is important.
11. At this school, I have the opportunity to do what I do best every day.
12. In the last seven days, I have received recognition or praise for doing good schoolwork.
13. My school is committed to building the strengths of each student.
14. In the last month, I volunteered my time to help others.
So, it’s an “opt-in” survey with questionable statistical validity. And I’m not that sure that all those questions are the best ones to measure engagement, though some seem like they would work.
Listen, I agree that student engagement is an issue. I just don’t think I’d use these survey results to back-up my position.
What do you think? And what questions do you think would be good to use for determining engagement?
Thanks to Dan Pink for the tip.
Those questions mirror the ones Gallup uses in its overall research on engagement, but I wonder if they’ve also validated them for school populations.
– I enjoy talking about school when I’m not in school
– I look forward to Monday mornings
– I look forward to telling my parents what happened in school
–
Larry,
While I appreciate and expect some level of cynicism surrounding this survey-I would encourage you to dig a bit deeper in the behavioral economics (school, student and teacher engagement model/science) Gallup has developed over four decades. It is an opt in by district/schools but all students participate and as a “believer” and one who has seen this work in action with all school constituent-I have yet to see an alternative that is as inclusive to the behavioral economics of schools. Can you imagine a different conversations with all constituents? People just are not as aware of the work Gallup has done over time with schools. Did your district chose to participate?
Debbie, No, our district doesn’t participate. There may not be a better tool out there but, nevertheless, being an “opt-in” clearly skews its validity.