Gallup just came out with their annual Gallup Student Poll and, at first glance, it looks pretty depressing. But, that’s only at first glance:

Their public announcement it says, “Gallup research strongly suggests that the longer students stay in school, the less engaged they become.”

Their public announcement, however, does not include what they put at the end of their official report:

Gallup Student Poll Methodology

The annual Gallup Student Poll is offered at no cost to public schools and districts in the United States. The online poll is completed by a convenience sample of schools and districts each fall. Schools participating in the annual Gallup Student Poll are not randomly selected and are neither charged nor given any incentives beyond receipt of school-specific data.

Participation rates vary by school. The poll is conducted during a designated survey period and available during school hours Tuesday through Friday only. The Gallup Student Poll is administered to students in grades 5 through 12. The primary application of the Gallup Student Poll is as a measure of non-cognitive metrics that predicts student success in academic and other youth development settings.

Limitations of Convenience Sample Polling

The overall data from the annual administration of the Gallup Student Poll does not reflect responses from a nationally representative sample of students, and the overall data are not statistically weighted to reflect the U.S. student population; thereby, overall data and scorecards cannot be used by local schools and districts as a fit data comparison. School and district data and scorecards provide meaningful data for local comparisons and may inform strategic initiatives and programming, though the results are not generalizable beyond the universe of the participating school or district.

Here are the questions they used to determine engagement:

8. I have a best friend at school.
9. I feel safe in this school.
10. My teachers make me feel my schoolwork is important.
11. At this school, I have the opportunity to do what I do best every day.
12. In the last seven days, I have received recognition or praise for doing good schoolwork.
13. My school is committed to building the strengths of each student.
14. In the last month, I volunteered my time to help others.

So, it’s an “opt-in” survey with questionable statistical validity. And I’m not that sure that all those questions are the best ones to measure engagement, though some seem like they would work.

Listen, I agree that student engagement is an issue. I just don’t think I’d use these survey results to back-up my position.

What do you think? And what questions do you think would be good to use for determining engagement?

Thanks to Dan Pink for the tip.