I was re-reading an article that Malcolm Gladwell wrote a couple of years ago titled Open Secrets, which has some intriguing pieces in it about the differences between a “puzzle” and a “mystery.” He gives credit to security expert Gregory F. Treverton for initially defining the difference, and Treverton wrote about it in Smithsonian Magazine (Risks and Riddles).

I’m wondering if this distinction might help inform the ongoing debates about “school reform.”

Let me first share some quotes from Gladwell:

The national-security expert Gregory Treverton has famously made a distinction between puzzles and mysteries. Osama bin Laden’s whereabouts are a puzzle. We can’t find him because we don’t have enough information. The key to the puzzle will probably come from someone close to bin Laden, and until we can find that source bin Laden will remain at large.

The problem of what would happen in Iraq after the toppling of Saddam Hussein was, by contrast, a mystery. It wasn’t a question that had a simple, factual answer. Mysteries require judgments and the assessment of uncertainty, and the hard part is not that we have too little information but that we have too much. The C.I.A. had a position on what a post-invasion Iraq would look like, and so did the Pentagon and the State Department and Colin Powell and Dick Cheney and any number of political scientists and journalists and think-tank fellows. For that matter, so did every cabdriver in Baghdad.

The distinction is not trivial. If you consider the motivation and methods behind the attacks of September 11th to be mainly a puzzle, for instance, then the logical response is to increase the collection of intelligence, recruit more spies, add to the volume of information we have about Al Qaeda. If you consider September 11th a mystery, though, you’d have to wonder whether adding to the volume of information will only make things worse. You’d want to improve the analysis within the intelligence community; you’d want more thoughtful and skeptical people with the skills to look more closely at what we already know about Al Qaeda. You’d want to send the counterterrorism team from the C.I.A. on a golfing trip twice a month with the counterterrorism teams from the F.B.I. and the N.S.A. and the Defense Department, so they could get to know one another and compare notes.

Here are some passages from Treverton’s piece:

A mystery cannot be answered; it can only be framed, by identifying the critical factors and applying some sense of how they have interacted in the past and might interact in the future. A mystery is an attempt to define ambiguities.

Puzzles may be more satisfying, but the world increasingly offers us mysteries.

Puzzle-solving is frustrated by a lack of information. By contrast, mysteries often grow out of too much information.

I’m not entirely convinced it has to be an either/or perspective and, in fact, think it can be damaging to look at any situation as black or white. But I wonder if worshiping at the alter of school data, as some appear to do, tilts us too far in the direction of looking at how to improve schools (and their teachers, administrators, and students) as a puzzle with primarily technical solutions, including daily prescribed curriculum, all-powerful standardized tests, and national standards.

Instead, I wonder if it would be better if we look at more as a mystery where data has its place, but it also has to be kept in its place. I’ve written about the distinction between being data-driven and being data-informed. It seems to me that being data-informed, recognizing that education might be a place where one size does not fit all, and encouraging school staff, parents and students to use their judgment and develop collaborative action might be a better direction in which to go.

What do you think? Does the puzzle/mystery framework hold any validity from your perspective?