The Truth Wears Off: Is there something wrong with the scientific method? by Jonah Lehrer is an exceptional article, and it was just released from behind The New Yorker pay-wall yesterday.
It reinforces why we need to be data-informed, but not data-driven — everywhere, including in schools.
David Brooks from The New York Times wrote a nice summary of the article:
He describes a class of antipsychotic drugs, whose effectiveness was demonstrated by several large clinical trials. But in a subsequent batch of studies, the therapeutic power of the drugs appeared to wane precipitously.
This is not an isolated case. “But now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain,” Lehrer writes. “It’s as if our facts were losing their truth: claims that have been enshrined in textbooks are suddenly unprovable.”
The world is fluid. Bias and randomness can creep in from all directions. For example, between 1966 and 1995 there were 47 acupuncture studies conducted in Japan, Taiwan and China, and they all found it to be an effective therapy. There were 94 studies in the U.S., Sweden and Britain, and only 56 percent showed benefits. The lesson is not to throw out studies, but to never underestimate the complexity of the world around.
It’s also a perfect article for Theory of Knowledge classes.