Larry Ferlazzo’s Websites of the Day…

…For Teaching ELL, ESL, & EFL

The Best Resources Showing Why We Need To Be “Data-Informed” & Not “Data-Driven”

| 10 Comments

'Marveling at the Data' photo (c) 2013, DoDEA - license: http://creativecommons.org/licenses/by/2.0/

Last year, two very talented educators — Ted Appel, the extraordinary principal we have at our school, and Kelly Young, creator of much of the engaging curriculum we use at our school through his Pebble Creek Labs — brought-up the same point in separate meetings with teachers at my school: The importance of not being “data-driven” and, instead, to be “data-informed.”

These conversations took place in the context of discussing the results of state standardized tests. Here’s the point made by Ted:

If schools are data-driven, they might make decisions like keeping students who are “borderline” between algebra and a higher-level of math in algebra so that they do well in the algebra state test. Or, in English, teachers might focus a lot of energy on teaching a “strand” that is heavy on the tests — even though it might not help the student become a life-long reader. In other words, the school can tend to focus on its institutional self-interest instead of what’s best for the students.

In schools that are data-informed, test results are just one more piece of information that can be helpful in determining future directions.

Since that conversation took place, I’ve written several posts about the topic. I thought it might be useful to bring together several related resources.

Here are my choices for The Best Resources Showing Why We Need To Be “Data-Informed” & Not “Data-Driven”:

First, I’m going to list the post I wrote immediately after that conversation – “Data-Driven” Versus “Data-Informed”

Next, a Dilbert cartoon that Alexander Russo shared on his blog:

Dilbert.com

The cartoon reminded of what the New York judge said earlier this month when he ruled that the School District can publicly release the names of teachers and their “Teacher Data Reports.” Here is what the judge said (and I kid you not):

“The UFT’s argument that the data reflected in the TDRs should not be released because the TDRs are so flawed and unreliable as to be subjective is without merit,” the judge wrote, citing legal precedent that “there is no requirement that data be reliable for it to be disclosed.”

Data-Driven…Off a Cliff is the title of an excellent post by Robert Pondiscio.

An article in Educational Leadership is a year-old, but it’s new to me and certainly worth sharing. It’s called The New Stupid, and has the subtitle “Educators have made great strides in using data. But danger lies ahead for those who misunderstand what data can and can’t do.” It’s written by Frederick M. Hess.

It’s an article worth reading (though I do have concerns about some of its points), and relates to what I’ve written about being “Data-Driven” Versus “Data-Informed.”

Here are a couple of excerpts:

…the key is not to retreat from data but to truly embrace the data by asking hard questions, considering organizational realities, and contemplating unintended consequences. Absent sensible restraint, it is not difficult to envision a raft of poor judgments governing staffing, operations, and instruction—all in the name of “data-driven decision making.”

and…

First, educators should be wary of allowing data or research to substitute for good judgment. When presented with persuasive findings or promising new programs, it is still vital to ask the simple questions: What are the presumed benefits of adopting this program or reform? What are the costs? How confident are we that the promised results are replicable? What contextual factors might complicate projections? Data-driven decision making does not simply require good data; it also requires good decisions.

The Truth Wears Off: Is there something wrong with the scientific method? by Jonah Lehrer is an exceptional article from The New Yorker. David Brooks from The New York Times wrote a nice summary of the article:

He describes a class of antipsychotic drugs, whose effectiveness was demonstrated by several large clinical trials. But in a subsequent batch of studies, the therapeutic power of the drugs appeared to wane precipitously.

This is not an isolated case. “But now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain,” Lehrer writes. “It’s as if our facts were losing their truth: claims that have been enshrined in textbooks are suddenly unprovable.”

 

 

The world is fluid. Bias and randomness can creep in from all directions. For example, between 1966 and 1995 there were 47 acupuncture studies conducted in Japan, Taiwan and China, and they all found it to be an effective therapy. There were 94 studies in the U.S., Sweden and Britain, and only 56 percent showed benefits. The lesson is not to throw out studies, but to never underestimate the complexity of the world around.

Talking To Students About Their Reading (& Their Data) is a post I’ve written.

“Using data for progress, not punishment”

In a Data-Heavy Society, Being Defined by the Numbers is by Alina Tugend at The New York Times.

Data-Driven Instruction and the Practice of Teaching is by Larry Cuban.

The Obituaries for Data-Driven ‘Reform’ Are Being Written is by John Thompson.

California Governor Puts the Testing Juggernaut On Ice is by Anthony Cody at Education Week.

Making the wrong “Data-Driven Decisions” is by Carl Anderson (thanks to Dean Shareski for the tip).

Data-Driven To Distraction appeared on Larry Cuban’s blog.

Larry Cuban has written another interesting post titled Jazz, Basketball, and Teacher Decision-making. John Thompson relates it to school data at Thompson: Duncan Can Shoot — But Can He Rebound?

“Not everything that matters can be measured”

“You Are Not An Equation” (And Neither Are Your Students)

Policy by Algorithm is a nice post over at Ed Week.

Professional Judgment: Beyond Data Worship is by Justin Baeder at Education Week.

This Is Why Our School is “Data-Informed” & Not “Data-Driven”

Bias toward Numbers in Judging Teaching is by Larry Cuban.

The False Allure Of Statistics is by John Thompson.

‘Moneyball’ and making schools better is by John Thompson.

Here’s Another Reason Why We Need To Be Data-Informed & Not Data-Driven

Data Gone Wild

“Why Do Good Policy Makers Use Bad Indicators?” is by Larry Cuban.

New Hope for the Obama/Gates School of Reform is by John Thompson.

“It’s amazing how much it’s possible to figure out by analyzing the various kinds of data I’ve kept,” Stephen Wolfram says. To which I say, “I’m looking at your data, and you know what’s amazing to me? How much of you is missing.”

This is the last paragraph of Robert Krulwich’s article at NPR, titled Mirror, Mirror On The Wall, Does The Data Tell It All? In it, he compares authors of books, one by Stephen Wolfram, creator of a the Wolfram search engine, and Bill Bryson, author of a biographical account of growing up in Iowa. The column, though not specifically about schools, hits a “bulls-eye” on our current data-driven madness.


What Does “Stop & Frisk” Have To Do With What’s Happening With Our Schools?

What Does The NYPD Have In Common With Many Data-Driven Schools?

Tired of the Tyranny of Data is by Dave Orphal.

Big Data Doesn’t Work if You Ignore the Small Things that Matter is from The Harvard Business Review.


Test Scores Often Misused In Policy Decisions
is from The Huffington Post.

The Data-Driven Education Movement
is from The Shanker Blog.

Data Overload

Invisible Data is from Stories From School.

Don’t Let Data Drive Your Dialogue is from The Canadian Education Association.

“The Goal Is The Goal”

On the Uses and Meaning of Data is by David B. Cohen.

Friday Thoughts on Data, Assessment & Informed Decision Making in Schools is from School Finance 101.

The New York Times Has Discovered The Perils Of Being Data-Driven — I Just Wish Arne Duncan Would, Too

Here’s a Part One and Part Two series of posts on the use of data in education, and they’re both from Larry Cuban’s blog.

Data: No deus ex machina is by Frederick M. Hess & Jal Mehta.

Bill Gates is naive, data is not objective is by Cathy O’Neil and is really good.

Bill Gates and the Cult of Measurement is by Anthony Cody.

Sure, Big Data Is Great. But So Is Intuition. is from The New York Times. Here’s an excerpt:

It’s encouraging that thoughtful data scientists like Ms. Perlich and Ms. Schutt recognize the limits and shortcomings of the Big Data technology that they are building. Listening to the data is important, they say, but so is experience and intuition. After all, what is intuition at its best but large amounts of data of all kinds filtered through a human brain rather than a math model?

At the M.I.T. conference, Ms. Schutt was asked what makes a good data scientist. Obviously, she replied, the requirements include computer science and math skills, but you also want someone who has a deep, wide-ranging curiosity, is innovative and is guided by experience as well as data.

“I don’t worship the machine,” she said.

Beware the Big Errors of ‘Big Data’ is from Wired.

The NYPD Probably Didn’t Stop All That Crime

Data-Informed Versus Data-Driven PLC Teams is from All Things PLC.

David Brooks, who generally loses all coherence when he writes explicitly about education issues, has just written an eloquent case for the importance of being data-informed, and not data-driven. Read his column titled What Data Can’t Do. Here’s an excerpt:

The Problem with Our Data Obsession is from MIT.

Data Without Context Tells a Misleading Story is from The New York Times.

Big Data is “not a replacement for the classic ways of understanding the world”

Quote Of The Day: “Data & data sets are not objective”

“Big (Dumb) Data” is by John Thompson.

Data are no good without theory is from The Washington Post.

The Perils of Economic Thinking about Human Behavior is from School Finance 101.

What You’ll Do Next is by David Brooks

At the risk of being accused of taking a “cheap shot,” I just can’t resist embedding two segments from The Colbert Show about the now well-known mistake by the two economists whose work has been cited endlessly to support austerity. And I can’t resist adding it to this list:

Quote Of The Day: “The Dictatorship of Data”

How The NBA Finals Taught A Lesson About Not Being “Data-Driven”

Second Quote Of The Day: The Dangers Of Being “Data-Driven”

The Great Lakes Center has released an excellent report on Data-driven Improvement and Accountability. The Washington Post published an excerpt, Six principles for using data to hold people accountable.

The Tyranny of the Datum is by John Kuhn.

Garbage In, Garbage Out: Or, How to Lie with Bad Data is from Medium.

How ‘data walls’ in classrooms can humiliate young kids is by Valerie Strauss.

Select Your Conclusions, Apply Data is from The Shanker Blog.

How ‘platooning’ and data walls are changing elementary school is from The Washington Post.

Big data: are we making a big mistake? is from The Financial Times.

Ainge: Analytics Sometimes Leads To Shortcuts is from RealGM Basketball.

Misusing Test Data is from Renee Moore’s blog.

Additional suggestions are welcome.

If you found this post useful, you might want to consider subscribing to this blog for free.

You might also want to explore the over 600 other “The Best…” lists I’ve compiled.

Print Friendly

Author: Larry Ferlazzo

I'm a high school teacher in Sacramento, CA.

10 Comments

  1. Pingback: i n f o d o n » February 18th

  2. Pingback: Grow Up, People! « getyourheadaround

  3. Pingback: Now It’s Hartford’s Turn To Show How NOT To Do Parent Involvement | Engaging Parents In School...

  4. Pingback: Grow Up, People! | Wrap Your Head Around

  5. Pingback: links for 2011-07-05 « Explore the Possibilities

  6. Pingback: Edtech 597: Response to article – Data Driven Schools | Blogs in the Classroom

  7. Pingback: Merit pay and ‘loss aversion’ | eSchool News

  8. Pingback: Teacher: How my 9th graders graded me

  9. Well said. Being data informed is certainly driving a great deal of change within education, balanced with other important information about students. There is a lot of great work being done with data analysis and data linkage tools for the future of education with P20 and SLDS initiatives. Linking K-12 data with college and career data will certainly have a positive, significant impact on student achievement.

    Linda Boudreau
    Data Ladder

    • A single comprehensive and timely education data standard that ensures usability is key to the success of these programs. States need to take advantage of the data analysis that is being made available.

      Linda Boudreau
      http://www.dataladder.com

Leave a Reply

Required fields are marked *.