Smart Guys Can Get It Wrong

Exif_JPEG_PICTUREYou know that feeling when one of your friends says something stupid that makes a different friend angry?  And you realize that you agree with the angry friend?  Yeah, so that happened to me a few weeks ago.  Let me explain.

I am a big fan of the FiveThirtyEight blog that Nate Silver runs with his team of statisticians/economists.  Nate has a knack for explaining technical mathematical stuff using everyday examples.  He started in sports and moved to politics (correctly predicting most of the 2014 races), and then ESPN brought his blog back over to their site where it lives now.  When he sticks to those two topics–sports/politics–he is a bastion of logic in a world of opinions.

Lately, though, Silver has been dipping his toes into the realm of educational policy and the ridiculous questionable data that supports some of the recent “reforms”.  In the recent article “The Science of Grading Teachers Gets High Marks“, Silver’s ed dude Andrew Flowers analyzes some of the discussion around the Vergara case.  He discusses the back and forth between statisticians at Harvard, Brown, and Columbia and Jesse Rothstein of Stanford.

While I agree with Flowers that the arguments over methods for analyzing teacher impact are a positive sign that science is working as it should, I side with Valerie Strauss when she writes for the Washington Post that,

“The quality of the underlying standardized  assessment is assumed to be at least adequate — or why use the student scores to evaluate their teachers? — when, in fact, many of them are less than adequate to provide a well-rounded, authentic look at what students have learned and are able to do.”

Flowers provided this throw-away phrase that was guaranteed to make educators angry,

“In order to perfectly isolate the effect of a teacher on a student’s test scores — setting aside whether higher test scores is the right goalstudents would need to be assigned to teachers randomly.”  [emphasis mine]

What?!?  How can you “set aside” the source of all of the data that you are analyzing (or, more accurately, discussing the analysis of)?  That’s like saying, “Setting aside the fact that koalas are not actually bears, observing them is a great way to learn about the bear behavior.” We MUST stop pretending that mathematical analysis can make up for crappy assessments.

Opinions?  You know what to do.

 

Image: “Friendly Female Koala” by QuartlOwn work. Licensed under CC BY-SA 3.0 via Wikimedia Commons.