Monday, October 5, 2015

How NOT to interpret confidence intervals/margins of error: Feel the Bern edition

This headline is a good example of a) journalists misrepresenting statistics as well as b) confidence intervals/margin of error more broadly. See the headline below:

In actuality, Bernie didn't exactly take the lead over Hillary Clinton. Instead, a Quinnipiac poll showed that 41% of likely Democratic primary voters in Iowa indicated that they would vote for Sanders, while 40% reported that they would vote for Clinton.

If you go to the original Quinnipiac poll, you can read that the actual data has a margin of error of +/- 3.4%, which means that the candidates are running neck and neck. Which, I think, would have still been a compelling headline. 

I used this as an example just last week to explain applied confidence intervals. I also used this as a round-about way of explaining how confidence intervals are now being used as an alternative/compliment to p-values. 

Monday, September 28, 2015

Aschwanden's "Science is broken, it is just a hell of a lot harder than we give it credit for"

Aschwanden (for did an extensive piece that summarizes that data/p-hacking/what's wrong with statistical significance crisis in statistics. There is a focus on the social sciences, including some quotes from Brian Nosek regarding his replication work. The report also draws attention to Retraction Watch and Center for Open Science as well as retractions of findings (as an indicator of fraud and data misuse). The article also describes our funny bias of sticking to early, big research findings even after those research findings are disproved (example used here is the breakfast eating:weight loss relationship).

The whole article could be used for a statistics or research methods class. I do think that the p-hacking interactive tool found in this report could be especially useful illustration of How to Lie with Statistics.

The "Hack your way to scientific glory" interactive piece demonstrates that if you fool around enough with your operationalized variables, you can probably, eventually, find a way to support your argument (AKA p-hacking). This example uses the question of whether to economy does better under Republican or Democratic leadership to prove a point. Within the tool, you can operationalize both leadership and measures of economic health in a number of different ways (via the check boxes on the left), resulting in a number of different outcomes.

 This is whole article is useful in the classroom simply as a way to discuss the shortcomings of p-values (with the interactive piece) but I think the whole article is accessible to undergraduates, especially as it is littered with embedded links that provide greater information to previous research scandals and debates.