Sunday, January 20, 2013

Stephen Colbert vs. Darryl Bem = effect size vs. statistical significance



Darryl Bem on The Colbert Report
I love me some Colbert Report. So imagine my delight when he interviewed social psychologist Darryl Bem. Bem is famous for his sex roles inventory as well as his Psi research. Colbert interviewed him about his 2012 Journal of Personality and Social Psychology article, Feeling the Future: Experimental Evidence for
Anomalous Retroactive Influences on Cognition and Affect, which demonstrated a better-than-chance ability to predict an outcome. Here, the outcome was guessing which side of a computer screen would contain an erotic image (Yes, Colbert had a field day with this. Yes, please watch the clip in its entirety before sharing it with a classroom of impressionable college students).

Big deal?
Needless to say, Colbert reveled in poking fun at the "Time Traveling Porn" research. However, the interview is of some educational value because it a)does a good job of describing the research methods used in the study. Additionally, b) he calls out Bem because the better-than-average, statistically significant predictions amounted to guessing correctly 53% of the time when chance would dictate 50% of the time. Which is statistically significant but not a very powerful finding. When I use this clip in class, I typically show my students the original research article and they can compare the low Cohen's d values to the high p-values.

No comments:

Post a Comment