TL;DR: This cool, interactive website asks you to participate in a replication. It also explains how a researcher decision on how to define "randomness" may have driven the main effect of the whole study. There is also a scatter plot and a regression line, talk of probability, and replication of a cognitive example. Long Version:  This example is equal parts stats and RM. I imagine that it can be used in several different ways: -Introduce the replication crisis by participating in a wee replication -Introduce a respectful replication based on the interpretation of the outcome variable  -Data visualization and scatterplots -Probability -Aging research Okay, so this interactive story from The Pudding is a deep dive into how one researcher's decision may be responsible for the study's main effect.  Gauvrit et al. (2017 ) argue that younger people generate more random responses to several probability tasks. From this, the authors conclude that human behavioral complexity...

NOTE: SPOILERS ABOUND. The Good Place is on NBC. And I love it.  At the heart of the show is one demon's (Michael) efforts to create a new version of hell that is only hellish because every person is already being punished by who and what they are. Right, I know. Anyway, in Season 3, Episode 11, Michael's bosses argue that this hell isn't working because it actually leads to self-improvement and fulfillment for everyone is supposed to be tormented. And the bosses argue that the self-improvement is a fluke. So one of the other characters, a philosopher named Chidi, suggests...SCIENTIFIC REPLICATION!! The whole episode is great, but here are some screenshots to get started.

This example provides evidence of data funny business beyond psychology, shows why pre-registration is a good thing, AND uses a chi-square. Bonus points for being couched in medicine and prominently featuring randomized controlled trials (RCT). Basically, Kaplan and Irving's  research checked out the results for RCTs funded with grants from the National Heart, Lung and Blood Institute. See below for how they selected their studies: And what did they find? When folks started registering their outcomes, folks started to get fewer "beneficial" results. Which probably REALLY means that some of those previous "beneficial" results were not so beneficial, or the result of some data massaging. See below: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0132382           Another reason to love this example: It is a real life chi-square that is easy to understand! I feel like I don't have enough great chi-square examples in my lif...

Maki Naro created a terrific comic strip detailing the replication, where it came from, where we are, and possible solutions.  You can use it in class to introduce the crisis and solutions. I particularly enjoy the overall tone: Hope is not lost. This is a time of change in statistics and methodology that will ultimately make science better. A few highlights: *History of science, including the very first research journal (and why the pressure to get published has lead to bad science) *Illustration of some statsy ways to bend the truth in science  *References big moments in the Replication Crisis  *Discusses the crisis AND solutions (PLOS, SIPS, COS)

As described in Wired's pop piece, the average body temperature for healthy adults isn't 98.6℉. Instead, data suggests that it is 97.7℉. Here is a link to the original study by  Hausmann, Berna, Ggujral, Ayubi, Howekins, Brownstein, & Dedeoglu . 1. This is an excellent theoretical example for explaining a situation where a one-sample t-test could answer your research question. 2. I created fake data that jive with the results, so you can conduct the test with your students. This data set mimicked the original findings for healthy adults (M = 97.7, SD = .72) and was generated with Andy Luttrell's Data Generator for Teaching Statistics . 97.39 97.45 97.96 97.35 96.74 99.66 98.21 99.02 96.78 97.70 96.90 97.29 97.99 97.73 98.18 97.78 97.17 97.34 97.56 98.13 97.77 97.07 97.13 9...

In my stats classes, we talk about the replication crisis. When introducing the topic, I use this  reading from NOBA . I think it is also important for my students to think about how science could create an environment where replication is more valued. And the Dutch Organization for Scientific Research has come up with a solution: It is providing grants to nine groups to either 1) replicate famous findings or 2) reanalyze famous findings. This piece from Science details their effort s. The Dutch Organization for Scientific Research provides more details on the grant recipients , which include several researchers replicating psychology findings: How to use in class: Again, talk about the replication crisis. Ask you students to generate ways to make replication more valued. Then, give them a bit of faith in psychology/science by sharing this information on how science is on it. From a broader view, this could introduce the idea of grants to your undergraduates or get yo...

This NPR story is about reproducibility in science that ISN'T psychology, the limitations of expert intuition, and the story is a summary of a recent research article from PLOS Biology  (so open science that isn't psychology, too!). Thrust of the story: Cancer researchers may be having a similar problem to psychologists in terms of replication.  I've blogged this issue before. In particular, concerns with replication in cancer research, possibly due to the variability with which lab rats are housed and fed . So, this story is about a study in which 200 cancer researchers, post-docs, and graduate students took a look at six pre-registered cancer stud y replications and guessed which studies would successfully replicate. And the participants systematically overestimated the likelihood of replication. However, researchers with high h-indices, were more accurate that the general sample. I wonder if the high h-indicies uncover super-experts or super-researchers who have be...

This NPR story is a summary of the decisively titled " The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses " authored by Dr. John Ioannidis. The NPR story provides a very brief explanation of meta-analysis and systematic reviews. It explains that they were originally used as a way to make sense of many conflicting research findings coming from a variety of different researchers. But these very influential publications are now being sponsored and possibly influenced by Big Pharma. This example explains conflicts of interest and how they can influence research outcomes. In addition to financial relationships, the author also cites ideological allegiances as a source of bias in meta-analysis. In addition to Dr. Ioannidis, Dr. Peter Kramer was interviewed. He is a psychiatrist who defends the efficacy of antidepressants. He suggests that researchers who believe that placebos are just as effective as anti-depressants tend to analy...

This syllabus for  PSY 607: Everything is Fucked ,  made the rounds last week. The syllabus is for a course that  purports  that science is fucked. The course readings are a list of articles and books that hit on the limitations of statistics and research psychology ( p -values, shortcomings of meta-analysis, misuse of mediation, replication crisis, etc.). PSY 607 isn't an actual class ( as author/psychologist/blogger Srivastava explains in this piece from The Chronicle ) but it does provide a fine reading list for understanding some of the current debates and changes in statistics and psychology.  Most of articles are probably too advanced for undergraduates but perfectly appropriate for teaching graduat e students about our field and staying up to date as instructors of statistics. Here is a link to the original blog post/syllabus. 

This hilarious video is making the rounds on the Interwebz. Kudos to John Oliver and his writing team for so succinctly and hilariously summarizing many different research problems...why replication is important but not rewarded, how research is presented to the public, how researchers over-reach about their own findings, etc.  I Tweeted about this, but am making it cannon by sharing as a blog post. Note: This video has some off-color humor (multiple references to bear fellatio) so it is best suited to college aged students. I will use this in my Online and Honors classes as discussion prompts. Here are some of the prompts I came up with: 1) In your own words, why aren't replications published? How do you think the scientific community could correct this problem?  2) In your own words, explain just ONE of how a RESEARCHER can manipulate their own data and/or research findings. It should be one of the methods of manipulation described in the video. Also, don't just na...

If you are unaware, Noba Project is a collaboration of many, many psychology instructors who create and make freely available text books as well as stand-alone chapters (modules) that cover a wide variety of psychology topics. You can build a personalized text book AND access test banks/powerpoints for the materials offered. Well, one of the new modules covers the replication crisis in psychology . I think it is thorough treatment of the issue and appropriate for undegraduates.

My favorite data is data that is both counter-intuitive and tests the efficacy of commonly held beliefs. Emily Oster's (writing for 538) presents such  data in her investigation of vitamin efficacy . The short version of this article: Data that associates vitamins with health gains are based on crap observational research. More recent and better research throws lots of shade on vitamin usage. Specific highlights that could make for good class discussion: -This article explains the flaws in observational research as well as an example of how to do good observational research well (via The Physician's Health Study , with large samples of demographically similar individuals as described in the portion of the article featuring the Vitamin E study). This point provides an example of why controlled, double-blind lab research is the king of all the research. -This is an accessible example as most of your students took their Flintstones. -The article also demonstrates The Thir...

Recently, a Facebook craze asked users to list three things you are thankful for for five days. Data scientis ts Winter Mason, Funda Kivran-Swaine,  Moira Burke, and Lada Adamic  at Fa cebook have analyzed this dat a to better understand the patterns of gratitude publically shared by Facebook users. The data analysts broke down data by most frequently listed gratitude topic: Most frequently "liked" gratitude posts: (lots of support for our friends in recovery, which is nice to see). Gender differences in gratitude...here is data for women. The wine gratitude finding for women was not present in the data for men. Ha. Idiosyncratic data by state. I would say that Pennsylvania's fondness for country music rings true for me. How to use in class: This example provides several interesting, easy to read graphs, and the graphs show how researchers can break down a single data set in a variety of interesting ways (by gender, by age, by state). Add...

Center for Open Science (COS) is an  organization  that seeks " to increase openness, integrity, and reproducibility of scientific research " . As a social psychologist, I am most  familiar  with COS as a repository for experimental data. However, COS also provides free consulting services as to teach scientists how to make their own research processes more replication-friendly .  As scholars, we can certainly take advantage of these services. As instructors, the kind folks at COS are willing to provide workshops to our students (including, but not limited to, online workshops). Topics that they can cover include:  Reproducible Research Practices, Power Analyses, The ‘New Statistics’, Cumulative Meta-analyses, and Using R to create reproducible code (or more information on scheduling, see their availability  calendar ). I once heard it said that the way you learn how to conduct research and statistics in graduate school will be the way you...

This article from Nature is  an excellent primer on the concerns surrounding the use of p -values as the great gate keeper of statistical significance. The article includes historical perspective on how p -values came to be so widely used as well as some discussion on solutions and alternative measures of significance. This article also provides good examples failed attempts at replication (good examples of Type I errors) and a shout out to Open Science Framework folks. Personally, I have revised my class for the fall to include more discussion of and use of effect sizes. I think this article may be a bit above an undergraduate, introduction to statistics class but it could be useful for us as instructors as well as a good reading for advanced undergraduates and graduate students.

This very accessible summary lists the ways people fib with, misrepresent, and overextend data findings. It was written as an attempt to give non-research folk (in particular, law makers), a cheat sheet of things to consider before embracing/rejecting research driven policy and laws. A sound list, covering plenty of statsy topics (p-values, the importance of replication), but what I really like is that they article doesn't criticize the researchers as the source of the problem. It places the onus on each person to properly interpret research findings. This list also emphasizes the importance of data driven change.

Here is a link to a recent co-authored publication that used Second Life to teach students about virtual data collection as well as the broader trend in psychology to study how virtual environments influence interpersonal interactions. Specifically, students replicated evolutionary psychology findings using Second Life avatars. We also discuss best practices for using Second Life in the class room as well as our partial replication of previously established evolutionary psychology findings (Clark & Hatfield, 1989, Buss, Larson, Weston, & Semmelroth, 1992).

While it isn't hypothesis driven research  data, this data was collected to see which states are the sweariest. The data collection itself is interesting and a good, teachable example. First, the article describes previous research that looked at swearing by state (typically, using publicly available data via Twitter or Facebook). Then, they describe the data collection used for the current research: " A new map, though, takes a more complicated approach. Instead of using text, it uses data gathered from ... phone calls. You know how, when you call a customer service rep for your ISP or your bank or what have you, you're informed that your call will be recorded?  Marchex Institute , the data and research arm of the ad firm Marchex,  got ahold of the data that resulted from some recordings , examining more than 600,000 phone calls from the past 12 months—calls placed by consumers to businesses across 30 different industries. It then used call mining technology to isola...

I decided that I shouldn't limit my Hunks of Statistics list to man-hunks. There are some lady-hunks as well. Like Sharon Begley. Sharon Begley, from thedailybeast.com