Skip to main content

An interactive description of scientific replication

TL;DR:

This cool, interactive website asks you to participate in a replication. It also explains how a researcher decision on how to define "randomness" may have driven the main effect of the whole study. There is also a scatter plot and a regression line, talk of probability, and replication of a cognitive example.

Long Version: 

This example is equal parts stats and RM. I imagine that it can be used in several different ways:

-Introduce the replication crisis by participating in a wee replication

-Introduce a respectful replication based on the interpretation of the outcome variable 

-Data visualization and scatterplots

-Probability

-Aging research

Okay, so this interactive story from The Pudding is a deep dive into how one researcher's decision may be responsible for the study's main effect. Gauvrit et al. (2017) argue that younger people generate more random responses to several probability tasks. From this, the authors conclude that human behavioral complexity peaks at 25.  

The Pudding authors argue that depending on how you define "randomness", the main effect goes away.

It demonstrates both a replication, a replication in which your students can participate. It also and happen. Modify the cut-off criteria for your experimental stimuli. 

Text in image: The study received coverage across dozens of outlets. The headline: A person’s ability to be random peaks around 25 years old and declines after 60. The researchers made their data and methods public so we explored the idea of making an age guessing game. This unearthed some questions for us, so the story became about the replication crisis; the ongoing concern that it is hard to reproduce many scientific studies. We think the findings of the study are at the mercy of a single decision the researchers made to filter out questionable responses. To us, this meant the participant either misunderstood the instructions, or intentionally subverted the experiment.

I think this has a place in any RM course to introduce The Replication Crisis. Before you get to the screen grab featured above, you have the option to participate in a replication of Gauvrit et al. See below for a screen grab of the instructions for one of the replication tasks: 

 


Using the blue and black toggle button, you can look at the regression line under the two conditions. The relationship goes away when the different criteria are applied to the When they exclude 
. This is a bonus lesson on interpreting scatterplots/regression, and .



Anyway. Imma using this in both our Professional Development course to introduce the replication crisis, and in my honors statistics class for a "Discussion Day" about replication in science.


Labels:

Comments

Post a Comment

Popular posts from this blog

Ways to use funny meme scales in your stats classes

Have you ever heard of the theory that there are multiple people worldwide thinking about the same novel thing at the same time? It is the multiple discovery hypothesis of invention . Like, multiple great minds around the world were working on calculus at the same time. Well, I think a bunch of super-duper psychology professors were all thinking about scale memes and pedagogy at the same time. Clearly, this is just as impressive as calculus. Who were some of these great minds? 1) Dr.  Molly Metz maintains a curated list of hilarious "How you doing?" scales.  2) Dr. Esther Lindenström posted about using these scales as student check-ins. 3) I was working on a blog post about using such scales to teach the basics of variables.  So, I decided to create a post about three ways to use these scales in your stats classes:  1) Teaching the basics of variables. 2) Nominal vs. ordinal scales.  3) Daily check-in with your students.  1. Teach your students the basics...
Post a Comment
Read more

Using pulse rates to determine the scariest of scary movies

  The Science of Scare project, conducted by MoneySuperMarket.com, recorded heart rates in participants watching fifty horror movies to determine the scariest of scary movies. Below is a screenshot of the original variables and data for 12 of the 50 movies provided by MoneySuperMarket.com: https://www.moneysupermarket.com/broadband/features/science-of-scare/ https://www.moneysupermarket.com/broadband/features/science-of-scare/ Here is my version of the data in Excel format . It includes the original data plus four additional columns (so you can run more analyses on the data): -Year of Release -Rotten Tomato rating -Does this movie have a sequel (yes or no)? -Is this movie a sequel (yes or no)? Here are some ways you could use this in class: 1. Correlation : Rotten Tomato rating does not correlate with the overall scare score ( r = 0.13, p = 0.36).   2. Within-subject research design : Baseline, average, and maximum heart rates are reported for each film.   3. ...
Post a Comment
Read more

If your students get the joke, they get statistics.

Gleaned from multiple sources (FB, Pinterest, Twitter, none of these belong to me, etc.). Remember, if your students can explain why a stats funny is funny, they are demonstrating statistical knowledge. I like to ask students to explain the humor in such examples for extra credit points (see below for an example from my FA14 final exam). Using xkcd.com for bonus points/assessing if students understand that correlation =/= causation What are the numerical thresholds for probability?  How does this refer to alpha? What type of error is being described, Type I or Type II? What measure of central tendency is being described? Dilbert: http://search.dilbert.com/comic/Kill%20Anyone Sampling, CLT http://foulmouthedbaker.com/2013/10/03/graphs-belong-on-cakes/ Because control vs. sample, standard deviations, normal curves. Also,"skewed" pun. If you go to the original website , the story behind this cakes has to do w...
1 comment
Read more