Monday, December 26, 2016

u/zonination's "Got ticked off about skittles posts, so I decided to make a proper analysis for /r/dataisbeautiful [OC]"

The subreddit s/dataisbeautiful was inundated by folks creating color distributions for bags of candy. And because 1) it is Reddit and 2) stats nerds take joy in silly things, candy graphing got out of hand. See below:

And because it is Reddit, and, to be a fair, statistically unreliable, other posters would claim that this data WASN'T beautiful because it was a small sample size and didn't generalize. One bag of Skittles, they claimed. didn't tell you a lot about the underlying population of Skittles.

Until reditor zonination came along, bought 35 enormous bags of Skittles, and went to work meticulously documenting the color distribution in each bag. And he used R. And he created multiple data visualizations. See below. Here is the reddit post, and here is his Imgur gallery with visualizations and a narrative describing his findings. (Y'all, I know Reddit has a bad reputation at times, but the discussion in this posting is hilarious if you are a stats nerd. Check it out.).

He explained his data with a heat map...
And a stacked bar chart, that really illustrates outlier bags 15 and 16. Imagine if you mistakenly tried to generalize from of those bags?
And he presents the increasingly popular Violin plot. well as the perpetual favorite, a bar chart with error bars.

AND...he shared his data and R code with the world.

How to use in class:
-Discuss proper sample sizes required in order to generalize to a population. I think rouge bags 15 and 16 are especially effective at demonstrating sample error.
-Your students understand the concept of Skittles. Therefore, they will be able to understand the nuances of these different kinds of data visualizations.
-Buy your students some Skittles and replicate.
-Data and code available to play around with.

Monday, December 19, 2016

Kevin McIntyre's Open Stats Lab

Dr. Kevin McIntryre from Trinity University has created the Open Stats Lab. OSL provides users with research articles, data sets, and worksheets for studies that illustrate statistical tests commonly taught in Introduction to Statistics.

Topics covered, illustrated beautifully by Natalie Perez

All of his examples come from Open Science Framework-compliant publications from Psychological Science. McIntyre presents the OSF data (in SPSS but .CSV files are available), the original research article, AND a worksheet to go along with each article.

Layout for each article/data set/activity. This article demonstrates one-way ANOVA.

I know, right? I think it can be difficult to find 1) research an UG can follow that 2) contains simple data analyses. And here, McIntryre presents it all. This project was funded by a grant from APS.

Wednesday, December 14, 2016

A wintery mix of holiday data.

Property of @JenSacco54
A wintery example of why range is a crap measure of variability

Monday, December 12, 2016

Wilson's "Find Out What Your British Name Would Be"

Students love personalized, interactive stuff. This website from Chirs Wilson over at Time allows your American students to enter their name and they recieve their British statistical doppleganger name in return. Or vice versa.

And by statistical doppleganger, I mean that the author sorted through name popularity databases in the UK and America. He then used a Least Squared Error model in order to find strong linear relationships for popularity over time between names.

How to use in class:
Linear relationship
Trends over time

Monday, December 5, 2016

Aschwanden's "You Can’t Trust What You Read About Nutrition"

Fivethirtyeight provides lots of beautiful pictures of spurious correlations found by their own in-house study.
At the heart of this article are the limitations of a major tool use in nutritional research, the Food Frequency Questionnaire (FFQ). The author does a mini-study, enlisting the help of several co-workers and readers. They track track their own food for a week and reflect on how difficult it is to properly estimate and recall food (perhaps a mini-experiment you could do with your own students?).

And she shares the spurious correlations she found in her own mini-research:

Aschwanden also discusses how much noise and lack of consensus their is in real, published nutritional research (a good argument for why we need replication!):

How to use in class:
-Short comings of survey research, especially survey research that relies on accurate memories
-Spurious correlations (and p-values!)
-Correlation does not equal causation
-Why replication is necessary

Also included is an amusing video that shows what it is like to be a participant in a nutrition study. It details the FFQ, or Food Frequency Questionnaire. And the video touches on serving sizes and portions, how how it may be difficult for many of to properly estimate (per the example) how many cups of spare ribs we consume per week. 

Monday, November 28, 2016

Teaching the "new statistics": A call for materials (and sharing said materials!)

This blog is usually dedicated to sharing ideas for teaching statistics. And I will share some ideas for teaching. But I'm also asking you to share YOUR ideas for teaching statistics. Specifically, your ideas for teaching the new statistics: effect size, confidence intervals, etc.

The following email recently came across the Society for the Teaching of Psychology listserv from Robert Calin-Jageman (

"Is anyone out there incorporating the "New Statistics" (estimation, confidence intervals, meta-analysis) into their stats/methods sequence?
I'm working with Geoff Cumming on putting together an APS 2017 symposium proposal on teaching the New Statistics.  We'd love to hear back from anyone who has already started or is about to.  Specifically, we'd love to:
        * Collect resources you'd be willing to share (syllabi, assignments, etc.)
        * Collect narratives of your experience (the good, the bad, the unexpected)
        * Know what tips/suggestions you might have for others embarking on the transition
We'll use responses to help shape our symposium proposal (and if you're interested in possibly joining, let us know).
In addition, we're curating resources, tips, on a "Getting started teaching the New Statistics" page on the OSF :"

I'll start by sharing to examples I have successfully used in class and have previously blogged about. Here is a post about a Facebook research study (Kramer, Guillory, & Hancock, 2014) that demonstrates how large sample sizes lead to statistical significance but very small effect sizes. This study also demonstrates how to mislead with graphs and the debate of whether or not Terms of Service agreements are the same thing as informed consent.

And I use this Colbert interview with Daryl Bem in which Bem is basically arguing for p-values without ever saying "p-values", and Colbert is arguing for effect size/clinical significance without ever saying those words. I follow up this video by sharing a table from the much-debated Bem, 2014 JPSP article that displays, again, small p-values and large effect sizes. NOTE: This interview is about the Bem, 2014 research that used erotic imagery as stimuli, so the  tone of the interview might be a little racy for inclass use at some universities/high school statistics classes.

Finally, I use Kristopher Magnussen's website to illustrate a quite a few statistical principles, including Cohen's d. 

So, I am sharing it here to reach out to all of you statistics instructors to see 1) if you are interested in sharing your ideas for the APS symposium/OSF resource, 2) would like to look out for the APS symposium if you are attending next year, 3) alert you to the great OSF resource listed above, and in the spirit of the holiday season, 4) share, share, share.  

Monday, November 21, 2016

Chokshi's "How Much Weed Is in a Joint? Pot Experts Have a New Estimate"

Alright, stick with me. This article is about marijuana dosage and it provides good examples for how researchers go about quantifying their variables in order to properly study them. The article also highlights the importance of Subject Matter Experts in the process and how one research question can have many stakeholders.

As the title states, the main question raised by this article is "How much weed is in a joint?". Why is this so important? Researchers in medicine, addictions, developmental psychology, criminal justice, etc. are trying to determine how much pot a person is probably smoking when most drug use surveys measure marijuana use by the joint. How to use in a statistics class:

Wednesday, November 16, 2016

The Onion's "Study: Giving Away “I Voted” Burger Instead Of Sticker Would Increase Voter Turnout By 80%"


A very funny example of conflict of interest, as this satirical study was sponsored by Red Robin. Click through to the original content to read how the study replaced "I Voted" stickers with "thick Red Robin Gourmet Cheeseburger complete with pickle relish, tomatoes, onions, lettuce, mayonnaise, and their choice of cheese".

Monday, November 14, 2016

Johnson & Wilson's The 13 High-Paying Jobs You Don’t Want to Have

This is a lot of I/O and personality a little bit of stats. But it does demonstrate correlation, percentiles, and it is interactive.

For this article from Time, Johnson and Wilson used participant score on a very popular vocational selection tool, the Holland Inventory (sometimes called the RAISEC) and participant salary information to see if there is a strong relationship between salary and personality-job fit. There is not.

How to use in class:

-Show your students what a weak correlation looks like when expressed via scatter plot. Seriously. I spend a lot of time looking for examples for teaching statistics. And there are all sorts of significant positive and negative correlation examples out there. But good examples of non-relationships are a lot rarer.

-If you teach I/O, this fits nicely into personality-job fit lecture. If you don't teach I/O but are a psychologist, this still applies to your field and may introduce your students to the field of I/O.

-This example is interactive in a few ways. Johnson and Wilson got this data from a previous Time article that included the RAISEC survey. The survey, via Time, also returns the respondents' results. It makes the example more self relevant, and also gives your students a bit of vocational advice.

Additionally, there is a search feature so that you can look up a job title and find the personality-job fit percentile for a given job. Or, you can cursor over any of the dots on the scatter plot to get the job title, salary, and personality-job fit for that job title.

You can also use the search feature to look up particular job titles.

Wednesday, November 9, 2016

CNN, exit polls, and chi-square examples.

CNN posted a whole mess of exit polling data that illustrates how different demographics voted last night. And through my "I teach too many stats classes" lense, I see many examples of chi-square.

I think they work at a conceptual level to clearly illustrate how chi-square looks at people falling into different categories, then measures whether the distribution of people is by chance or not.

If you actually wanted to test these using chi square, I would suggest you should problem delete the other/no answer column (or else they will all come out as statistical significant, I would wager).

EDIT (11/14/16); Daniel Findley made of video demonstrating how to use Excel to conduct chi-square tests on the marital status data. Check it out here.

Monday, November 7, 2016

Collin's "America’s most prolific wall punchers, charted"

Collin gleaned some archival data about ER visits in America from US Consumer Product Safety commission. For each ER visit, there is a brief description of the reason for the visit. Collin queried punching related injuries. See his Method section below describes how he set the parameters for his operationalized variable. With a bit of explaining, you could also describe how Collin took qualitative data (the written description of the injury) and converted it into quantitative data:

Then he made some charts.

The age of wall punchers is right skewed. And probably could be used in a Developmental Psychology class to illustrate poor judgement in adolescents as well as the emergence of the prefrontal cortext/executive thinking skills in one's early 20s.
The author looked at wall punching by month of the year and uncovered a fairly uniform distribution.

 How to use in class:
-Taking qualitative data and coding it (here, turning "ER Visit" into "Wall Punch: Yes or No"
-Uniform Distributions
-Method section
-Archival data
-Criteria for operationlizing a variable when coding data

Monday, October 31, 2016

Harris' "Reviews Of Medical Studies May Be Tainted By Funders' Influence"

This NPR story is a summary of the decisively titled "The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses" authored by Dr. John Ioannidis.

The NPR story provides a very brief explanation of meta-analysis and systematic reviews. It explains that they were originally used as a way to make sense of many conflicting research findings coming from a variety of different researchers. But these very influential publications are now being sponsored and possibly influenced by Big Pharma.

This example explains conflicts of interest and how they can influence research outcomes. In addition to financial relationships, the author also cites ideological allegiances as a source of bias in meta-analysis. In addition to Dr. Ioannidis, Dr. Peter Kramer was interviewed. He is a psychiatrist who defends the efficacy of antidepressants. He suggests that researchers who believe that placebos are just as effective as anti-depressants tend to analyze meta-analysis data in such a way as to support that belief.

Ways to use in class:
-Meta-analysis as a way to sort out conflicting research findings.
-An example of conflict of interest.
-An example of experimenter bias (in the form of both the conflict of interest as well as individuals who believe that anti-depressants are ineffective).
-If you are like me and teach lots of pre-PT/OT/PA and nursing students, this is a applicable example for that crowd.
-Confirmation bias

Thursday, October 27, 2016

Turner's "E Is For Empathy: Sesame Workshop Takes A Crack At Kindness" and the K is for Kindness survey.

This NPR story is about a survey conducted by the folks at Sesame Street. And that survey asked parents and teachers about kindness. If kids are kind, if the world is kind, how they define kindness, etc..

The NPR story is a round about way of explaining how we operationalize variables, especially in psychology. And the survey itself provides examples of forced choice research questions and dichotomous responses that could have been Likert-type scales.

The NPR Story:

The Children's Television Workshop, the folks behind Sesame Street, have employees in charge of research and evaluation (a chance to plug off-the-beat-path stats jobs to your students). And they did a survey to figure out what it means to be kind when you are a kid. They surveyed parents and teachers to do so.

The main findings are summarized here. Parents and teachers are worried that the world isn't kind and doesn't emphasize kind. But both groups think that kindness is more important than academic achievement. 

Monday, October 24, 2016

Hancock's "Skip The Math: Researchers Paint A Picture Of Health Benefits And Risks"

Two scientists, Lazris and Rifkin, want to better illustrate the risks and benefits associated with preventative medicine. They do so by asking people to imagine theaters filled with 1,000 people, and describing the costs and benefits for different preventative procedures by discussing how many people in the theater will be saved or perish based on current efficacy data.

One such video can be viewed here and illustrates the absolute and relative risks associated with mammography. They are attempting to demystify statistics and better explain the risks and benefits by showing an animated theater filled with 1,000 women, and showing how many women actually have their lives saved by mammograms (see screen shot below)... well as the number of women who received false positives over the course of a life time...

A screen shot of the video, which is trying a new way to illustrate risk.
...the video also illustrates how a "20% reduction in breast cancer deaths" can actually be equal to 1 life saved out of 1,000.

This video touches on the confusion about relative versus absolute risk as well as the actual effectiveness of preventative medicine (and why it is so important to conduct efficacy research for medical interventions). I have a few discussion days with my Honor students and a discussion board with my online students that involve this piece from that questions whether methods of early cancer detection save lives or just uncover non-cancerous variation within the human body. This topic leads to lively discussions.

How to use in class:
-Relative risk
-Absolute risk
-False positives
-Medical examples (and I have plenty of pre-medical professional students)
-An example of why we teach our students to make graphs and charts. Sometimes, data is better shared via illustration
-Using statistics to inform important real-life decisions

Monday, October 17, 2016

Pew Research's "The art and science of the scatterplot"

Sometimes, we need to convince our students that taking a statistics class changes the way they think for the better.

This example demonstrates that one seemingly simple skill, interpreting a scatter plot, is tougher than it seems. Pew Research conducted a survey on scientific thinking in America (here is a link to that survey) and they found that only 63% of American adults can correctly interpret the linear relationship illustrated in the scatter plot below. And that 63% came out a survey with multiple choice responses!

How to use in class:
-Show your students that a major data collection/survey firm decided that interpreting statistics was an appropriate question on their ten-item quiz of scientific literacy.
-Show your students that many randomly selected Americans can't interpret a scatter plot correctly.

And for us instructors:
-Maybe a seemingly simple task like the one in this survey isn't as intuitive as we think it is!

Monday, October 10, 2016

Pew Research's "Growing Ideological Consistency"

This interactive tool from Pew research illustrates left and right skew as well as median and longitudinal data. The x-axis indicates how politically consistent (as determined by a survey of political issues) self-identified republicans and democrats are across time. Press the button and you can animate data, or cut up the data so you only see one party or only the most politically active Americans.
The data for both political part goes from being normally distributed in 1994 to skewed by 2014. And you can watch what happens to the median as the political winds change (and perhaps remind your students as to why mean would be the less desirable measure of central tendency for this example). I think it is interesting to see the relative unity in political thought (as demonstrated by more Republicans and Democrats indicating mixed political opinions) in the wake of 9/11 but more politically consistent (divided?) in the more recent past.

Depending on how deep you feel like going with this example, it can also illustrate research methods for your students as Pew has been gathering this research for years. 

Monday, October 3, 2016

Dr. Barry Marshall as an example of Type II error.

I just used this example in class and I realized that I never shared it on my blog. I really love this example of Type II error (and some other stuff, too). So here it goes.

Monday, September 26, 2016

Hyperbole and a Half's "Boyfriend doesn't have ebola. Probably. "

I've been using this example in class for a few years but never got around to blogging about it until now.

It seems that the first chapter of every statistics class provides a boring explanation of what a variable is, and examples of variables, and operationalizing variables, and quantifying the abstract for the purposes of conducting statistical analyses.

I try to make that boring topic funnier and applicable to real life via this post entitled "Boyfriend doesn't have ebola. Probably." from Allie Brosh, editor of Hyperbole and a Half.

In this posting, she rips apart the good old FACES scale after a trip with her boyfriend to the ER.

Monday, September 19, 2016

If your students get the joke, they get statistics.

Gleaned from multiple sources (FB, Pinterest, Twitter, none of these belong to me, etc.). Remember, if your students can explain why a stats funny is funny, they are demonstrating statistical knowledge. I like to ask students to explain the humor in such examples for extra credit points (see below for an example from my FA14 final exam).

Using for bonus points/assessing if students understand that correlation =/= causation

What are the numerical thresholds for probability? 

How does this refer to alpha? What type of error is being described, Type I or Type II?

What measure of central tendency is being described?

Sampling, CLT
Because control vs. sample, standard deviations, normal curves. Also,"skewed" pun. If you go to the original website, the story behind this cakes has to do with a section of crappy that is kind of funny and therapeutic for us teachers.

NOTE: The website the cake example comes from contains a lot of NSFW language. Which I, personally, have no problem with, but you might.
Because bar graphs, error bars, and understanding the joke behind this graph.
What kind of error, Type I or Type II?
 Reliability, n-size
What does correlation give us? What does it not?

What does the r^2 here indicate? Why would it be difficult to guess the direction of the relationship?

What is the joke here? For more rigor: What does et al. stand for? What are the APA rules for when to use et al.?