Thursday, May 30, 2019

Incorporating Hamilton: An American Musical into your stats class.

While I was attending the Teaching Institute at APS, I attended Wind Goodfriend's talk about using cases studies in the classroom.

Which got me thinking about fun case studies for statistics. But not, like, the classic story about Guinness Brewery and the t-test. I want case studies that feature a regular person in a regular job who used their idiosyncratic expertise to deduce from data in order to do something great. An example popped into my head while I was walking my dog and listening to the Hamilton soundtrack: Hercules Mulligan.

Okieriete Onaodowan, portraying Hercules Mulligan in Hamilton
He was a spy for America during the American Revolution. He was a tailor and did a lot of work for British military officers. This gave him access to data that he shared through a spy network in order to infer the timing of British military operations.

Here is a better summary, from the CIA: 


I like this example because he wasn't George Washington. And he wasn't Alexander Hamilton. He had this seemingly innocuous job, but he knew his job well, and he was smart, and he was able to see patterns in the information and turn them into valuable knowledge. He made the decision to use that knowledge in a spy network, which could have got him killed but instead saved George Washington's life on one notable occasion (it should also be noted, that his slave DID NOT make the choice to risk his life).

Similarly, I believe we should be teaching our students to be the best <insert dream job here> they can be, with the understanding that they will benefit from statistical reasoning skills and the ability to detect patterns that no one else can see.

Or, if you want to here Hercules Mulligan via Lin-Manuel Miranda, go to 2:40 on this video. Aside: Hercules Mulligan's solo in The Battle of Yorktown is my favorite rally song. I am working on having the swagger of Herc.


Saturday, May 25, 2019

The Good Place and Replication

NOTE: SPOILERS ABOUND.

The Good Place is on NBC. And I love it.  At the heart of the show is one demon's (Michael) efforts to create a new version of hell that is only hellish because every person is already being punished by who and what they are. Right, I know.

Anyway, in Season 3, episode 11, Michael's bosses argue that this hell isn't working because it actually leads to self-improvement and fulfillment for everyone is is supposed to be tormented. And the bosses argue that the self-improvement is a fluke. So one of the other characters, a philosopher named Chidi, suggests...SCIENTIFIC REPLICATION!! The whole episode is great, but here are some screen shots to get started.








Wednesday, May 15, 2019

Diversity in Tech by DataIsBeautiful

I am a fan of explaining the heart of a statistical analysis conceptually with words and examples, not with math. Information Is Beautiful has a gorgeous new interactive, Diversity in Tech, that uses data visualization to present gender and ethnic representation among employees at various big-name internet firms.

I think this example explains why we might use Chi-Square Goodness of Fit. I think it could also be used in an I-O class.

So, what this interactive gives you is a list of the main, big online firms. And then the proportions of different sort of people who fall into each category. See below:



When I look at that US Population baseline information, I see a bunch of expected data. And then when I see the data for different firms, I see Observed data. So, I see a bunch of conceptual examples for chi-square Goodness of Fit.

For example, look at gender. 51% of the population is female. That is you Expected data. Compare that to data for Indiegogo. They have 50% female employees. That is your Observed data. Eyeballing it, you can guess that wouldn't be a significant chi-square. The two distributions are very similar. Again, this is a CONCEPTUAL introduction to what chi-square looks at, not a computation one.





Another important conceptual piece to chi-square is the fact that you need your O and E values to be pretty far apart in order to get a big test statistic. So, you also ask your students to compare: Which do you think would have a larger chi-square test statistic for Latino Employees: Amazon or Ebay? The Expected value is 18%. Since Amazon's 13% is closer to the Expected value of 18% than Ebay's 4%, we would expect Amazon to have a smaller X2 value than Ebay.


I bet you could also use this data while teaching I-O. One way to demonstrate fair hiring practices is to demonstrate that your workforce mimics the ethnic breakdown of America, or your state, or your region. As such, you could ask your students to pretend to be consultants and make recommendations for which firms have the biggest disparities, using this data as evidence.

Sunday, May 12, 2019

Snake Oil Superfoods by InformationIsBeautiful

In my stats classes, we discuss popular claims that have been proven/disproven by research. So, learning styles. Vitamins. One claim we dig into are the wide array of claims made about the health benefits of different foods and folk beliefs about nutrition. But how to get into it? That is such a big field, looking at different foods used for different conditions. Send your students to InformationIsBeautiful's Snake Oil Super foods, which sorted through all of good studies and created an interactive data viz to summarize.


For instance, these are three foods, backed by science, for very specific issues:

BUT GET THIS: If you scroll over any of them, you get a quick summary of the findings AND a link to the research article. See below for Oats. NOICE.



The information isn't limited to slam dunks, either, it fleshes out promising foods and weak links as well.


AND...this is great...below the visualization there is all sorts of information on their methodology AND a link to their data. See below.




How to use in class:
1) Again, this can be a useful discussion prompt for evaluating scientific/click bait nutritional research.
2) I think this reminds students that research has to research very specific claims. No where in the data viz does it say "Garlic be good", instead, it looks at specific health claims and garlic. While Garlic:Cancer Treatment is in the "No Evidence" category, Garlic:Colds is in the "Inconclusive" category.
3) Plenty of links to published research, speadsheet provided to play with.

Monday, May 6, 2019

A big metaphor for effect sizes, featuring malaria.

TL; DR- Effect size interpretation requires more than numeric interpretation of the effect size. You need to think about what would be considered a big deal, real life change worth pursuing, given the real-world implications for your data.

For example, there is a  malaria vaccine with a 30% success rate undergoing a large scale trial in Malawi. If you consider that many other vaccines have much higher success rates, 30% seems like a relatively small "real world" impact, right?

However, two million people are diagnosed with malaria every year. If science could help 30% of two million, the relatively small effect of 30% is a big deal. Hell, a 10% reduction would be wonderful. So, a small practical effect, like "just" 30%, is actually a big deal, given the scale of the issue.

How to use this news story:
a) Interpreting effect sizes beyond Cohen's numeric recommendations.
b) A primer on large-scale medical trials and their ridiculously large n-sizes and transitioning from lab to real world trials.

So, I heard about this story about a new malaria vaccine trial in Malawi while I was getting ready to work and listening to the NPR Hourly News update, which is very on-brand for me.

The aspect of this report that really caught my attention: In the lab, this vaccine "only" has a 30% success rate. Which makes me think of the struggle of understanding (and explaining!) effect sizes in stats class.


Weren't p-values sort of nice in that they were either/or? You were significant, whatever that means, or not. The binary was comfortable.

But now we are using effect sizes, among other methods, to determine if research findings are "big" enough to get excited about. And effect size interpretation is a wee bit arbitrary, right? They can be nil, small, medium, or large, corresponding to the result of your effect size calculation. What does that mean? When does it count? When is your research with implementing or acting on or replicating? Like, COHEN says it is. About his own rules of thumb: "This is an operation fraught with many dangers (1977).

In addition to the rules of thumb, you need to know what you are measuring and doing with your data in real life and what is a big deal for what you are measuring.

So, is a 30% success rate a big deal? Or, when is a small numeric effect actually a big-deal real life effect?

WHO and the Bill and Melinda Gates foundation think the malaria vaccine has big real world potential. Malaria is awful. It kills more kids than adults (see below). Over 200 million cases in the world with almost half a million deaths, annually. For a problem of this magnitude, a possible 30% reduction would be massive.

Look at the shaded confidence interval! Well done, NPR!

Aside from my big effect size metaphor, this article can also be used in class as it describes a pilot program that takes the malaria vaccine trials from the lab to the messy real world: