Monday, December 28, 2015

Hickey's "The 20 Most Extreme Cases Of ‘The Book Was Better Than The Movie"

Data has been used to learn a bit more about the age old observation that books are always better than the movies they inspire.

Fivethirtyeight writer Walk Hickey gets down to the brass tacks of this relationship by exploring linear relationships between book ratings and movie ratings. 

The biggest discrepancies between movie and book ratings were for "meh" books made into beloved movies (see "Apocalypse Now").

How to use in class:

-Hickey goes into detail about his methodology and use of archival data. The movie ratings came from Metacritic, the book ratings came for Goodreads.
-He cites previous research that cautions against putting too much weight into Metacritic and Good reads. Have your students discuss the fact that Metacritic data is coming from professional movie reviewers and Goodreads ratings can be created by anyone. How might this effect ratings?
-He transforms his data into z-scores.
-The films that have the biggest movie:book rating discrepancies also serve as good examples of influential observations in linear relationships. How might such outliers effect the accuracy of the regression line predicted by this data?
-He does bring up the fact that this is a truncated data set: All of the stories that are included are books that garnered enough attention to be made into a movie.

Monday, December 21, 2015

Esther Inglis-Arkell's "I Had My Brain Monitored While Looking at Gory Pictures. For Science!"

The writer helped out a PhD candidate by participating in his research, and then described the research process for readers. I like this because it is describes the research process purely from the perspective of the research participant who doesn't know what the exact hypothesis is.

This could be useful for explaining what research participation is like for introductory students. You could used it in a methods class by asking the students to figure out why they used the procedures that they did, and what procedures and scales she describes in her narrative. She describes the informed consent, a personality scale (what do you think the personality scale was trying to assess?), and rating stimuli in two ways (brain scan as well as paper and pencil assessment...why do you think they needed both?)

Details to Like:

-She is participating is psychology research (neruo. work that may benefit those with PTSD someday)
-She describes what is entailed when wearing an electrode cap
-Taking baseline measurements, including personality scales
-She provided a rating of the pictures when she was offline
-Mention informed consent in passing

Monday, December 14, 2015

"Guess the Correlation" game

Found this gem, "Guess the Correlation", via the subreddit r/statistics. The redditor who posted this resource (ow241) appears to be the creator of the website. Essentially, you view different scatter plots and try to guess r. Points are rewarded or taken away based on how close you are to true r. The game tallies your average amount of error as well. It is way more addictive than it sounds. I think that accuracy increases with time and experience.

True r for this one was .49. I guess .43, which isn't so bad.

I think this is a good way for statistics instructors to procrastinate. I think it is also a good way to help your students build a more intuitive ability to read scatter plots and predict the strength of linear relationships.

Monday, December 7, 2015

Free, statsy resources available from the Society for the Teaching of Psychology

If you haven't already, please consider joining Teaching of Psychology (Division 2 of APA). Your membership fees help fund plenty of great initiatives, including:

Teaching Statistics and Research Methods: Tips from TOP by Jackson & Grigs

This free e-book is a compilation of scholarship of teaching publications.

Office of Teaching Resources in Psychology's (OTRP) Teaching Resources

This page is divided by topical area in psychology (including Statistics) and includes instructional resources for every topic. Most of the material was created as part of OTRP's Instructional Resource Reward. Among the useful resources are a free booklet containing statistics exercises in both SPSS and R as well as an intense primer on factorial research design.

UPDATE (2/24/16): This new resource provides a number of hands-on activities to demonstrate/generate data for all of the concepts typically taught in intro statistics.  

Project Syllabus 

Project Syllabus is a collection of, in deed, syllabi for a wide variety of different psychology classes. They are sorted by general topic but under each topic, can contain a variety of different classes. Under the "Statistics", a wide variety of syllabi are available, for everything from Intro. to more advanced classes (like Multivariate).

Facebook page for Society for the Teaching of Psychology

This is the friendliest, most helpful Facebook page you will ever come across. Folks post questions about teaching, requests for ideas for activities, syllabi, text book recommendations, etc. Typically, they are quickly flooded with responses from their peers in the trenches.

Monday, November 30, 2015

Explaining the replication crisis to undergraduates

If you are unaware, Noba Project is a collaboration of many, many psychology instructors who create and make freely available text books as well as stand-alone chapters (modules) that cover a wide variety of psychology topics. You can build a personalized text book AND access test banks/powerpoints for the materials offered.

Well, one of the new modules covers the replication crisis in psychology. I think it is thorough treatment of the issue and appropriate for undegraduates.

Monday, November 23, 2015's Football Freakanomics

EDIT: All of this content appears to have been removed from If anyone has any luck finding it, please email me at

The NFL and the statistics folks over at Freakonomics got together and made some...learning modules? Let's call them learning modules. They are interactive websites that teach users about very specific questions related to football (like home field advantage, instances when football player statistics don't tell the whole story about a player/team, whether or not firing a head coach improves a failing team, the effects of player injury on team success, etc.) and then answer these questions via statistics.

Most of the modules include interactive tables, data, and videos (featuring the authors of Freakanomics) in order to delve into the issue at hand.

For example:

The Home Field Advantage: This module features a video, as well as a interesting interactive map that illustrates data about the exact sleep lost experienced by teams that travel from one coast to another and the teams that have to travel the most during seasons. This module uses data to demonstrate that home field advantage does exist, but it also describes all of the factors that may cause home-field advantage (player sleep quality/stadium noise/etc.). This opens up a discussion of multivariate statistics and co-variates.

How to use in your class: Engaging examples of real life questions that can be answered via data collection and analysis.

Monday, November 16, 2015

Neighmond's "Why is mammogram advice still such a tangle? Ask your doctor."

This news story discusses medical advice regarding dates for recommended annual mammograms for women.

Of particular interest for readers of this blog: Recommendations for regular mammograms are moving later and later in life. Because of the very high false positive rate associated with mammograms and subsequent breast tissue biopsies. However, women who have a higher probability (think genetics) are still being advised to have their mammograms earlier in life. Part of the reason that these changes are being made is because previous recommendations (start mammograms at 40) were based on data that was 30-40 years old (efficacy studies/replication are good things!). Also, I generally love counter-intuitive research findings: I think they make a strong argument for why research and data analysis are so very important.

I have blogged about this topic before. This piece by Christy Ashwanden contains some nice graphs and charts that demonstrate that enthusiastic preventative care to detect cancer (including breast cancer) isn't necessarily saving any lives. Another piece that touches on this topic is Sharon Begley's extensive article about how more medicine isn't necessarily better medicine. I use this article in my online class as a discussion prompt. And my online class is aimed at adult, working students who are RNs and are earning their BSNs, and they always have some great responses to this article (generally, they view it favorably).

Thursday, November 12, 2015

Come work with me.


I wanted to post a blog about a job opportunity that available in my department here at Gannon University. Currently, we are seeking a tenure-track assistant professor who specializes in clinical or counseling psychology and would be interested in teaching theories of personality, psychological assessment, and other specialty undergraduate courses.

Gannon is a true undergraduate institution. We teach a 4/4 course load, typically with two and sometimes three unique teaching preps.

I started at Gannon in 2009. In that time, I've received $1000s of dollar in internal grant funding to pursue my work in the scholarship of teaching. In addition to supporting the scholarship of teaching, Gannon provides internal support so that faculty can create global education opportunities as well as service learning opportunities for our students. For instance, one of my colleagues is currently writing a proposal for a History of Psychology class that would include an educational trip to Europe. Another colleague will be teaching his Psychology of Poverty class for the first time in the Spring. This class includes a requirement of 30 service learning hours spent at local not-for-profits that serve the poor in our community.

I've also been able to pursue more traditional research opportunities, and the expectations for such research are in line with a university that focuses so much on undergraduate education.

I would say that I and happy here, have a very good work/life balance (I am married with a toddler and another baby on the way), and fairly compensated for my work. I really like the department that I work in and Gannon provides many leadership and committee opportunities that further enhance my work life.

Erie, PA, is either a small large town or a large small town (~100,000 people). It has most of the amenities that you could want (cool microbrewery scene, fun downtown, lots of outdoorsy fun as we're right on Lake Erie, mall, zoo) and the cost of living is very low. We're also ~two hours away from Pittsburgh, Buffalo, and Cleveland, if you feel like getting out of town.

If you are interested in learning more about the position, click here.

Monday, November 9, 2015

Smith's "Rutgers survey underscores challenges collecting sexual assault data."

Tovia Smith filed a report with NPR that detailed the psychometric delicacies of trying to measure the sexual assault rates on a college campus. I think this story is highly relevant to college students. I also think it also provides an example of the challenge of operationalizing variables as well as self-selection bias.

This story describes sexual assault data collected at two different universities, Rutgers and U. Kentucky. The universities used different surveys, had very different participation rates, and had very different findings (20% of Rutgers students met the criteria for sexual assault, while only 5% of Kentucky students did).

Why the big differences?

1) At Rutgers, students where paid for their participation and 30% of all students completed the survey. At U. Kentucky, student participation was mandatory and no compensation was given. Sampling techniques were very different, which opens the floor to student discussion about what this might mean for the results. Who might be drawn to complete a sexual assault survey? Who is enticed by completing a survey for compensation? How might mandatory survey completion effect college students' attitudes towards a survey and their likelihood to take the survey seriously? Is it ethical to make a survey about something as private as sexual assault mandatory? Is it ethical to make any survey mandatory?

2) Rutgers used a broader definition of sexual assault. For instance, one criteria for sexual assault was having a romantic partner threaten to break up with you if you didn't have sex with them. Jerk move? Absolutely. But does should this boorish behavior be lumped into the same category as rape? Again, this bring up room for class discussion about how such definitions may have influenced the research findings. How can we objectively, sensitively define sexual assault?

Here is an additional news story on the survey out of University of Kentucky. Here is more information about Rutgers' survey (you can take a look at the actual survey on p. 44 of this document).

Monday, November 2, 2015

Barry-Jester, Casselman, & Goldstein's "Should prison sentences be based on crimes that haven't been committed yet?"

This article describes how the Pennsylvania Department of Corrections is using risk assessment data in order to predict recidivism, with the hope of using such data in order to guide parole decisions in the future.

So, using data to predict the future is very statsy, demonstrates multivariate modeling, and a good example for class, full stop. However, this article also contains a cool interactive tool, entitled "Who Should Get Parole?" that you could use in class. It demonstrates how increasing/decreasing alpha and beta changes the likelihood of committing Type I and Type II errors.

The tool allows users to manipulate the amount of risk they are willing to accept when making parole decisions. As you change the working definition of a "low" or "high" risk prisoner, a visualization will start up, and it shows you whether your parolees stay out of prison or come back.

From a statistical perspective, users can adjust the definition of a low, medium, and high risk prisoners and then see how many 1) people who are paroled and reoffend (Type II error: False negative) versus 2) people who are denied parole but wouldn't have reoffended (Type I error: False positive). When you adjust the risk level (below, in Column 2) and then see your outcomes (below, in column 3), it really does reflect on the balance between power and confidence.

Here, I have set the sliding scale so that there is a broad range for designating a prisoner as "Medium Risk". As such, you have 23% of paroled prisoners landing back in jail and 17% of your unparoled prisoners sitting in jail even though they wouldn't have re-offended. As we expand our range of "significance" (here, prisoners we parole), we increase the possibility of false positives (here, folks who re-offend) but have a smaller amount of false negatives
Meanwhile, if you have very stringent standards, you will have fewer false positive (only 10% of those paroled will re-offend) but then you have a lot more false negatives (people denied parole who wouldn't have re-offended). 

Friday, October 30, 2015

r/faux_pseudo's "Distribution of particles by size from a Cracker Jack box

I love my fellow Reddit data geeks over at r/dataisbeautiful. Redditor faux_pseudo created a frequency chart of the deliciousness found in a box of Cracker Jacks.

I think it would be funny to ask students to discuss why this graph is misleading (since the units are of different size and the pop corn is divided into three columns). You could also discuss why a relative frequency chart might provide a better description. Finally, you could also replicate this in class with Cracker Jacks (one box is an insufficient n-size, after all) or try it using individual servings of Trail Mix or Chex Mix or order to recreate this with a smaller, more manageable sample size.

Also, as always, Reddit delivers in the Comments section:

Monday, October 26, 2015

Orlin's "What does probability mean in your profession?"

Math with Bad Drawings is a very accurately entitled blog. Math teacher Ben Orlin illustrates math principles, which means that he occasionally illustrates statistical principles. He dedicated one blog posting to probability, and what probability means in different contexts.

He starts out with a fairly standard and reasonable interpretation of p

Then he has some fun. The example below illustrates the gap that can exist between reality and reporting.

And then how philosophers handle probability (with high-p statements being "true").

And in honor of the current Star Wars frenzy:

And of Orlin's Twitter followers, JP de Ruiter, came up with this gem about p-values:

Monday, October 19, 2015

Barry-Jester's "What A Bar Graph Can Tell Us About The Legionnaires’ Outbreak In New York" + CDC learning module

Statistics afficionados over at fivethiryeight applied statistics (specifically, tools used by epidemiologists) to the Summer of 2015 outbreak of Legionnaires' Disease in New York. This story can be specifically used in class as a way of discussing how simple bar graphs can be modified as to display important information about the spread of disease.

This news story also includes a link to a learning module from the CDC. It takes the user through he process of creating an Epi curve. Slides 1-8 describe the creation of the curve, and slides 9-14 ask questions and provide interactive feedback that reinforce the lesson about creating Epi curves.

Graphs are useful for conveying data, but even one of our out staples, the bar graph, can be specialized as the share information about the way that disease spread.

1) Demonstrates statistics being used in a field that isn't explicitly statisticy.
2) A little course online via the CDC for your students to learn to make epi curves.

Monday, October 12, 2015

U.S. Holocaust Mueseum's "Deadly medicine, creating the master race" traveling exhibit

Alright. This teaching idea is pretty involved. It is bigger than any one instructor and requires interdepartmental effort as well as support from The Powers that Be at your university.

The U.S. Holocaust Museum hosts a number of traveling exhibits. One in particular, "Deadly Medicine: Creating the Master Race", provides a great opportunity for the discussions of research ethics, the protection and treatment of human research subjects, and how science can be used to justify really horrible things.

I am extraordinarily fortunate that Gannon University's Department of History (with assistance from our Honors program as well as College of the Humanities, Education, and Social Sciences) has worked hard to get this exhibit to our institution during the Fall 2015 semester. It is housed in our library through the end of October.

How I used it in my class: My Honors Psychological Statistics class visited the exhibit prior to a discussion day about research ethics. In preparation of the discussion day, they also read the US Department of Health and Human Service's list of individuals who fall under protected class status, listened to a news story about recent revelations regarding WWII-era research on mustard gas using American soldiers who belonged to minority groups, and read a description of the Hoffman Report and the APAs cooperation in development of interrogation techniques used during Operation Iraqi Freedom.

The discussion prompts my students generated were largely about war time research ethics, and the consensus was that even during war time, research ethics still need to be enforced.

Highlights from the discussion in my class:

-Students discussed how a understanding of the social circumstances surrounding these unethical research decisions was critical for understanding how such choices could be made. For instance, prejudice in America was far more acceptable during WWII than today. Post-9/11 America was not very tolerant of anyone who didn't fully support the president. How much freedom did German doctors have to deviate from the ultimate solution?

-My students also got into an interesting discussion on whether or not it would be ethical to analyze the data that came out of Nazi research. Some students argued that if the data could be used to gain insight into the conditions so loathed by the Nazis. As such, any research findings could be a glimmer of good coming out of an awful situation. Other students returned to what they have learned about research ethics and argued that since informed consent was not gained and research participation was not voluntary, such data was completely tainted. Another student brought up the fact that they have a sibling that would have probably been labeled "undesirable" by the Nazi regime and that they would want any data related to their sibling destroyed because they would feel that such data would put their sibling on display.

-Discussion of how the Department of Health and Human Services could be strengthened to avoid future ethical problems. Suggestions included a clearer definition of minimum risk and examples of minimum risk across a broad array of situations as well as better power for fining for unethical research studies.

-A broad discussion as to whether or not the argument that research must be conducted "For the Greater Good" is ever a sound argument or a reason for research ethics to be ignored.

Frankly, it was an awesome discussion.

How this exhibit can supplement a research methods class:

1) The main thrust of this display really is eugenics, in particular, the elimination of people with any perceived or real mental disorders, ranging from epilepsy to low IQs to behavioral problems.

I think this makes this exhibit of particular interest to psychology majors, as these are the very groups that many of our students wish to serve. It is sickening contrast to see how many groups that have currently have protected class status (for research purposes) were the exact groups targeted and exterminated by the Nazis.

2) The use of science to protect unconscionable choices. The display begins by describing how the eugenic movement took Darwin's original work and turned it into an argument for a) the creation of a master race as well as b) the dehumanizing of anyone who didn't fit the description of the master race. All of this was backed up by science and by renowned scientists from this period. This is a good way of introducing why ethical review boards are necessary but also how they can be limited by certain regimes that have their own agendas and end goals. It also might be a good exercise to show students current standards for IRB approval and contrast this with the horrible things that were done during the Holocaust.

3) This also serves as an important tool for social psychology. It conveys how laws and rules were enacted over time that eventually made it possible for large groups of people to dehumanize and murder a perceived sub-human class of people. This display also describes the persuasive tools used by the Nazis in order to justify this behavior (better use of resources by not providing for members of society unable to care for themselves).

Practical concerns for getting this exhibit to your university:

If you look at the logistics for the exhibit, it is pretty involved to bring this to your campus (application material here), both financially as well as the physical hosting and securing of the exhibit. The exhibit consists of a bunch of portable walls that tell the story of the use of eugenics and the study of eugenics by the Nazis. It contains a few flat-screen TVs with documentaries and witness testimony.

I think that it might be a nice opportunity to bring together faculty in biology, history, political science, psychology, philosophy, etc. as well as student groups (centers for social concern, Hillel, history clubs, ethics/philosophy clubs, Psi Chi) that may be interested in assisting with hosting duties/fees. Student involvement is especially important as the exhibit requires volunteer docents to curate the exhibit.

Monday, October 5, 2015

How NOT to interpret confidence intervals/margins of error: Feel the Bern edition

This headline is a good example of a) journalists misrepresenting statistics as well as b) confidence intervals/margin of error more broadly. See the headline below:

In actuality, Bernie didn't exactly take the lead over Hillary Clinton. Instead, a Quinnipiac poll showed that 41% of likely Democratic primary voters in Iowa indicated that they would vote for Sanders, while 40% reported that they would vote for Clinton.

If you go to the original Quinnipiac poll, you can read that the actual data has a margin of error of +/- 3.4%, which means that the candidates are running neck and neck. Which, I think, would have still been a compelling headline. 

I used this as an example just last week to explain applied confidence intervals. I also used this as a round-about way of explaining how confidence intervals are now being used as an alternative/compliment to p-values. 

Monday, September 28, 2015

Aschwanden's "Science is broken, it is just a hell of a lot harder than we give it credit for"

Aschwanden (for did an extensive piece that summarizes that data/p-hacking/what's wrong with statistical significance crisis in statistics. There is a focus on the social sciences, including some quotes from Brian Nosek regarding his replication work. The report also draws attention to Retraction Watch and Center for Open Science as well as retractions of findings (as an indicator of fraud and data misuse). The article also describes our funny bias of sticking to early, big research findings even after those research findings are disproved (example used here is the breakfast eating:weight loss relationship).

The whole article could be used for a statistics or research methods class. I do think that the p-hacking interactive tool found in this report could be especially useful illustration of How to Lie with Statistics.

The "Hack your way to scientific glory" interactive piece demonstrates that if you fool around enough with your operationalized variables, you can probably, eventually, find a way to support your argument (AKA p-hacking). This example uses the question of whether to economy does better under Republican or Democratic leadership to prove a point. Within the tool, you can operationalize both leadership and measures of economic health in a number of different ways (via the check boxes on the left), resulting in a number of different outcomes.

 This is whole article is useful in the classroom simply as a way to discuss the shortcomings of p-values (with the interactive piece) but I think the whole article is accessible to undergraduates, especially as it is littered with embedded links that provide greater information to previous research scandals and debates. 

Monday, September 21, 2015

Correlation example using research study about reusable shopping bags/shopping habits

A few weeks ago, I used an NPR story in order to create an ANOVA example for use in class. This week, I'm giving the same treatment to a different research study discussed on NPR and turning it into a correlation example.

A recent research study found that individuals who use reusable grocery store bags tend to spend more on both organic food AND junk food.

Here is NPR's treatment of the researchHere is a more detailed account of the research via an interview with one of the study's authors. Here is the working paper that the PIs have released for even more detail. 

The researchers frame their findings (folks who are "good" by using resuable bags and purchasing organic food then feel entitled to indulge in some chips and cookies) via "licensing", but I think this could also be explained by ego depletion (opening up a discussion about that topic).

So, I created a little faux data set that replicates the main finding: Folks who use reusable bags have high rates of both organic and junk food purchasing. I decided to replicate the findings as a very simple correlation. This data set represents one year's worth of grocery store spending on Junk Food and Organic Food among reusable bag users.  I created this data set using Richard Landers' data generator website.

$ on Junk Food$ on Organic Food

This radio story let's you discuss cutting edge consumer psychology research. It let's you break up your classroom time with an NPR radio story. It provides another example for data analysis in which your students know the end result, which I think is appropriate for an introductory correlation lecture. Additionally, this story is an example of wide spread data collection/data mining, as the data in question (real data, not my faux data) came from consumer loyalty grocery store cards. 

Monday, September 14, 2015

Mersereau's "Wunderground Uses Fox News Graphing Technique to Boast Forecast Skills"

Mersereau, writing for Gawker website The Vane, provides another example of How Not To Graph. Or How To Graph As To Not Lie About Data But Make Your Data Look More Impressive Than Is Ethical.

Weather Underground (AKA Wunderground, weather forecasting service/website) was bragging about it's accuracy compared to the competition. At first glance (see below), this graph seems to reinforce the argument...until you take a look at the scale being used. The beginning point on the X axis is 70, while the high point is 80. So, really, the differences listed probably don't even approach statistical significance.

This story, somewhat randomly, also includes some shady graphs created by Fox News. I don't understand the need for the extra Fox News graphs, but they also illustrate how one can create graphs that have accurate numbers but still manage to twist the truth.

Monday, September 7, 2015

Dayna Evans "Do You Live in a "B@%$#" or a "F*%&" State? American Curses, Mapped"

Warning: This research and story includes every paint-peeling obscenity in the book. Caution should be used when opening up these links on your work computer and you should really think long an hard before providing these links to your students. However, the research I'm about to describe 1) illustrates z-scores and 2) investigated regional usage of safe-for-the-classroom words like darn, damn, and gosh.

So, a linguist, Dr. Jack Grieve decided to use Twitter data to map out use of different obscenities by county of the United States. Gawker picked up on this research and created a story about it. How can this be used in a statistics class? In order to quantify greater or lessor use of different obscenities, he created z-scores by county and illustrated the difference via a color coding system. The more orange, the higher the z-score for a region (thus, greater usage) while blue indicates lesser usage. And, there are three such maps (damn, darn, and gosh) that are safe for use in class:

Fine Southern tradition of, frankly, giving a damn.

Northern Midwest prefers "Darn"...

...while Tornado Alley likes "Gosh". And New Jersey/NYC/Long Island/Boston doesn't like any of this half-assed swearing.

How to use in class? Z-scores, use of archival Twitter data. You can also discuss how mode of data collection effects outcomes. They collected data via Twitter. Is this a representative sample? Nope! Does the data reflect on the way that people speak or the way that people self-present?

Monday, August 31, 2015

Caitlin Dickerson's "Secret World War II Chemical Experiments Tested Troops By Race"

NPR did a series of stories exposing research that the U.S. government conducted during WWII. This research exposed American soldiers to mustard gas for research purposes. In some instances, the government targeted soldiers of color, believing that they had tougher/different skin that would make them more resistant to this form of chemical warfare.

Here is the whole series of stories (from the original research, exposed via Freedom of Information Act, to NPR working to find the effected veterans).

None of the soldiers ever received any special dispensation or medical care due to their involvement. Participants were not given the choice to discontinue participation without prejudice, as recalled below by one of the surviving veterans:

"We weren't told what it was," says Charlie Cavell, who was 19 when he volunteered for the program in exchange for two weeks' vacation. "Until we actually got into the process of being in that room and realized, wait a minute, we can't get out of here." 
Cavell and 11 other volunteers were locked inside a gas chamber with mustard gas piping inside. Blocks of ice sat on shelves overhead with fans blowing across them to increase the humidity in the room, which intensified mustard gas's effects on the body. After an hour, the officer released six of the men back to their barracks. Cavell and five others were told to stay put. 
Inside the chamber, Cavell's skin started to turn red and burn in the places where he sweat the most: between his legs, behind his neck and under his arms. Blisters that eventually increased to the size of half dollar coins started to grow in the same places. At the end of the second hour, the officer ordered Cavell back to his barracks and to continue wearing his gas-saturated uniform. 
Cavell, now 88 years old, says the officer threatened him and the other test subjects: If they told anyone about their knowledge or participation in the experiments, they would receive a dishonorable discharge and be sent to military prison at Fort Leavenworth, Kan. 
"They put the fear of God in just a bunch of young kids," he says.

This story illustrates why we need Institutional Review Boards. For me, at least, this story hits home because we aren't teaching our students about the horrors Nazi experimentation. Instead, we're talking about terribly racist research that the U.S. government conducted on The Greatest Generation during WWII.

Needless to say, this is a good example for a discussion about research ethics. In particular, I think this opens up discussion of whether or not informed consent can really be obtained from active duty military personnel, the importance of outside approval of research that is this potentially dangerous, and the importance of the informed consent.

Here is a link to the original study.

Monday, August 24, 2015

McFadden's "Frances Oldham Kelsey, F.D.A. Stickler Who Saved U.S. Babies From Thalidomide, Dies at 101"

This obituary for Dr. Frances Oldham Kelsey that tells an important story about research ethics, pharmaceutical industries, and the importance of government oversight in the drug creation process (.pdf here).

Dr. Kelsey, receiving the President's Award for Distinguished Federal Civilian Service (highest honor given to federal employees)

Dr.  Kelsey was one of the first officials in the United States to notice (via data!) and raise concerns about thalidomide, the now infamous anti-nausea drug that causes terrible birth defects when administered to pregnant women. The drug was already being widely used throughout the Europe, Canada, and the Middle East to treat morning sickness, but Dr. Kelsey refused to approve the drug for widespread use in the US (despite persistent efforts of Big Pharm to push the drug into the US market). Time proved Dr. Oldham Kelsey correct (clinical trials in the US went very poorly), and her persistence, data analysis, and ethics helped to limit the negative effects of the drug on American children.

I like this example because it shows that government safe guards can work and that we need data in order to provide evidence to protect public health. Also, the hero of this story is a woman...with her the 1960s...which makes her efforts all the more impressive.

Monday, August 17, 2015

ANOVA example using Patty Neighmond's "To ease pain, reach for your play list."

I often share news stories that illustrate easy-to-follow, engaging research that appeals to undergraduates. For the first time, I'm also providing a mini data set that 1) mimics the original findings and 2) provides an example of ANOVA.

This story by Patty Neighmond, reporting for NPR, describes study investigating the role of music in pain reduction. The study used three groups of kids, all recovering from surgery. The kids either 1) listened to music, 2) listened to an audio books, or 3) sat with noise-cancelling ear phones for 30 minutes. The researchers found that kids in both the music and audio book experienced pain reduction levels comparable to over-the-counter pain medication while the control group enjoyed no such benefits.

And the research used the 10-point FACES scale, allowing for a side discussion about how we collect data from humans who don't have the best vocabularies, communication skills, or English comprehension.

This study can also be used as a way to explain ANOVA. The researchers didn't use ANOVA (and the data I provide below IS NOT the original data), but the original design and findings do provide us with three levels of a factor as well as some significant post-hocs.

Here is a data set generated via Richard Landers' data set generator and modified as to use a 1-10 FACES scale used in the original research (yes, the n-size is small for this design). It approximates the original findings: Statistically significant ANOVA, with post-hocs that demonstrate that Audio Book and Music conditions do not differ significantly but that participants in theses two groups report significantly less pain that the control condition).

Audio Book
Music Control
5 5 4
6 4 8
7 4 7
2 7 6
6 6 10
3 4 6
4 6 10
8 4 8
5 3 5
4 5 6

Now, if you have the students listen to the news story first, they are going to know the results. I don't think this is a bad thing, necessarily. I think this could be used when first introducing students to ANOVA as an example with training wheels (interesting news story to listen to for four minutes followed by a statistical exercise for which they know the outcomes). Additionally, I teach statistics to a lot of non-psychology majors (nursing, pre-physical therapy, pre-physician assistant) so a health psychology example fits nicely into my classes.

This could also be used as a prompt a discussion about turning this study into a more complicated one-way ANOVA (include more levels of your factor, like 30 minutes of video games or TV) as well as how you could turn the original study into a factorial ANOVA (by including severity of surgery, length of hospital stay, age of child, etc.). 

Friday, August 14, 2015

Memes pertaining to the teaching of statistics, research methods, and undergraduate advising.

For those who teach statistics, research methods, and psychology major advisers. Some of these have been posted before. Some of these have not. I created all of them except for the first one.

Additionally, I created a bunch of Psychology Advising memes as I am currently editing the "Advising" portion of my rank and tenure application.

Monday, August 10, 2015

Kristopher Magnusson's "Understanding the t-distribution and its normal approximation"

Once again, Kristopher Magnusson has combined is computer programming and statistical knowledge to help illustrate statistical concepts. His latest interactive tool allows students to view the t-curve for different degrees for freedom. Additionally, students can view error rates associated with different degrees of freedom.

Note that the critical region is one-tailed with alpha set at .05. If you cursor around the critical region, you can set the alpha to .025 to better illustrate a two-tailed test (in terms of the critical region at which we declare significance). 

Error rates when n < 30

Error rates when n > 30

This isn't the first time Kristopher's interactive tools have been featured on this blog! He has also created websites dedicated to explaining effect size, correlation, and other statistical concepts.

Sunday, August 2, 2015

Aarti Shahani's "How will the next president protect our digital lives?"

I think that it is so, so important to introduce statistics students to the big picture of how data is used in their every day lives. Even with all of the material that we are charged with covering in introduction to statistics, I think it is still important to touch on topics like Big Data and Data Mining in order to emphasize to our students how ubiquitous statistics are in our lives. 

In my honors section, I assign multiple readings (news stories, TED talks, NPR stories) prior to a day of discussion devoted to this topic. In my non-honors sections of statistics and my online sections, I've used electronic discussion boards to introduce the topic via news stories. I also have a manuscript in press that describes a way to introduce very basic data mining techniques in the Introduction to Statistics classroom.

That's why I think this NPR news story is worth sharing. Shahani describes and provides data (from Pew) to argue that Americans are worried about the security of their data and goes on to suggest that the federal government (in particular, our next president) need to strengthen the security surrounding our data. One option is to levy heavy fines against companies that get hacked. 

She also touches on the fact that the internet and our data collection have expanded and grown far faster than our legal systems ability to keep up with it or for the free-market to generate a solution to this problem. 

I like this piece because it touches on just how enormous the problem is and how many stake holders there are. I also like that the support the importance of government intervention via data that demonstrates that Americans are very concerned about the privacy of their data. 

Monday, July 27, 2015

One article (Kramer, Guillory, & Hancock, 2014), three stats/research methodology lessons

The original idea for using this article this way comes from Dr. Susan Nolan's presentation at NITOP 2015, entitled "Thinking Like a Scientist: Critical Thinking in Introductory Psychology". I think that Dr. Nolan's idea is worth sharing and I'll reflect a bit on how I've used this resource in the classroom. (For more good ideas from Dr. Nolan, check out her books, Psychology, Statistics for the Behavioral Sciences, and The Horse that Won't Go Away (about critical thinking)).

Last summer, the Proceedings of the National Academy of Sciences published an article entitled "Experimental evidence of massive-scale emotional contagion through social networks". Gist: Facebook manipulated participants' Newsfeeds to increase the number of positive or negative status updates that each participant viewed. The researchers subsequently measured the number the positive and negative words that the participants used in their own status updates. They found significance and, thus, support that emotional contagion/spreading of emotions can occur via Facebook.

I assure you, your students are very familiar with Facebook. Additionally, emotional contagion theory is pretty easy to understand. As such, the article itself is accessible and interesting to students.

Pedagogically, there are three statistical lessons (p-value vs. effect size, how to create a misleading graph, informed consent in the age of Big Data).

Lesson 1. This article is a good example of p-values vs. effect size. For one analysis, the p-value is gorgeous (< .001) and the effect size is itty-bitty (.001). Guess why? N = 689,003. Here is an additional resource if you want to delve further into p-values vs. effect sizes with your students.

Lesson 2. The figures overemphasize the findings because they didn't scale they y-axis at zero. See below.
If you look at the graph in passing, the differences seem...reasonable, especially the one in the upper right-hand quadrant. However, if you look at the actual numbers on the y-axis, the differences are not of great practical value. This disparity also harkens back to the p vs. effect size issues, as the practical (yet-statistically-significant) implications of these findings are unimpressive.

Lesson 3. The researchers sorta-kinda obtained informed consent. How did they go about doing so? The researchers argued that by agreeing to the the Facebook Terms of Service, users are providing consent to experimental research. However, the participants 1) were not aware that they were part of this particular study and 2) were never given the option to opt out of the study. Several good pieces have been written on this aspect of the study, including The Washington Post (.pdf here) and Wall Street Journal (.pdf here). Of particular interest here (to me, at least) is the disconnect between a bunch of industry statisticians crunching numbers and manipulating user experiences, which they do EVERY DAY as part of their job, versus how social psychologists perceive the exact same practices (and place a greater emphasis on research ethics and participant rights). This all resulted in the"Editorial Expression of Concern and Correction" that have been appended to the source article. Facebook also claims to have changed their research process as a result of this study (described in the WSJ article).

How I used this in my classes:

I used the graph during the first few weeks of class as an example of how NOT to create a bar graph. I also used the study itself as a review of research methods, IV, DV, etc.

I used this as a discussion point in my Honors Psychological Statistics class (the topic of the week's discussion was research ethics and this was one of several case studies) and it seemed to engage the students. We discussed User Agreements, practical ways to increase actual reading of User Agreements, and whether or not this was an ethical study (in terms of data collection as well as potential harm to participants).

In the future, I think I'll have my students go over the federal guidelines for informed consent and compare those standards to Facebook's attempt to gain informed consent via user agreement.

Aside: I learned about this article and how to use it in the classroom at the National Institute for the Teaching of Psychology. Guys, go to NITOP. Totally worth your time and money. Also, family friendly if you and your partner and/or kids would like a trip to Florida in early January. NITOP is accepting proposals for various submission until October 1st (with some exceptions). 

Monday, July 20, 2015

"Correlation is not causation", Parts 1 and 2

Jethro Waters, Dan Peterson, Ph.D., Laurie McCollough, and Luke Norton made a pair of animated videos (1, 2) that explain why correlation does not equal causation and how we can perform lab research in order to determine if causal relationships exist.

I like them a bunch. Specific points worth liking:

-Illustrations of scatter plots for significant and non-significant relationships.

Data does not support the old wive's tale that everyone goes a little crazy during full moons.

-Explains the Third Variable problem.
Simple, pretty illustration of the perennial correlation example of ice cream sales (X):death by drowning (Y) relationship, and the third variable, hot weather (Z) that drives the relationship.
-In addition to discussing correlation =/= causation, the video makes suggestions for studying a correlational relationship via more rigorous research methods (here violent video games:violent behavior).
Video games (X) influence aggression (Y) via the moderator of personality (Z)

In order to test the video game hypothesis without using diary/retrospective data collection, the video describes how one might design a research study to test this hypothesis.

-Finally, at the end of  the video, they provide citations to the research used in the video. You could take this example a step further and have your students look at the source research.

Special thanks to Rajiv Jhangiani for introducing me to this resource!

Tuesday, July 14, 2015

Free online research ethics training

Back in the day, I remember having to complete an online research ethics course in order to serve as an undergraduate research assistant at Penn State.

I think that such training could be used as an exercise/assessment in a research methods class or an advanced statistics class.

NOTE: These examples are sponsored by the American agencies and, thus, teach participants about American laws and rules. If you have information about similar training in other countries (or other free options for American researchers), please email me and I will add the link.

Online Research Ethics Course from the U.S. Health and Human Service's Office of Research Integrity.

Features: Six different learning modules, each with a quiz and certificate of completion. These sections include separate quizzes on the treatment of human and animal test subjects. Other portions also address ethical relationships between PIs and RAs and broader issues of professional responsibility when reporting results.

National Institute of Health's Protecting Human Subject Participants

Features: Requires free registration. Four different, quizzed learning modules. This one includes some lessons about the historical need for IRBs, the Belmont Report, the need to respect and protect our participants. This also provides a certificate at the end of the training.

Monday, July 13, 2015

Dread Fall 2015 Semester

It's coming, guys.

But let's get ahead of it. I thought I would re-share some resources that you may want to consider working into your curriculum this year. I picked out a few lessons and ideas that also require a bit of forethought and planning, especially if they become assessment measures for your class.

Center for Open Science workshops:

As previously discussed on this blog, COS offers free consultation (face-to-face or online) to faculty and students in order to teach us about the open framework for science. They provide guidance about more more traditional statistical issues, like power calculations and conducting meta-analysis in addition to lessons tailored to introducing researchers to the COS framework.

Take your students to an athletic event, talk about statistics and sports:

I took my students to a baseball game and worked some statsy magic. You can do it, too. If not a trip to the ballpark, an on-campus or televised athletic event will work just fine.

Statistic/research method discussions:

I have just added a new searchable label to this blog, discussion prompts. Such items are typically news stories that are great for generating conversations about statistic/research related topics. This is a topic to think about prior the beginning of the semester in case you want to integrate one of the prompts into your assessments (discussion boards, small group discussion, discussion days, reflective writing, etc).

Formal, free, online research ethics training:

This post describes two free resources that provide thorough research ethics training (as well as assessment quizzes and training completion certificates, two things that are especially helpful if you want to use these as a class assignment).

First day of class persuasion:

Statistics are everywhere and a part of every career. We know this. Here are some resources for trying to convince your students.

Monday, July 6, 2015

Ben Blatt's "Bad Latitude" and "You Live in Alabama. Here’s How You’re Going to Die"

Ben Blatt of Slate mined through Center for Disease control data in order to provide us with 13 different maps of the United States and mortality information for each state. Below, information on disproportionately high cases of death in each state.

While the maps are morbid and interesting, the story behind the maps (read the story here about how data can be easily misrepresented by maps) make this a good example of how easily data can be distorted.

The story along with the maps unveils several issues that statisticians/researchers must consider when they are presenting descriptive statistics. In this instance, Blatt had to sort through the data to eliminate the most common causes of death (heart disease, cancer, etc.) in order to uncover unique data for each state.

Relatedly, he highlights the fact that "disproportionately" does not mean "most":

"But this map—like many maps which purport to show attributes meant to be “distinct” or “disproportionate”—can be misleading if not read properly. For one thing, you cannot make comparisons between states. Looking at this map, you probably would not guess that Utah has the sixth-highest diabetes rate in the country. Diabetes just happens to be the one disease that affects Utah most disproportionately. Louisiana has a higher diabetes death rate than any state, but is affected even more disproportionately by kidney disease."

Additionally, some of the maps highlight the difference between median and mean splits...

"It should be noted that each state’s rate is compared with the national average, not the median. That’s why it’s possible for 30 states to have more deaths than the national average."

Anyway, I think this could be useful in class because the title of the maps are interesting, but I think it also forces students to think about some unique issues in statistics.

Or, put bluntly and in a manner inappropriate for the undergraduate classroom:

Monday, June 29, 2015

Scott Ketter's "Methods can matter: Where web surveys produce different results than phone interviews"

Pew recently revisited the question of how survey modality can influence survey responses. In particular, this survey used both web and telephone based surveys to ask participants about their attitudes towards politicians, perceptions of discrimination, and their satisfaction with life.

As summarized in the article, the big differences are:

"1) People expressed more negative views of politicians in Web surveys than in phone surveys." 

"2) People who took phone surveys were more likely than those who took Web surveys to say that certain groups of people – such as gays and lesbians, Hispanics, and blacks – faced “a lot” of discrimination." 

"3) People were more likely to say they are happy with their family and social life when asked by a person over the phone than when answering questions on the Web."  

The social psychologist in me likes this as an example of the Social Desirability Bias. When speaking directly to another human being, we report greater life satisfaction, we are more critical of politicians, and more sympathetic towards members of minority groups.

The statistician in me thinks this is a good example for discussing sources of error in research. Even a completely conscientious research using valid, reliable measures may have their data effected based on how it is collected. It might be interesting to asks students to generate lists of research topics (say, market research about cereal preference versus opinions about abortion) and whether students think you could get "true" answers via telephone or web surveys. What is a "true" answer, how could we evaluate or measure this? How could we come up with an implicit or behavioral measure of something like satisfaction with family life, then test which survey modality is most congruent with an implicit or behavioral measure? What do students think would happen if you used face-to-face interviews or paper and pencil surveys in a classroom of people completing surveys?

Additionally, you can't call yourself a proper stats geek unless you follow Pew Research Center on either Twitter (@pewresearch) or on Facebook . So many good examples of interesting data!