Ages ago, I shared how I teach ANOVA at a conceptual level. I describe within and between group variance using beliefs about the human role in climate between and within different religious groups. This data is now old. And it described global warming, not climate change, which is a crucial language distinction. So you  can imagine my delight when Pew recently released  updated and improved data investigating this issue.  In my attempt to keep the mood light when discussing an example featuring 1) religion, 2) climate change, and 3) politics, I ask students to think about how many different opinions are probably represented around their family's Thanksgiving table. Despite having much in common as a family, like, perhaps, geography, shared stories, and religion, there are still a lot of within-group differences of opinion. This leads to a discussion about people of different religions having between and within group differences of opinion regarding beliefs about global cl...

In our stats classes, we need MANY examples to convey both theories behind and the computation of statistics. These examples should be memorable. Sometimes, they can make our students laugh, and sometimes they can be couched in research. They should always make our students think. In this spirit, I've collected three small examples from the Pew Research Center's  Race and Ethnicity  archive (I hope to update with more examples as time permits). I don't know if any data collection firm is above reproach, but Pew Research is pretty close. They are non-partisan, they share their research methodology, and they ask hard questions about ethnicity and race. If you use these examples in class, I think that it is crucial to present them within context: They illustrate statistical concepts, and they also demonstrate outcomes of racism.   1. "Most Blacks say someone has acted suspicious of them or as if they weren't smart" Lessons: Racism, ANOVA theory: between-group dif...

Today's post is one, small, to-the-point example of sampling bias. On May 27, 2020, my family and I were awaiting lift-off for the (subsequently grounded) NASA/SpaceX launch. To no one's surprise, I was following NASA on Twitter during the hoopla, and I noticed this Tweet: https://twitter.com/NASA/status/1265724481009594369 And I couldn't help but think: That is some sampling bias. Admittedly, their sample size is very impressive, with over 54K votes. But this poll went out to a bunch of people who love NASA so much that they follow it on Twitter.  What is a less biased response to this question? As always, Pew Research Center had my back. 58% of Americans responded that they definitely/probably weren't interested in traveling into space: https://www.pewresearch.org/fact-tank/2018/06/07/space-tourism-majority-of-americans-say-they-wouldnt-be-interested/ If you want to expand upon this example in class, you could ask your students to Google around for information on the ...

Create an account with Pew Research, and you can download some of their data sets, including a) syntax files, b) detailed methodology, and c) codebook, including detailed screenshots of what the survey felt like to participants.  I think there are three ways to use this in class: -Show your students what proper data documentation looks like -Get some data, run some analyses -Get some data, look up Pew's reports based on the data, see if you can replicate the findings. How to Properly Document Your Research Process. Pew documents the hell out of these data sets. Included are: Syntax files: Methodology: Surveys, featuring the questions but also screenshots of the user experience: Get some data, run some analyses. MY FIRST EVER FACTOR ANALYSIS EXAMPLE, y'all. Per the methodology documentation, Pew creates its own scales. Within this data set (American Trends Panel Wave 34), they use several scales to measuring attitudes about medical treatments. ...

This is for my psychometric instructors. (Glorious, beloved) Pew Research Center compared participant behavior when they have to answer the same question in either a) forced-choice or b) check-all format. Here are the links to the short report and to the long report . What did they find? Response options matter, such that more participants agreed with statements when they were in the forced-choice format. See below: So, this is interesting for an RM class. I also like that the short report explained the two different kinds of question responses. The article also explores a variety of reasons for these findings, as well as other biases that participants exhibit when responding to questionnaires:

You know how every few months, someone Tweets about stock photos that are generated when you Google "professor"? And those photos mainly depict white dudes? See below. Say "hi" to Former President and former law school professor Obama, coming it at #10, several slots after "novelty kid professor in lab coat". Well, Pew Research decided to quantify this perennial Tweet, and expand it far beyond academia. They used Machine Learning to search through over 10K images depicting 105 occupations and test whether or not the images showed gender bias.  How you can use this research in your RM class: 1. There are multiple ways to quantify and operationalize your variables . There are different ways to measure phenomena. If you read through the report, you will learn that Pew both a) compared actual gender ratios to the gender ratios they found in the pictures and b) counted how long it took until a search result returned the picture of a woman for a given j...

Question-wording matters, friends! This example shows how question order and question-wording can affect participant response. This is a good example for all of your research methods and psychometrics students to chew on. Pew Research asked people if they believed in evolution . They did so in three different ways, which lead to three different response patterns. 1) Prior to asking about evolution, the asked whether or not the participant believes in God. 2) Asked participants if they believed in evolution. If they said "yes", they asked the participant whether or not they believe that a higher power guides evolution. 3) They asked participants if they believed in evolution and gave participants three response options:     a) Don't believe in evolution.     b) Believe in evolution due to natural selection.     c) Believe in evolution guided by a higher power. Responses to Option 1: Responses to Options 2. and 3. Oh, the classroom discus...

Pew Research created a survey that asks participants to identify news statements as opinions or facts. They had 5000+ complete this survey AND you can complete the survey and see your results.  Description of quiz AND research methodology! An example question from the survey. This one made me think of Ron Swanson. How to use in Stats/RM: 1. A good way of introducing the truism "The plural of anecdote isn't data.". Facts and opinions aren't always the same thing, and distinguishing between the two is key to scientific thinking. Ask your student think of of objective data that could prove or disprove these statements. Get them thinking like researchers, developing hypotheses AND operationalizing those hypotheses. 2. At the end of the quiz, they describe your score in terms of percentiles. Specifically, in terms of the percentages of users who scored above and below you on the quiz items. 3. You can also access Pew's report of their survey f...

Pew Research Center  is an excellent source for data to use in statistics and research methods classes. I have blogged about them before (look  under the Label pew-pew! ) and I'm excited to share that Pew is starting up a series of videos dedicated to research methods. The new series will be called Methods 101 . The first describes sampling techniques in which weighing is used to adjust imperfect samples as to better mimic the underlying population. I like that this is a short video that focuses on one specific aspect of polling. I hope that they continue this trend of creating very specific videos covering specific topics. Looking for more videos? Check out Pew's YouTube Channel . Also, I have a video tag for this blog. 3/25/2018 They have posted their second video, this one on proper wording for research questions as to avoid jargon and bias.

Sometimes, we need to convince our students that taking a statistics class changes the way they think for the better. This example demonstrates that one seemingly simple skill, interpreting a scatter plot, is tougher than it seems. Pew Research conducted a survey on scientific thinking in America ( here is a link to that survey ) and they found that only 63% of American adults can correctly interpret the linear relationship illustrated in the scatter plot below. And that 63% came out a survey with multiple-choice responses! How to use in class: -Show your students that a major data collection/survey firm decided that interpreting statistics was an appropriate question on their ten-item quiz of scientific literacy. -Show your students that many randomly selected Americans can't interpret a scatter plot correctly. And for us instructors: -Maybe a seemingly simple task like the one in this survey isn't as intuitive as we think it is!

This interactive tool from Pew research illustrates left and right skew as well as median and longitudinal data. The x-axis indicates how politically consistent (as determined by a survey of political issues) self-identified republicans and democrats are across time. Press the button and you can animate data, or cut up the data so you only see one party or only the most politically active Americans. http://www.people-press.org/2014/06/12/section-1-growing-ideological-consistency/#interactive The data for both political part goes from being normally distributed in 1994 to skewed by 2014. And you can watch what happens to the median as the political winds change (and perhaps remind your students as to why mean would be the less desirable measure of central tendency for this example). I think it is interesting to see the relative unity in political thought (as demonstrated by more Republicans and Democrats indicating mixed political opinions) in the wake of 9/11 but more politicall...

This finding is super-duper intuitive: A positive, strong correlation exists between national per capita income and rates of internet access and smartphone ownership within that nation. Because it is intuitive, it makes a good example for your class when you teach correlation to your baby statisticians. This graph is  more engaging than your average graph because the good people at Pew made it interactive. You can see which country is represented by which dot. You can also see regional trends as the countries are color-coded by continent/region. For more context and information on this survey, see this more extensive report on the relationship between smartphone/internet access and economic advancement . This report further breaks down technology usage by education level, age, individual income, etc. This data is also useful for demonstrating the distribution of wealth in the world and variability that exists among countries in the same region/on the same continent,

I am a big fan of Pew Research Center . They collect, share, and summarize data about a wide variety of topics. In addition to providing very accessible summaries of their findings, they also provide more in-depth information about their data collection techniques, including original materials used in their data collection and very through explanations of their methods. One topic they collect Pew studies is religion and attitudes (religious and secular) held by people of different religions. And it got me thinking that I could use their data in order to explain within and between group differences at the heart of a conceptual understanding of ANOVA. Specifically, Pew gathered data looking at between-group differences in beliefs in global climate change by religion ... Chart created by Pew Research ... and belief in climate change within just Catholics, divided up by political affiliation. Chart created by Pew Research The questionnaires differed slightly for the...

This report from Pew  highlights the differences in opinions between the average American versus members of the American Association for the Advancement of Science (AAAS). For various topics, this graph reports the percentage of average Americans or AAAS members that endorse each science related issues as well as the gap between the two groups. Below, the yellow dots indicate the percentage of scientists that have a positive view of the issue and the blue indicate the same data for an average American. If you click on any given issue, you see more detailed information on the data. In addition to the interactive data, this report by Funk and Rainie summarizes the main findings. You can also access the original report of this data  (which contains additional information about public perception of the sciences and scientists). This could be a good tool for a research methods/statistics class in order to convince students that learning about the rigors of the scientif...

Pew Research went around the globe and asked folks in 40 different countries if a variety of different behaviors qualified as "Unacceptable", "Acceptable", or "Not a moral issue". See below for a broad summary of the findings. Summary of international morality data from Pew The data on this website is highly interactive...you can break down the data by specific behavior, by country, and also look at different regions of the world. This data is a good demonstration of why graphs are useful and engaging when presenting data to an audience. Here is a summary of the data from Pew.  It nicely describes global trends (extramarital affairs are largely viewed as unacceptable, and contraception is widely viewed as acceptable). How you could use this in class. 1) Comparison of different countries and beliefs about what is right, and what is wrong. Good for discussions about multiculturalism, social norms, normative behaviors, the influence of religion ...