Sunday, January 19, 2020

NYT American dialect quiz as an example of validity and reliability.


Ameri-centric teaching example ahead: Have your students take this quiz, and the internet will tell them which regions of US talk the same as them. Use it to teach validity.

Longer Version:

The NYT created a very pretty version ( of a previous available quiz (
) that tells the user what version of American English they speak. The prediction is based upon loads and loads of survey data that studies how we talk. It takes you through 25 questions that ask you how your pronounce certain words, and which regional words you use to describe certain things.

Here are my results:

Indeed, I spent elementary school in Northern Virginia, my adolescence in rural Central PA, college at PSU, and I now live in the far NW corner of PA. As this test indeed picked up on where I've lived and talked, I would say that this is a valid test based just on my useage.

How to use in class: Ask every student to take this quiz and reflect on the validity and reliability of the scale and use it as an example to explain validity and reliability. Here is how I used it:

Social but not educational benefit:
I asked my students to take the quiz prior to learning about validity in my I/O class. They enjoyed the quiz. Before class started that day, I could hear them discussing their results. Which entailed talking about where they grew up, where their parents grew up, where they spent that summer, etc. So, it was actually a fine "getting to know you" exercise for the first week of the semester.

Educational benefit: Use it to illustrate the 90 million different kinds of validity and reliability:

Content Validity: The test studied both pronunciations as well as different regional terms for the same thing (Water bubbler? Really, Wisconsin?). This demonstrates two dimensions of content for the way we talk: Accents or different ways to say the same word, as well as different words to say the same thing.
Criterion Related Validity: Did the quiz successful guess where your student is from?
Face validity: This test is very high in face validity.
Predictive Validity: If this test was given to someone else, would it accurate measure where they are from? Do your students think that there are shifts in the way people talk as a society over time that would hurt the predictive validity? Like old people who say dungarees instead of jeans?
Convergent Validity: Do you think this test might also predict where your parents grew up? Or where your roommates or close friends grew up? Do you think it can predict where you live now or just where you grew up (for example, did I speak differently during my six years of graduate school in the Midwest? Does having a husband with a thick Long Island accent change how I talk now vs. how I talked in high school).
Reliability: This test claims to measure where you are from.

Do you think that is true? Or does it test where you live now? Is this consistent overtime for a military brat who moves all over the world? Or someone who grew up in the South and then moves to New England?

I think it might be fun to discuss the discriminant validity of individual questions as well.  For instance, my use of the word "hoagie" to describe a specific kind of sandwich really pined me as a Pennsylvanian.

Friday, January 10, 2020

I started an I/O teaching blog.

This may or may not interest my Teaching of Statistics crowd, but I've started a new blog filled with amusing, one-off ideas and examples for teaching Industrial/Organizational Psychology. Like this gem about using data to detect bias in workplace staffing. But in Paw Patrol:

I'm teaching it this Spring and have realized that while compiling my own ideas for teaching I/O, I might as well share them with everyone as well.

You can find the new blog here.

Wednesday, January 8, 2020

Spring 2020 Syllabuses

Hi, all-

Happy 400th blog post to me!

Anyway, here and there, people request my syllabuses. I'm happy to share them. I'm also sort of proud to share them this semester. Per ideas I've seen circulating on Twitter and at conferences, I've made three big changes this year:

1. I made them more visually appealing. Not just for aesthetic's sake, though, but to better organize information.
2. I've included a section that features a) hints for success from past students and b) my hints for success/extra resources.
3. I more directly addressed the fact that our students can face crises during the semester and I am here to help them with their problems. I also included a list of additional resources for assisting students.

Thank you to everyone who has every shared their good ideas for improving our syllabuses.

Anyway, here are my Honors Psych Stats syllabus and my Online Psych Stats syllabus.