Skip to main content

Transforming your data: A historical example

TL:DR: Global water temperature data from <1940 was collected by sailors collecting buckets of water from the ocean and recording the temperature of their bucket water. But some recorded data was rounded (thanks, Air Force!). Then, researchers had to transform their data.

^Go to the 3 minute mark to see the bucket-boat-water-temperature technique in action

Here is the original research, published in Nature. NPR covered the research article. Reporter Rebecca Hersher didn't discuss the entire research paper. Instead, she told the story of how the researchers discovered and corrected for their flawed ocean water temperature data.

This story might be a little beyond Intro Stats, but it tells the story of messy, real archival data used to inform global climate change and b) introduces the idea of data transformations. Below, I will highlight some of the teaching items.

Systematic bias: The data were all flawed in the same way as they were transcribed without any data to the right of the decimal point. Which is not ideal, but you can correct it.

Archival data: Sometimes, the data you need already exists somewhere. We have only been tracking weather for 100 years. But now we have this opportunity for older data, right, so we can better understand the larger cycle of global water temperatures.

I also like how they could tell that something was happening in their data but didn't know WHY something was happening in their data.

Well-annotated code books are good. That Air Force PDF uncovered by the researchers' friends is also a good example of why you must maintain a codebook and extensive notes on how you treat your data.

Research is so much more than numbers. This is the story of graduate student Duo Chan and his efforts to make his archival data as accurate as possible. It sounds like it was a pain in the ass. Such is science.

Measurement techniques. We can accurately measure water temperature now. I assume that it involves sharks with lasers on their heads. I don't know, but I can assume that we've moved on from  The Bucket Method, which was their best at the time.

Data transformations can be tricky to explain to the novice, like transformations to make data less skewed. This could be a simple way to introduce the topic. Yes, I know that actual transformation is much more involved than this, but it is a simple way to introduce the topic.

Popular posts from this blog

Ways to use funny meme scales in your stats classes

Have you ever heard of the theory that there are multiple people worldwide thinking about the same novel thing at the same time? It is the multiple discovery hypothesis of invention . Like, multiple great minds around the world were working on calculus at the same time. Well, I think a bunch of super-duper psychology professors were all thinking about scale memes and pedagogy at the same time. Clearly, this is just as impressive as calculus. Who were some of these great minds? 1) Dr.  Molly Metz maintains a curated list of hilarious "How you doing?" scales.  2) Dr. Esther Lindenström posted about using these scales as student check-ins. 3) I was working on a blog post about using such scales to teach the basics of variables.  So, I decided to create a post about three ways to use these scales in your stats classes:  1) Teaching the basics of variables. 2) Nominal vs. ordinal scales.  3) Daily check-in with your students.  1. Teach your students the basics...

Leo DiCaprio Romantic Age Gap Data: UPDATE

Does anyone else teach correlation and regression together at the end of the semester? Here is a treat for you: Updated data on Leonardo DiCaprio, his age, and his romantic partner's age when they started dating. A few years ago, there was a dust-up when a clever Redditor r/TrustLittleBrother realized that DiCaprio had never dated anyone over 25. I blogged about this when it happened. But the old data was from 2022. Inspired by this sleuthing,  I created a wee data set, including up-to-date information on his current relationship with Vittoria Ceretti, so your students can suss out the patterns that exist in this data.

Tyler Vigen's Spurious Correlations

Tyler Vigen has has created  a long list of easy-to-paste-into-a-powerpoint graphs that illustrate that correlation does not equal causation. For instance, while per capita consumption of cheese and number of people who die by become tangled in their bed sheets may have a strong relationship (r = 0.947091), no one is saying that cheese consumption leads to bed sheet-related death. Although, you could pose The Third Variable question to your students for some of these relationships). Property of Tyler Vigens, http://i.imgur.com/OfQYQW8.png Vigen has also provided a menu of frequently used variables (deaths by tripping, sunlight by state) to help you look for specific examples. This portion is interactive, as you and your students can generate your own graphs. Below, I generated a graph of marriage rates in Pennsylvania and consumption of high fructose corn syrup. Generated at http://www.tylervigen.com/