Part 1 Part 2 Part 3

The Racer Game Part 2

Collecting, Organizing and Drawing Conclusions from Data

Much like the behind-the-scenes experts on Top Gear who run data-gathering tests to spin into easy-to-digest soundbites for everyday audience members, you now have the start of the information you need to put together some statistics based on the Racer game.

Introduction

In this lab, your goal is to complete a research study based on the Racer game. Just like last time, each car can be raced multiple times on a track and the time to complete each race will be recorded. This time, however, there are three drivers (just think of them as our rather roguish Top Gear hosts!) – Classic, Cruiser, and HotRod. After you race the cars, you can analyze the data in the table.

2A: Designing an Experiment

Many people assume that statistics focuses on calculations. Have you spent time in Las Vegas, Atlantic City, or even just a plain old casino? It’s easy to think that betting on something like, say, the World Cup or Super Bowl comes down to a bunch of math problems pitted against each other – when in reality, there are a lot of other variables that figure in when calculating those odds. Most real-world statisticians spend much more time properly collecting data (or cleaning/organizing data) than they do on the calculations. For example, before collecting any data for an experiment involving the Racer game, a researcher should look at all of the steps below:

Develop a hypothesis about the population of all races. Here are a few examples:

• I believe that on average, the Classic car is faster than the HotRod Car.

• I believe that on average, the HotRod car will complete a race faster than the Cruiser car.

• I believe that at least one of the average Finish Times is different for the Classic, Cruiser, or HotRod cars.

Determine how you will collect your sample

• What will you consider as an observation? For example, will it be a player, an individual race, a pair of races, a car, or a track?

• How many samples will you collect?

• What is the explanatory variable? Is it categorical or quantitative?

• What is the response variable? Is it categorical or quantitative?

• How will randomization be incorporated into the data collection?

• What specific data needs to be recorded with each race?

• What graphs or statistical methods do you plan to use once you collect the data? In the example below, we discuss using a paired t-test.

Identify potential confounding variables. Once again, it’s key to take into account possible confounding variables (variables that the researcher did not include in the study but that might be connected to both the independent variable and the dependent variable), to determine if the car speeds are in fact truly different. Here’s an example: some racetracks are longer than others. We want to hold the track constant for everyone in your experiment to ensure that the type of track does not bias our results. Are there any other potential confounding variables that might get in the way of determining which car is faster?

2B: Collecting Data

Unlike the cavalier folks in Top Gear, who thumb their noses at the rules, researchers need clear protocols (instructions) so that the experimental data is collected using the same process every time. Documentation of these protocols also enables other researchers to verify the validity of the data. If collecting data as a class, clearly identify the exact protocols and timeframe for collecting data. For example, here are the protocols for data collected in an earlier class:

All data was collected during class time on a computer in the statistics lab room.

Students logged onto the site and went to the Racer game.

Make sure you enter the correct GroupID and PlayerID.

Click on the Paired Test button on the main menu after logged in.

Each student in the class will race exactly two times on the Oval Track. Prior to racing the cars, students were randomly assigned which car they would race first.

Click on the left box to select the car you will race first.

Then select the car you will race second.

Then select the SELECT TRACK button.

Then click the START button.

Use the arrow keys to race the car.

NOTE: If you are unable to play the game, either try a different browser or go to the game website directly here.

2C: Examining the Data

Before you start any calculations, it is essential to examine your data. Ask yourself the following questions for the data collected in the Racer game:

Were there any players who did not properly follow protocols? Did they play the game an incorrect number of times? Did they use the wrong cars or tracks? Can their data be included?
Were there any outliers or skewed data? How does this influence our analysis? Should these outliers be removed?
Some people have a natural talent for video games; to others, it’s a completely foreign concept. How can we account for player skill level?
How do we account for the influence of order in which cars are raced? For example, would you expect players to perform better after practicing with the first car?
How does the variation in effect size (the difference between means) compare to the random variation in our data?
Are there any other issues that may cause our data to be unreliable or invalid?
What conditions are required to conduct a statistical analysis? How can you evaluate whether these conditions were met? If the sample size is small, it is particularly important to verify that the conditions are satisfied. What should you do if the needed conditions are not met?

Watch the following video to see an example of how data cleaning can influence a statistical analysis.

2D: Analyzing Data

Before we analyze the data your class has collected, let’s look at a sample dataset, called sample2. Get started by completing the questions below to make sure you understand how this app works, then analyze your class data.

To use the app, start with the following settings then answer the questions below.

Group ID: sample2

Track: OvalTrack

X Variable: Body

Y Variable: FinishedTime

Color by: Body

Check: Show Summary Statistics

Instructors Note: Go to faculty resources to access student data

2E: Get Curious

Develop your own experimental design by rewriting the steps from Section 2B: Collecting Data. List the specific protocols that need to be followed for your new experiment. Make sure that you also address all three steps in Section 2A: Designing an Experiment section.

Using your class data, answer the seven questions identified in Section 2c: Examining the Data.

Model Conditions:

a. List the assumptions required to conduct a hypothesis test.

b. Describe how you evaluated whether these conditions were met.

c. What should you do if the needed assumptions are not met?

If you collected data properly, you can use the Racer App above to analyze it. Use the data to conduct a hypothesis test to determine if there is a difference in the population average finishing time of the Classic and HotRod cars.

a. Provide the test statistic and corresponding p-value.

b. Give the 95 percent confidence interval and provide an interpretation.

c. What conclusions can you draw from your class experiment? Clearly state your overall recommendation to someone wanting to win a race on the oval track.

Watch the video titled "Deception at Duke": https://www.youtube.com/watch?v=eV9dcAGaVU8.

a. Briefly summarize the video.

b. Discuss how this Racer Lab is related to the errors discussed in this video.

c. What policies should universities set in place to ensure this type of error does not occur again?

How much do your conclusions depend on the data cleaning that was done?

Could different response variables (Finish Time versus Top Speed versus Time to 30) result in different conclusions? Why or why not?

If our p-value is large, can we be confident that the average finish times are the same for both cars?

Assume your class repeated this experiment

a. Would you expect to get similar results?

b. Would you expect to get identical results?

c. If we collected data from another sample of players from the population, do you think it would affect our conclusion?

d. What exactly is the population for this study? Did we collect a true simple random sample?

Many published articles show a p-value without discussing whether or not they modified or removed data. Does a small p-value guarantee that the study was done correctly? Why or why not?

Does the hypothesis test or confidence interval provide more helpful information for our study? In particular, does the hypothesis test and p-value provide a good measure of the difference between average car speeds?

The game on this page is simplistic. More advanced versions of this game, allowing for numerous types of experiments, are available here: Racer Game and here: RaceKart Game. What research questions could be addressed with these advanced games that could not be addressed with the simplistic game on this page?

2F: Data Literacy

Read the brief article discussing the ASA’s statement on statistical significance and p-values: https://doi.org/10.1080/00031305.2016.1154108. Pick one of the six principles and write one to two paragraphs discussing how this principle relates to this Racer Lab activity. More than one of the principles can be applied to this lab.

Read the article discussing Amy Cuddy’s research: https://www.nytimes.com/2017/10/18/magazine/when-the-revolution-came-for-amy-cuddy.html. Discuss how this Racer Lab is related to the errors discussed in this article.

Go back to Part 1 Continue to Part 3

2C: Examining the Data

2D: Analyzing Data

2E: Get Curious

2F: Data Literacy

Dataspace

Data Stories

Stats Games

Other Links

Questions?