Part 1 Part 2 Part 3

The Racer Game Part 3

Introduction

In this lab, your goal is to complete a research study based on the Racer game. Each car you select can be raced multiple times on a track and the time to complete each race will be recorded. After you race the cars, the data will be available for you to analyze.

3A: Designing an Experiment

Many people assume that statistics focuses on calculations. However, most real-world projects spend much more time properly collecting data (or cleaning/organizing data) than they do on the calculations. For example, before collecting any data for an experiment involving the Racer game, the following steps should be addressed:

Develop a hypothesis about the population of all races. Here are a few examples:

• I believe that on average, Bayes car is faster than the Gauss Car

• I believe that on average, the Gauss car will complete a race faster than the Nightingale car.

• I believe that at least one of the average Finish Times is different for the Bayes, Gauss, or Nightingale cars.

Determine how you will collect your sample

• What will you consider as an observation? For example, will it be a player, an individual race, a pair of races, a car, or a track?

• How many samples will you collect?

• What is the explanatory variable? Is it categorical or quantitative?

• What is the response variable? Is it categorical or quantitative?

• How will randomization be incorporated into the data collection?

• What specific data needs to be recorded with each race?

• What graphs or statistical methods do you plan to use once you collect the data? In the example below, we discuss using a paired t-test.

Identify potential confounding variables. In order to evaluate whether the car speeds are truly different, we should take into account possible confounding variables (variables that the researcher did not include in the study but that might be connected to both the independent variable and the dependent variable). For example, some racetracks are longer than others. We want to hold the track constant for everyone in your experiment to ensure that the type of track does not bias our results. Are there any other potential confounding variables that might get in the way of determining which car is faster?

3B: Collecting Data

Researchers need clear protocols (instructions) so that the experimental data is collected using the same process every time. Documentation of these protocols also enables other researchers to verify the validity of the data. If collecting data as a class, clearly identify the exact protocols (instructions) and timeframe for collecting data. For example, here are the protocols for data collected in an earlier class:

All data was collected during class time on a computer in the statistics lab room.

Students logged onto the site and went to the Racer game.

Make sure you enter the correct GroupID and PlayerID.

Click the Paired Test button on the main menu.

Each student in the class will race exactly two times on the Oval Track. Prior to racing the cars, students were randomly assigned which car they would race first.

Use the pull down menu to select which car you will race first.

Then select which car you will race second.

Then select the Track.

Then click the Continue button.

Use the arrow keys to race the car.

NOTE: If you are unable to play the game, either try a different browser or go to the game website directly here.

3C: Examining the Data

Before conducting any calculations, it is essential to examine your data. For example, the following questions should be asked for any data that was collected in the Racer game.

Were there any players who did not properly follow instructions? Did they play the game an incorrect number of times? Did they use the wrong cars or tracks? Can their data be included?
Were there any outliers or skewed data? How does this influence our analysis? Should these outliers be removed?
How can we account for player skill level?
How do we account for the influence of order in which cars are raced? For example, would you expect players to perform better after practicing with the first car?
How does the variation in effect size (the difference between means) compare to the random variation in our data?
Are there any other issues that may cause our data to be unreliable or invalid?
What conditions are required to conduct a statistical analysis? How can you evaluate whether these conditions were met. If the sample size is small, it is particularly important to verify that the conditions are satisfied. What should you do if the needed conditions are not met?

Watch the following video to see an example of how data cleaning can influence a statistical analysis.

3D: Analyzing the Data

Before we analyze the data your class has collected, we will first look at a sample dataset, called sample2. Complete the questions on the left to make sure you understand how this app works, then analyze your class data.

To use the app below, start with the following settings then answer the questions to the left.

Group ID: sample2

X Variable: Body

Y Variable: Finish Time

Check: Add Boxplot

3E: Get Curious

Assume you are designing an experiment and write out specific protocols that would need to be completed. Make sure that you address all three steps in the Designing an Experiment section.

Using your class data, answer the seven questions identified in the Examining the Data section.

The game on this page is simplistic. More advanced versions of this game, allowing for numerous types of experiments are available here: and here: What research questions could be addressed with the advanced games that could not be addressed with the simplistic game on this page?

If the data collection was conducted properly, you can use the app to analyze your data. However, if your data was not properly collected, your instructor will provide a cleaned dataset for you. Use the data to conduct a test and make a decision about whether there is a difference in the population average finishing time of the Bayes and Gauss cars.

a. Provide the test statistic and corresponding p-value:

b. Give the 95% confidence interval and provide an interpretation:

c. What conclusions can you draw about your classes experiment? Clearly state your overall recommendation to someone wanting to win a race on the oval track.

Does a small p-value guarantee that one type of car will always be faster than the other? Why or why not?

How much do your conclusions depend upon the data cleaning that was done?

Could different response variables (Finish Time vs Top Speed vs Time to 30) result in different conclusions? Why or why not?

Does the hypothesis test or confidence interval provide more helpful information for our study? In particular, does the hypothesis test and p-value provide a good measure of the difference between average car speeds?

If this experiment was repeated, do you expect to get similar results? Do you expect to get identical results?

a. Emphasize the difference between sample statistics and population parameters.

b. Talk about having a different sample of players.

c. What exactly is the population for this study? Did we collect a true simple random sample?

If our p-value is large, can we be confident that the average finish times are the same for both cars?

3C: Examining the Data

3D: Analyzing the Data

3E: Get Curious

3F: Data Literacy

Dataspace

Data Stories

Stats Games

Other Links

Questions?