3  Analysis Primer

You have created and implemented a study, and now you have data. Congratulations! The following guide describes how to move forward with the process of analyzing your data. Remember, at the outset of your project you had a guiding research question, along with one or more hypotheses. Let these be your guide as you begin to explore your data. While the following advice is not meant to be exhaustive, it should serve as a means to plan your analysis, to protect you against common pitfalls, etc.

3.1 Preparing For Your Analysis

3.1.1 Required Software

The official software that we will be using as a class is The R Project for Statistical Computing. R is an advanced, capable, and free software environment for analyzing data. It can be used to prepare, analyze, and graph your data.

Note that we will also be using Positron. This is another free piece of software that makes working with R easier.

Both of these are available on campus computers, including in the common areas of the library and the computer labs on campus. You may optionally install both pieces of software on your own computer if you would prefer to use it. Note that this requires either a Windows PC, a Mac, or a Linux computer. At this time you cannot use R or Positron with a Chromebook or iPad. If you do not own a compatible computer, you will need to use a computer on campus to complete your assignments.

Note

If you haven’t updated your personal computer in a long time, you may have to first apply those operating system updates before your computer will be able to run R or Positron. Please consider doing these updates outside of class before we begin using these tools in class.

To install these packages, you must install them in the following order using the links below:

1. Install R
2. Install Positron

Tip

If you need help installing software on your own computer, please reach out to Dr. Van Horn. Capital’s IT department on campus may be able to help as well.

3.1.2 Convert Your Hypotheses Into The Language Of Your Dataset

This is THE most important step to begin your data analysis.

You have one or more hypotheses, and your project was designed to get data that will allow you to answer those hypotheses. The first step you should do is to sort through your data and determine how you will use this new treasure trove to answer your questions. For each hypothesis, determine which portions of your dataset will be needed. How this process will unfold will differ for every research project, but here is an example:

Imagine you were interested in the effect of social media use on both anxiety and depression in young adults. Your hypotheses are:

1. Increased social media usage leads to increased levels of anxiety
2. Increased social media usage leads to a decrease in depression

Imagine that you used a survey to ask:

• FIVE questions about social media usage, each of which is meant to estimate how much time the participant spends on social media. These are marked as columns U1-U5 in the dataset below.
• FIVE questions, each meant to assess the level of anxiety in your participant. These are marked as columns A1-A5 in the dataset below
• FIVE questions, each meant to assess the level of depression in your participant. These are marked as columns D1-D5 in the dataset below

Your resulting dataset would likely look something like this (let’s assume you have three participants), wherein each column reflects a single question from your survey, and each row represents a participant’s responses on each question:

U1	U2	U3	U4	U5	A1	A2	A3	A4	A5	D1	D2	D3	D4	D5
3	4	1	5	3	4	3	4	4	5	2	3	2	4	4
2	4	3	1	5	5	5	4	5	3	4	5	3	2	3
3	4	4	5	2	1	4	2	1	3	5	5	3	4	1

Converting your hypotheses into the language of your dataset would like the following for this fictitious study:

• Time on social media (usage): The two hypothesis above both require a measure of how much time each participant spends online. In this study, five questions (U1-U5) were asked to measure time spent online. Assuming these were all Likert questions on the same scale, a composite score called “time” could be created, which is the sum of each person’s response on the U1-U5 questions (see the section on Composite Scores / Likert Scoring below for details on how to do this). This total score will provide an estimate of each participants time on social media.
• Anxiety: A similar composite estimate called “anxiety” of each person’s anxiety could be created using questions A1-A5
• Depression: A similar composite estimate called “depression” of each person’s depression could be created using questions D1-D5

The resulting spreadsheet, which has been reduced for simplicity, would look like this (again, see the section below titled “Composite Scores / Likert Scoring” for further details on how to complete this step):

U1	…	U5	time	A1	…	A5	anxiety	D1	…	D5	depression
3	…	3	16	4	…	5	20	2	…	4	15
2	…	5	15	5	…	3	22	4	…	3	17
3	…	2	18	1	…	3	11	5	…	1	18

We have now simplified our dataset to three variables of interest: time, anxiety, and depression. These three values can be used to answer our two hypotheses from above. For this imagined study, I would do the following to test these hypotheses:

• Hypothesis 1: Increased social media usage leads to increased levels of anxiety. We would want to compare the “time” and “anxiety” scores in our data to test this hypothesis. Specifically, I would use a Pearson’s Correlation Coefficient (see the section on “Inferential Statistics” below for advice on how to choose a statistical test). If the correlation coefficient was greater than 0, it would appear to confirm that as time online increases, so does anxiety.
• Hypothesis 2: Increased social media usage leads to a decrease in depression. I would use the same test as the previous hypothesis, this time using the “time” and “depression” data. Here, a negative correlation would suggest our hypothesis was supported.

This is just a simple example. But the goal for all projects is the same: take your hypotheses and data and determine which parts of your data will be used to test each hypothesis. The chapter on Statistics will show how to accomplish things like turning U1 - U5 into a composite score like time above.

3.2 Preparing Your Data

3.2.1 Getting a Copy of your Data

Please consult Qualtrics documentation on downloading your data here: Exporting Data with Qualtrics

You will want to export your data as a CSV file (comma separated values). This format can be opened with Excel, SPSS, Google Docs, Apple Numbers, etc. When exporting you have two options:

1. Use numeric values: In this format, answers to true/false, multiple choice, and ratings will displayed as numbers (e.g., True = 1 and False = 2). If you have Likert questions, this option will ensure that your dataset contains numeric values (i.e., columns for Likert questions will contain 1, 2, 3, 4, 5, etc. instead of “Very infrequently,” “Infrequently,” etc.). This is likely the format that you will use for creating Composite Scores.
2. Use choice text: This will give you a data set with properties opposite to the numerical option above. While you likely don’t want to use this file type for your actual analysis, it can be useful to download a separate copy in this format. This will allow you to double check, for example, that a Likert response of 1 = “Infrequently” rather than “Frequently.” You should download a copy of your data in this format to double check each question to ensure you’re treating the numbers correctly.

Tip

You will want to download both versions of your data and put them into your “class folder,” in which all of your analysis files will be contained.

3.2.2 Removing Outliers

As a general rule, you should exercise care when removing outliers. Perhaps most critically, you should never remove participant data based only on a hunch. Possible reasons to remove a participant’s data include:

1. They did not finish the study. Look through your data for participants that did not complete the survey (some/all columns will be empty for those individuals). Note that many participants are likely to have chosen not to respond to at least one of your questions. You should not necessarily remove these individuals from your analysis (though their existence will complicate things for you). You are looking for people who skipped many/all questions. These rows can be removed from the dataset before completing your analysis.
2. They finished the study, but in an unreasonable amount of time. Every Qualtrics dataset comes with an additional column that indicates how long it took each participant to complete the study. Imagine that you expected your survey to take 5 minutes to complete. However, someone finished it in just under 2 minutes. Did they rush through, not thinking through each response? It is difficult to say. While it is tempting to simply remove them, you want to avoid this, as systematically removing fast respondents will bias your results. One possibility is to examine the completion times of all participants, and then calculate the mean and standard deviation of these completion times. If a single participant was faster than the mean completion time by more than 2 standard deviations, you might want to remove their data after examining it more closely. Some can be spotted quite easily with a cursory glance: say most people took 7 minutes to complete the survey, yet one individual completed it in 23 seconds. You’re probably safe to remove that person from your data set :)

If you have doubts, you should contact Dr. Van Horn prior to removing any participants from your dataset.

Very Important Point

If you have used a survey system such as Qualtrics to manage your survey do NOT delete your participants from the data in the system itself. This is often impossible to undo, and you may find that you want a deleted participant’s data back. Instead, download/export your dataset and then remove the offending participant’s data with R during your analysis (instructions are provided in the statistics chapter).

3.2.3 Cleaning Your Data

Prior to doing an analysis it is often necessary to first clean your data. This is sometimes referred to as “data wrangling,” “data munging,” and “data janitorial” work. This process can take many forms—too many to list here—but must be completed prior to doing your analysis. Here are a few examples. You will want to scan through your own data to find similar problems:

• Data inconsistencies: Imagine you asked for each participant’s age in a survey. You intended for them to enter a number, such as 22, 18, or 43. However, someone entered “18 years old.” Another entered “27 y.o.” You will need to fix each mistake to ensure a consistent formatting, such as turning “18 years old” into simply “18”.
• Uniformity: You may have asked a participant to state whether they (1) have college degree, or (2) do not have a college degree. Unfortunately, they have entered “currently in college.” You will need to re-code their response to be choice #2. A carefully designed study will avoid many problems such as this—but if there is a way to provide an unexpected answer, you can count on it happening.
• And then some… There are countless ways that your data will need cleaned up—too many to even think of, let alone list here. If you encounter a problem that likely needs cleaned up, please reach out to Dr. Van Horn for assistance.

Important

This data cleaning work can and should be done in R. Don’t waste your time manually fixing data in a spreadsheet program such as Excel.

3.2.4 Re-coding Data

Now that you can see your actual data, it is likely that you’re not happy with how a particular question turned out. Imagine the following scenario: You had a hypothesis that first, second, third, and fourth year students would have different habits regarding their social media usage. To test that hypothesis, you asked the question, “What is your current year in college?” with the options, “First, Second, Third, Fourth+”. Unfortunately, in your data you discover that while there were many first and third year students, you didn’t receive many responses from second or fourth+ year students. Now what do you do? You might decide to adopt a different, yet similar approach. Perhaps you decide instead to compare “underclassmen” to “upperclassmen,” where underclassmen contains first and second year students, and upperclassmen contains third and fourth year students. What you’ll need to do is take each participant’s actual response and re-code it into your new scheme. However, do not overwrite the original data (you want to keep it around to be able to check for mistakes, or to reverse your coding if you decide to try something else). Here’s what I would do: take the following data set:

Age	Class rank	Facebook user?
19	first year	yes
18	third year	yes
21	third year	no
18	first year	yes

Instead of overwriting the class rank column, insert a new column next to it, and add each participant’s coded response:

Age	Class rank	Coded rank	Facebook user?
19	first year	underclassmen	yes
18	third year	upperclassmen	yes
21	third year	upperclassmen	no
18	first year	underclassmen	yes

Now you can use this new coded column as a variable in your analysis instead of the “class rank” column. As with cleaning your data, the possibilities for needing to code your data are endless. If you think you have a need to do this and are not sure how to accomplish your goal, please reach out to Dr. Van Horn.

3.2.5 Composite Scores / Likert Scoring

Rather than ask a single question, such as “How much anxiety do you feel on the typical day?”, you were advised to ask a series of specific questions that could be combined into a more stable estimate. For this example, you might have asked questions such as, “How often do you lie in bed at night worrying about the next day?,” “How often do you think about what others might be saying about you in public?,” etc. Many existing measures of anxiety and depression, for example, use this approach.

What do you do with the resulting data from such sequences of questions? Well, if you followed my advice, you used an identical Likert scale for each question. Let’s imagine that the scale was from 1=Very Infrequently to 5=Very Frequently. On this scale, a higher number for each response indicates greater levels of anxiety. What you might do in this case is combine all anxiety questions that share this Likert scale into a single score. The simplest way to do this would be to add each participant’s response on each individual question into a single “composite” score. If you had asked five anxiety questions in your survey, each on the same scale from 1-5, then this composite score could range from 5 (if the respondent answered “1” on every question) to 25 (if they answered “5” for every question). Thus, a higher score means greater anxiety. This measurement will be a far more accurate estimate of their overall anxiety than any single question will be.

Thankfully, computers make adding these scores up easy. Hint: if you’re doing this by hand, you’re doing it wrong. Not only does adding up these scores for hundreds of participants equate to pretty awful work, it’s highly error-prone to do this manually. Instead, let your computer do the work for you. Imagine you had asked 5 anxiety questions to three participants, and that this was the data you received:

A1	A2	A3	A4
3	1	3	2
4	3	5	2
4	4	5	3

What you’ll want to do is create a new column in your spreadsheet, give it a useful name such as “anxiety score” and then add up that composite score. Here’s what the above spreadsheet should look like once you’ve finished:

A1	A2	A3	A4	anxiety score
3	1	3	2	9
4	3	5	2	14
4	4	5	3	16

Instructions are provided here for how to accomplish this in R.

Once that composite score is calculated, that value should become the focus of your analysis (both descriptive and inferential) for hypotheses related to each participants’ anxiety level.

3.3 Completing Your Data Analysis

Data analysis, while unique for each group, will broadly consist of two steps. Every group should begin with step 1:

1. Descriptive Statistics
2. Inferential Statistics

While you are being asked to complete these two assignments as separate steps, please be aware that you will ultimately combine all of your analysis work into a single results section in your final paper/poster. The difference between these two forms of statistics is purely academic. When a research project is being analyzed, some combination of these two branches of statistics are completed, and the final result will be a seamless combination of both. Once again, an example can be illustrative:

Imagine that you had the hypothesis that college students spend more time on social media per day than non-students. To test this hypothesis, you ask your participants two questions:

1. Are you currently a college student?
2. How many hours per day do you spend on social media?

The inferential statistical test you would likely use to test this hypothesis would be a t-test. If you remember from your statistics courses, t-tests allow you to determine if the mean of one group (students) is different than another group (non-students). Let’s imagine that you found a significant result. t-tests result in a t-value and a p-value that tell you whether the result is significant or not. However, when you report a t-test in APA format you must also report the mean and standard deviation of each group. These mean and standard deviation values are technically “descriptive” statistics, but they are ultimately combined with your t-test results into a single statement such as this (see Reporting Statistics in APA Format for examples):

College students (M = 3.66, SD = .40) reported significantly higher amounts of time on social media than non-students (M = 3.20, SD = .32), t(1) = 5.44, p < .05.

The reason I am having you complete these two types of analyses in separate steps is purely pragmatic: I just don’t want you to get overwhelmed. It can be very challenging (and sometimes impossible) to know what inferential statistics to conduct without really understanding your dataset. The best way to get to know your dataset is to use descriptive statistics to describe what’s there (and visualize it all with charts where possible).

3.3.1 Descriptive Statistics

Every group should begin here. This is the simplest form of data analysis. The goal with descriptive, or summary statistics is to describe what your data “look like.” Some examples make the goal here more clear:

• How many men/women/non-binary respondents does your sample contain? Count how many of each response you received, and then make a bar chart showing those counts.
• What is the average number of hours your respondents spend on social media?
• For a given Likert question, how many people chose each response option on your Likert scale? For Likert questions, the recommended summary visualization is the “diverging stacked bar cart.”

The goal for each group is to create a variety of summary statistics and accompanying figures. Remember, having visualizations for your analyses is very important–you will need to eventually make a poster, and having figures in your paper will make your job explaining your results much simpler. I have provided a document on Canvas that can be used to help determine what kind of chart to use for different types of data. The document is called “Choosing Graphs - Flow Chart” on Canvas.

How many questions from your dataset should you perform a descriptive analysis on? That depends on each project. But as a general rule, an analysis (and figure) should be completed for all questions that are useful for describing and gaining an understanding of your sample. If you have any questions or concerns about your particular dataset, please contact Dr. Van Horn.

3.3.2 Inferential Statistics

Broadly speaking, the goal of inferential statistics is to use your sample to make inferences about the greater population. If you were interested in the online habits of young adults (your population of interest), you likely gave out a survey to a sample of people from that demographic. While your descriptive statistics allowed you to describe your sample, those descriptive statistics tell you nothing about the greater population—and it is that population that you’re really interested in. Inferential statistics allow you to take your sample and derive conclusions about that population. Put another way, inferential statistics is what you use to test your hypotheses.

Common inferential statistics include things such as t-tests, z-tests, ANOVA, Pearson’s correlation, and Chi-square tests (among others). Deciding which tests to use is unique to each project, so there unfortunately isn’t a simple checklist of things to do that can be written here. However, I have made a document available on Canvas that is designed to help you sort out which statistical test is right for each hypothesis. The document is titled “Choosing Statistical Tests - Flow Chart” on Canvas. Note that most groups have multiple hypotheses, so the flow chart will need to be applied to each hypothesis/data combination. I have also provided numerous examples in the Choosing the Best Statistical Test section to help you complete these tests.

3.4 Choosing the Best Statistical Test

Every research project is unique, and so each analysis strategy will be similarly unique. There are, however, some general strategies for determining what statistical test is appropriate for each hypothesis. Below is an attempt to make this as simple as possible. But keep in mind that every analysis requires thoughtful care for determining what should be done. So while the following advice is meant to help you choose how to proceed, do not just simply choose from this list. If you have questions or doubts, please reach out to Dr. Van Horn.

Important

I have provided these examples to help remind you of days gone past from your statistics course(s). Please use your analysis software to replicate these examples before trying them on your own data! Being able to verify you’re doing things right with these examples will help you feel confident about using these tools on your own project. You will want to conduct these analyses with software because a statistical program will also provide you with a p-value, which you will need when reporting your results.

3.4.1 Chi-squared test (Χ² test)

Data Type	Example(s)
discrete	number of responses (e.g., Likert responses)
nominal	gender, class rank

Typical visualization: bar chart

For our purposes, there are actually TWO chi-squared tests: (1) chi-squared goodness of fit test, and (2) chi-squared test for independence. The chi-squared test (both types) is commonly used to test hypotheses using survey data (although not necessarily so). Broadly speaking, chi-squared tests determine if the counted frequencies in your data differ from what is expected. Chi-square tests, with a little practice, can easily be calculated by hand. The most challenging task is simply counting the number of responses–however, this step can be done quite easily with any statistical program (while the test can be calculated by hand, the actual chi-square test should also be done using software).

3.4.1.1 Example #1

Let’s imagine that we wanted to know if students of different ranks (first, second, third, and fourth year) use Capital’s dining hall with equal proportions. To investigate this, the class rank of each student entering Capital Court is recorded across an entire day. The number of each rank observed is counted, and the fictitious results are:

Rank	Observed # Students
first year	328
second year	313
third year	278
fourth year	157
	1076

It appears at a glance that we observed a different number of students from each group. But is that difference statistically significant? That’s impossible to tell by just looking at the raw counts. The chi-squared goodness of fit test allows us to determine statistical significance. The standard null hypothesis is that we expect equal outcomes in each condition (although the null can be specified such that differences are expected). To determine the expected outcomes, you simply divide the probability space into equal parts. In this case, there are four conditions (each of the class ranks). So if the total probability space is 100% (remember, it is always the case that probabilities must sum to 100%), and there are four conditions, each condition should have 25% of the probability space. To determine the expected number of students from each condition, we take 25% of the total for each condition. The following table contains this additional information:

Rank	Observed # Students	Probability space	Expected # Students
first year	328	0.25	(1076 * .25) = 269
second year	313	0.25	(1076 * .25) = 269
third year	278	0.25	(1076 * .25) = 269
fourth year	157	0.25	(1076 * .25) = 269
	1076		1076

The null model in this case is quite simple: we expect no difference in turnout between different class ranks. Like most null hypotheses, we probably don’t expect this to be true. In this case we might have expected that students in later years turn out in smaller numbers. But does our data tell us to reject the null? In comes the chi-squared test. The test is calculated as follows:

\(\chi^2 = \Sigma \frac{(O_i - E_i)^2}{E_i}\)

where \(O_i\) is the observed value for each condition, \(E_i\) is the expected value for each condition, and \(\Sigma\) means to sum all the fractions. So for our example, the calculation would be:

\(\chi^2 = \Sigma \frac{(O_i - E_i)^2}{E_i} = \frac{(328 - 269)^2}{269} + \frac{(313 - 269)^2}{269} + \frac{(278 - 269)^2}{269} + \frac{(157 - 269)^2}{269} = 67.07063\)

The resulting value of 67.071 is the chi-squared statistic. To know if it is statistically significant we compare that number to a critical value in a chi-squared distribution (remember those tables in the back of your statistics textbooks?). We choose a reasonable alpha level (typically .05), determine the degrees of freedom, and find that critical value in our textbook table (or using a table online).

• degrees of freedom for this test is the number of groups N - 1 = 3
• alpha = .05

The critical value would be 7.81 (remember, you find this number in a chi-squared distribution table such as this one). Our chi-squared statistic is 67.071, which is greater than 7.81. This means our result is significant, we can reject the null hypothesis of equal group sizes, and can say that students of different rank use Capital’s food service disproportionately.

3.4.1.2 Example #2

The chi-squared test for independence can be used to investigate two variables at once. Let’s say you asked two questions in your survey:

1. Do you have a Facebook account (yes/no)?
2. I pay attention to the news (choose from: “Infrequently,” “Sometimes,” “Frequently”)

Imagine we are interested in knowing if Facebook users feel more attentive to the news. Counting up the responses for each question, we can summarize our findings with a 2-by-2 table:

	Infrequently	Sometimes	Frequently
Facebook: Yes	200	150	50	(r1 = 400)
Facebook: No	250	300	50	(r2 = 600)
	(c1 = 450)	(c2 = 450)	(c3 = 100)	Grand Total = 1000

(r1 and r2 are row totals, and c1, c2, c3 are column totals)

The final calculation for this chi-squared test will be the same as the first example. And the logic is still the same: (i) the null hypothesis assumes no difference in outcomes for the different cells in the above table, (ii) you have observed values, and (iii) you calculate expected values. Expected values are calculated in the same logical fashion as the first example, but the presence of row and column totals makes it slightly more complicated. Expected values are calculated as:

\(E_{r,c} = \frac{(n_r \times n_c)}{n}\)

Applying that equation to each combination of rows/columns, we get following table:

	Infrequently	Sometimes	Frequently
Facebook: Yes	(400 * 450) / 1000	(400 * 450) / 1000	(400 * 100) / 1000	(r1 = 400)
Facebook: No	(600 * 450) / 1000	(600 * 450) / 1000	(600 * 100) / 1000	(r2 = 600)
	(c1 = 450)	(c2 = 450)	(c3 = 100)	Grand Total = 1000

Completing those calculations, we can generate the following expected values for our table of outcomes:

	Infrequently	Sometimes	Frequently
Facebook: Yes	180	180	140
Facebook: No	270	270	60

We now have SIX observed values and SIX corresponding expected values. Good news: we apply the same equation from example 1 to calculate our chi-squared value:

\(\chi^2 = \Sigma \frac{(O_i - E_i)^2}{E_i}\)

\(\chi^2 = \Sigma \frac{(O_i - E_i)^2}{E_i} = \frac{(200 - 180)^2}{180} + \frac{(150 - 180)^2}{180} + \frac{(50 - 140)^2}{140} + \frac{(250 - 270)^2}{270} + \frac{(300 - 270)^2}{270} + \frac{(50 - 60)^2}{60} = 16.2\)

• degrees of freedom is (N₁ - 1)(N₂ - 1) = (2 - 1)(3 -1) = 2
• alpha = .05

Using a chi-square distribution table, we can see that our critical value is 5.99. Our chi-square value is greater than this critical value, thus we can reject the null hypothesis. It seems from our data being a Facebook user changes one’s perceived awareness of news.

3.4.2 Pearson’s Correlation

Data Type	Example(s)
continuous	age, hours spent on social media, Big-5 personality trait score

Typical visualization: scatter plot

Pearson’s correlation (the “r” coefficient) is used to test for a relationship between two continuous variables. Both variables must be continuous data (e.g., you cannot use Likert data in a Pearson’s correlation). This test is used when you only have quasi-independent variables. If you wanted to know if blood pressure varied systematically with a person’s weight, you would use this test. Both blood pressure and weight are continuous values (you can weigh someone with arbitrary precision, down to milligrams if you’d like), and they are quasi-independent variables because they are factors that you as the experimenter do not directly manipulate (participants simply join your study with a given weight and blood pressure). If you have a real independent variable (such as you would in an experimental design), not only could you do a Pearson’s correlation, you would also likely pursue a linear regression analysis.

Calculating Pearson’s correlation leaves you with a coefficient r, which can vary between -1 and +1. The sign of the coefficient tells you the nature of the relationship. A positive value means that as one variable increases, so does the other. We might expect weight and blood pressure to be positively correlated. A negative value implies a negative relationship: as one variable increases, the other decreases. The magnitude of the value (closer to either -1 or +1) tells you the strength of the relationship. The closer the value gets to -1 or +1, the stronger the correlation between the two values. A coefficient \(\approx 0\) means that there is no relationship between the variables.

The example below should serve as practice. Try manually entering the data from this example into your statistical software, and then calculate the correlation. Your goal is to confirm that you can derive the same outcome before moving on to your own dataset.

3.4.2.1 Example #1

Imagine we measured the height and weight of N = 10 men and obtained the following values:

height	weight
64.6	182
68.3	187
68.8	192
69.8	196
69.8	201
70.3	204
70.7	210
70.8	212
72.3	215
74.0	220

A Pearson’s correlation coefficient analysis should return the following result:

There is indeed a significant correlation between men’s height and weight for the obtained sample, r = 0.931, p < .001.

For this result, we see a positive correlation coefficient. As one variable (height) increases, so does the other variable (weight). This makes sense because as people grow taller they accumulate more mass. This correlation is very close to 1. Values this close to -1 or +1 rarely occur in the social sciences. The actual p value for this result is .00009058, which is a very small number. I’ve chosen to simply write it as p < .001. Typically, you will want to list the actual p value. So if your p value was .0013, you would write p = .001. And remember, in hypothesis testing significance is typically defined as having a p value < .05. The smaller the better. Receiving a p value > .05 means your result is not statistically significant.

Tip

While it is important to include the actual p value, rounding is recommended for very long decimal values. For example, a p value of .001631839921 should be rounded to three decimals places, or .002

3.4.3 t-test

Data Type	Example(s)
ratio	height, weight, hours spent online each day

Typical visualization: bar chart (plotting the means of each condition)

Often in research we are interested in the average outcome of a group of individuals. For example, we might want to know how much time the average college student spends per day on their phone, how many alcoholic drinks are consumed each week on average, or how many followers the average user has on Instagram. This curiosity often takes us one step further, causing us to question whether differences exist between groups of individuals. Perhaps we want to know who drinks more per week: men or women? Or perhaps we want to assess whether seniors have less anxiety than first year students. The t-test provides a method for comparing the mean of two groups.

Here are some important considerations for t-tests:

• A single group test is known as a one sample t-test. This is used to test whether the mean of a single measured variable differs from a specified null hypothesis value (often zero). For example, it can answer the question, “does the average student have more than one social media account?”
• A two group, between subjects test is known as a two sample t-test. This test compares the means of two different groups of people, such as in the anxiety example above.
• A two group, within subjects test is known as a repeated-measures t-test (or paired samples or related samples t-test). In this design, there are two conditions, but the participants are the same in both conditions. Does the average person have more follows on Instagram than on Twitter? A repeated-measures t-test could be used to answer that question.
• You cannot perform a t-test on >2 groups. If you have 3+ conditions, you should consider an ANOVA (reach out to Dr. Van Horn if you find yourself in this situation).
• These tests should only be performed on ratio scale data. This means, most notably, that you cannot perform a t-test on Likert scale data!
• There are other assumptions, such as independence and normality (this primer cannot recreate a proper course in statistics–please spend a little time re-familiarizing yourself with the t-test if you plan to use it).

3.4.3.1 Example #1: one sample t-test

Let’s imagine that we wanted to know if the average college student consumed an alcoholic beverage each week. To assess this, we asked 20 students how many alcoholic drinks they consumed last week (never mind that this isn’t a very good design), and we obtained the following data:

Number of drinks
1
0
2
2
0
4
2
0
0
3
1
0
3
2
0
1
1
0
2
0
M = 1.2

Our null hypothesis would be that the mean number of drinks consumed in one week = 0. A one sample t-test reveals that the average number of drinks (M = 1.2) was actually greater than zero (t(19) = 4.3289, p = 0.0003617). If you need to use a one sample t-test yourself, please try to recreate this result before attempting it on your own dataset.

3.4.3.2 Example #2: two sample t-test

It seems that younger adults (age 18-30) are more likely to have a greater number of followers on Instagram than middle-aged adults (age 31-43), presumably because of their “digital native” status. A sample of users from Instagram were collected from each age group (N = 20 from each group, 40 in total), and their follower count was recorded. Imagine the results received were:

Young Adult	Middle-Aged
38	34
44	47
50	64
36	51
30	59
38	50
63	40
52	39
56	54
35	44
46	38
49	41
43	48
25	28
32	51
24	37
63	35
53	70
31	35
50	41
M = 42.9	M = 45.3

Our null hypothesis is that there is no difference in the means between these two groups (although we actually suspect there is). A two sample t-test reveals that there is, in fact, no difference between the mean number of followers for these two groups (t(38) = -0.67782, p = 0.502). If you need to use a two sample t-test yourself, please try to recreate this result before attempting it on your own dataset.

3.4.3.3 Example #3: repeated-measures t-test

We are curious whether the average anxiety of students increases during final exam week relative to the rest of the semester. For such a study, we devise a within subjects design in which we use Cohen’s perceived stress scale to measure N = 20 students’ anxiety levels once mid-semester, and then again during finals week. The results for the 20 students’ anxiety scores are included below (higher scores indicate higher anxiety levels):

Student	Non-finals anxiety	Finals anxiety
1	38	22
2	44	18
3	33	34
4	35	32
5	26	47
6	23	40
7	27	53
8	32	52
9	40	47
10	38	21
11	21	25
12	22	48
13	21	48
14	16	40
15	26	15
16	25	43
17	16	48
18	23	37
19	28	19
20	7	34
	M = 27.05	M = 36.15

Our null hypothesis is that there is no difference in the means between these two time periods (as usual, we actually suspect a difference). A repeated-measures t-test reveals that there is indeed an increase in anxiety during finals week (t(19) = -2.3091, p = 0.03234). If you need to use a repeated-measures t-test yourself, please try to recreate this result before attempting it on your own dataset.

3.5 Prerequisites

The following step only needs to be completed the first time you use R/Positron on a given computer. After installing Positron, run the following command in the “Console” tab in the lower-left window:

install.packages(c("tidyverse", "rmarkdown", "qualtRics", "HH"))

3.6 Quarto Document Setup

All analyses and related assignments will be completed using Quarto. Quarto allows users to mix text and analysis commands together into a single document, producing high-quality documents that contain simple and complex statistical analyses, the results of those analyses, figures/charts, and any free-form prose that you want to include. Detailed instructions and a walk-through of Quarto will be given in class.

For each assignment that uses Quarto (all statistical report assignments), you will need to prepare a Quarto document using Positron. Again, you will be shown how to do this in class for your first Quarto-based assignment. Nonetheless, here is the workflow for creating Quarto documents in this class:

1. Open Positron.
2. Choose the “New File” option under the button in the top left of the window that reads, “(+) New”. Select “Quarto Document” from the resulting drop down menu.
3. A new file tab called “Untitled-X” will open in Positron. The file will contain some minimal text such as:

---
title: "Untitled"
format: html
---

4. Delete the default text included in the newly opened tab, and replace it with the following using copy/paste (do not type the content manually):

---
title: "Stats Report"
author: "YOUR NAMES HERE"
date: last-modified
date-format: D MMMM, YYYY
format:
  html:
    self-contained: true
    theme: cosmo
    toc: true
---

6. Add the following “code chunks” below to your newly opened file tab, then save the file to your class project folder.

3.6.1 Load dataset

First, we load a few libraries, as well as the survey responses into two variables—one for your numerical data, and another for your text data.

library(HH)
library(qualtRics)
library(tidyverse)

dn = read_survey("results_numeric.csv")
d = read_survey("results_text.csv")

Important note: the code above assumes that you’ve first downloaded TWO copies of your data from Qualtrics. Under the “Data & Analysis” tab in your Qualtrics survey, you can click the “Export & Import” button to download a copy of your data. Do this twice, once selecting “export values” and once selecting “export labels”. Put both files in your analysis folder on your computer, renaming them “results_numeric.csv” and “results_text.csv” respectively.

This will load TWO copies of your data into R, one in the variable d (the choice text data) and one in the variable dn (which contains numeric responses). Both copies, if downloaded at the same time, contain the same participants, in the same order. So the participant in the 5th row of the d data frame is the same person as the 5th row in the dn data frame.

You can use these two copies of your dataset interchangeably, using whichever version is appropriate for your analysis.

3.6.2 Remove outliers

This step isn’t required. However, it is often necessary to remove “bad” participant data for one reason or another. If you do remove 1+ participants, please provide a written record for why you removed each one. If you’re curious about whether you should remove a participant, please see the removing outliers section of the project manual for advice.

3.6.2.1 Removing individuals manually

Important

Only include the following outlier removal code chunk if you have identified a need to do so. Typically, you will NOT need to do this.

If you’ve identified a list of outliers (i.e., respondents) that you’ve manually identified (say, by finding them in Excel when looking over your data, etc.), you can remove them in the following way. Note that you must identify the offending respondent’s row number in your dataset. If you have a reason to manually remove specific participants from your dataset, use the following code block. Note that you’re using R to exclude them from any analyses, rather than deleting them from your data spreadsheet. Don’t delete participants from your spreadsheet!

Here’s how to remove specific participants from your dataset:

1. Open your data in Excel
2. Find the row that corresponds to the participant that needs removed.
3. Subtract 3 from that number (the first three rows in your data are meta data, so what is row 4 in Excel will be participant #1 in R).
4. Edit the code block below by changing c(1, 2, 3, 4, 5, 6, 7) to contain the participant numbers you want removed.

# Identify participants by row number, separated by commas
bad_sbjs = c(1, 2, 3, 4, 5, 6, 7)

# Remove identified rows
d = d[-bad_sbjs, ]
dn = dn[-bad_sbjs, ]

3.6.3 Remove incomplete participants

Some participants never finish the study. These are marked by the field “Finished” in the exporting Qualtrics survey response data. In R, the uncompleted respondents are marked as FALSE for this field. It is safe to remove these participants. Note that this is different from participants that chose not to answer one or more questions—these participants still show up as TRUE in the “Finished” data field.

d = d[d$Finished == TRUE, ]
dn = dn[dn$Finished == TRUE, ]
d = d |> filter(Status != "Survey Preview")
dn = dn |> filter(Status == 0)

3.6.4 Sample Information

After removing any outliers and unfinished respondents, determine your sample size with the following:

N = nrow(d)
N