Which statistical test to use in your thesis: a decision-tree guide
One of the most common comments at a thesis defense is “that test doesn’t fit”. It happens when Pearson is used with non-normal data, a t-test compares three groups, or a chi-square is run on the same people measured twice. The good news is that the choice follows clear rules.
To choose, answer four questions in this order. Get them right and the test picks itself.
1. What do you want to do?
- Compare: check whether two or more groups (or measurements) differ. For example, stress among public and private school teachers.
- Relate: check whether two variables move together. For example, more working hours, more stress.
- Predict: estimate how much several variables explain another. For example, which factors explain satisfaction.
- Compare with what’s expected: check whether your sample resembles the population, for example in its share by region.
Look at your hypothesis. If it says “there is a relationship between…”, you relate. If it says “there are differences between…”, you compare.
2. What type of variable do you have?
- Numeric: a scale’s total score, age, grades. You can average it.
- Ordinal: a single Likert item, low/medium/high levels, a ranking. There’s an order, but the distance between values isn’t fixed.
- Categorical: yes or no, gender, major, passed or not. Only groups.
The total score of a multi-item scale is treated as numeric. A single item is treated as ordinal. If you turn scores into levels with cutoffs, those levels are ordinal.
3. Different people or the same people?
If each person is in only one group (men and women, three sites), the groups are independent. If the same people are measured several times (before and after a program), the samples are related. This question separates, for example, the independent-samples t-test from the paired t-test.
4. Are your data normal?
With numeric variables, run a normality test: Shapiro-Wilk with fewer than 50 cases and Kolmogorov-Smirnov with the Lilliefors correction with more. If you compare groups, check normality within each group.
- p ≥ .05: the data are normal. Use a parametric test, which compares means.
- p < .05: they aren’t normal. Use a non-parametric test, which compares ranks.
If you haven’t run it yet:
Shapiro-Wilk normality test (free)The decision table
- Two independent groups: Student’s t-test (normal) or Mann-Whitney U (not normal).
- Two measurements of the same people: paired t-test or Wilcoxon. If the answer is yes or no: McNemar.
- Three or more independent groups: one-way ANOVA or Kruskal-Wallis.
- Three or more measurements of the same people: repeated-measures ANOVA or Friedman.
- Relationship between two numeric or ordinal variables: Pearson (normal) or Spearman (not normal or ordinal).
- Relationship removing the effect of a third variable: partial correlation.
- Relationship between two categorical variables: chi-square test of independence.
- One categorical variable compared with what’s expected: chi-square goodness of fit.
- Predicting one variable from several: multiple linear regression.
If you’d rather answer questions than read the table:
Which statistical test should I use? (free selector)The mistakes committees flag most
Using Pearson with non-normal data
It’s the most frequent mistake in correlational theses. If the normality test is significant for either variable, use Spearman. Using Spearman with normal data isn’t a serious mistake; it just loses some power.
Comparing three groups with several t-tests
Each test carries a 5% risk of a false positive. With three comparisons the overall risk rises to almost 15%. That’s what ANOVA and Kruskal-Wallis are for, with corrected pairwise comparisons.
Forgetting they’re the same people
A pretest and posttest of the same group are related samples. Analyzing them as independent groups ignores that each person is compared with themselves and loses precision.
Reporting only the p-value
APA style asks for the effect size next to every test. The p-value says whether there’s an effect; the effect size says whether it matters.
How to report it:
Effect size: what it is and how to report it in APA 7Before your defense
Check that each hypothesis’s test matches what the hypothesis says, the variable type and your normality result, and that you explain it in the methods. If a committee member asks “why did you use that test?”, the answer should be one sentence: “because it compares two independent groups and the data weren’t normal according to Shapiro-Wilk”.
To review the whole methodology, not just the test:
Thesis methodology checklist