Test-retest and split-half reliability: ICC, Spearman-Brown and when to use each

Most theses report reliability with Cronbach’s alpha, which measures internal consistency: whether the items of a scale point to the same thing. But there are two other questions an advisor or reviewer may ask. Does the instrument give similar results when applied twice to the same people? That is test-retest reliability. Do the two halves of the scale agree? That is split-half reliability.

Test-retest reliability

You give the same questionnaire to the same people twice, 2 to 4 weeks apart, and compare the scores. If the trait is stable (for example, a personality trait or an attitude), the scores should be similar. Too short an interval and people remember their answers; too long and the trait may truly have changed.

Why the intraclass correlation and not Pearson

Many older studies use Pearson’s correlation between test and retest, but it has a flaw: it only checks whether people keep their order. If everyone scores 5 points higher on the retest, Pearson gives 1 even though no score matches. The absolute agreement intraclass correlation coefficient (ICC) counts those differences as error, which is why it is recommended today (Koo & Li, 2016).

How to interpret it

• Below 0.50: poor.
• 0.50 to 0.75: moderate.
• 0.75 to 0.90: good.
• Above 0.90: excellent.

Report the 95% confidence interval too. With 15 or 20 people the interval is wide, and the prudent move is to interpret its lower bound. Test-retest studies usually recommend 30 people or more.

How to report it

Test-retest reliability was assessed with the intraclass correlation coefficient (two-way model, absolute agreement, single measures), with 15 participants and a three-week interval. The result was ICC = .91, 95% CI [.74, .97], F(14, 14) = 19.06, p < .001, indicating excellent reliability.

The same coefficient works for inter-rater agreement: for example, when two teachers grade the same work with a rubric. Each rater goes in one column.

Split-half reliability

Here you apply the instrument once, split the items into two halves, add up each half and correlate them. Since each half has half the items, and a shorter scale is always less reliable, the correlation is corrected with the Spearman-Brown formula: 2r / (1 + r).

Example: if the correlation between halves is 0.70, Spearman-Brown gives 2 × 0.70 / 1.70 = 0.82. It is read like Cronbach’s alpha: 0.70 or more is acceptable and 0.80 or more is good.

How to split the items

• Odd versus even: the most recommended, because it spreads easy and hard items, and end-of-questionnaire fatigue, more evenly.
• First half versus second half: SPSS’s default. Use it if you need to match an SPSS result.
• With an odd number of items the halves are unequal, and the unequal-length Spearman-Brown is used.
• Guttman’s coefficient is an alternative that doesn’t assume both halves have the same variance. If it differs a lot from Spearman-Brown, report Guttman.

Which one should I use?

1. Internal consistency (Cronbach’s alpha or McDonald’s omega): almost always. It is expected by default.
2. Split-half: if your university or advisor asks for it, or as extra information alongside alpha. In practice, alpha is the average of all possible split halves.
3. Test-retest: if your variable should be stable over time and you can apply the instrument twice to the same people. It adds a lot in instrument validation studies.

In SurvIQ’s Thesis Mode, pilot test responses are ready for alpha, and you can download them to use in these calculators. SurvIQ doesn’t write the thesis: it calculates, lays out the tables and shows what to review.