Survey and statistics glossary

Short definitions of the terms you run into when you build a survey or a thesis: sampling, validity, reliability, statistical tests and experience metrics. Each term links to the free tool that computes it.

Design and sampling

Population
The whole group you want to draw conclusions about: every customer of a store, every teacher in a school network. If you survey all of them it is a census and you need no sample.
Sample
The part of the population that answers the survey. It represents the population well when it is chosen at random and is large enough for the margin of error you accept.
Sample size calculator
Sample size
How many responses you need. It depends on population size, confidence level and margin of error. For a large population at 95% confidence and a 5% margin, it is about 385 responses.
Sample size calculator
Margin of error
How far a percentage from your survey may be from the true population value. If 60% are satisfied with a ±5% margin, the true value is very likely between 55% and 65%.
Margin of error calculator
Confidence level
How sure you want to be that the true value falls within your margin of error. The usual is 95%: if you repeated the survey 100 times, about 95 intervals would contain the true value.
Sample size calculator
Probability sampling
Every person in the population has a known chance of being chosen, by random draw. It is what lets you use the sample size formula and generalize the results.
Non-probability sampling
You survey whoever is easy to reach (convenience) or whoever the researcher picks (purposive). It is the most common in theses, but results do not generalize to the whole population; size is better justified with a power analysis.
Sample size for correlation and regression
Power analysis
Works out how many people you need to detect an effect of a given size with a statistical test, if that effect exists. The usual is 80% power and α = .05.
Sample size for correlation and regression
Response rate
Completed responses divided by people invited. A low rate does not invalidate the survey, but it raises the risk that only the happiest or the angriest answered.
Response rate calculator
Likert scale
A set of statements answered with levels of agreement, for example from 1 (strongly disagree) to 5 (strongly agree). The scale score is the sum or average of the items.
Likert scale generator
Response bias
When the way a question is asked pushes toward an answer: leading questions, two questions in one, or the urge to give the socially acceptable answer. Neutral, simple questions reduce it.
Survey question checker
Operationalization
Turning an abstract variable (for example, workplace climate) into something measurable: dimensions, indicators and questionnaire items, with their response scale.
Operationalization of variables generator
Pilot test
Running the questionnaire with a small group (20 to 30 people like your sample) before the study. It finds confusing questions and gives a first reliability estimate.
Cronbach’s alpha calculator

Validity

Validity
How well an instrument measures what it claims to measure. It is checked in parts: content validity (with judges), construct validity (with factor analysis) and criterion validity (against another measure).
Thesis methodology checklist
Content validity
Whether the items cover what you want to measure and leave out what does not belong. It is assessed with expert judges and summarized in a coefficient per item.
Aiken’s V calculator
Expert judgment
Between 3 and 10 specialists rate each item for clarity, pertinence and relevance. Their ratings are summarized with Aiken’s V, Lawshe’s ratio or the CVC; signed forms alone do not show how much they agreed.
Aiken’s V calculator
Aiken’s V
A 0 to 1 coefficient summarizing how much judges agree an item is adequate. Items are usually kept with V ≥ .70 or, more strictly, when the lower bound of its interval exceeds .70.
Aiken’s V calculator
Content validity ratio (Lawshe)
Each judge says whether an item is essential. CVR = (essential − N/2) / (N/2). The minimum value to keep the item depends on the number of judges.
Lawshe's content validity ratio (CVR) calculator
Construct validity
Whether items group into the dimensions the theory proposes. It is tested with exploratory or confirmatory factor analysis of your own data.
Exploratory factor analysis calculator
Exploratory factor analysis
A technique that finds which items move together and groups them into factors. You report explained variance and each item’s loading on its factor (loadings of .40 or more are usually accepted).
Exploratory factor analysis calculator
KMO and Bartlett’s test
They tell you whether factor analysis makes sense with your data. KMO ranges from 0 to 1 and you want .70 or more; Bartlett should be significant (p < .05).
KMO and Bartlett’s test calculator

Reliability

Reliability
How consistent a measurement is: whether a scale’s items point the same way and whether the score would repeat if measured again. It is computed with your own sample or pilot data.
Cronbach’s alpha calculator
Cronbach’s alpha
The most used internal consistency coefficient for Likert scales. It ranges from 0 to 1: .70 or more is acceptable and .80 or more good. Very close to 1 may mean redundant items.
Cronbach’s alpha calculator
McDonald’s omega
A reliability coefficient based on factor loadings, so it does not assume all items weigh the same. It reads like alpha and more and more journals prefer it.
McDonald’s omega calculator
KR-20 (Kuder-Richardson)
Reliability for two-option items: right or wrong, yes or no. It is the dichotomous case of Cronbach’s alpha and is common in knowledge tests.
KR-20 calculator (Kuder-Richardson)
Test-retest and intraclass correlation
Giving the same instrument twice to the same people and checking whether scores match. It is summarized with the intraclass correlation coefficient (ICC): .75 or more is good.
Intraclass correlation coefficient (ICC) calculator
Cohen’s kappa
Agreement between two raters sorting the same things into categories, discounting the agreement expected by chance. Above .60 is considered substantial.
Cohen’s and Fleiss’ kappa calculator

Analysis and tests

Null hypothesis
The statement that there is no difference or relationship. A statistical test measures how unusual your data would be if the null hypothesis were true.
Compare two percentages
p-value
The probability of getting a result like yours, or more extreme, if there were really no difference. With p < .05 the null hypothesis is rejected. It does not say how large the effect is.
Compare two percentages
Statistical significance
A result is significant when its p-value is below the chosen level (almost always .05). It means it is unlikely to be chance, not that it matters in practice.
Compare two percentages
Normality test
Checks whether data follow a normal distribution, which decides between a parametric and a non-parametric test. Shapiro-Wilk works better with small samples; Kolmogorov-Smirnov (Lilliefors) is used with large ones.
Normality test (Shapiro-Wilk and Kolmogorov-Smirnov)
Parametric and non-parametric tests
Parametric tests (Student’s t, ANOVA, Pearson) assume normal data and compare means. Non-parametric tests (Mann-Whitney, Kruskal-Wallis, Spearman) compare ranks and work with non-normal or ordinal data.
Thesis methodology checklist
Pearson correlation
Measures the linear relationship between two numeric variables, from −1 to 1. It assumes normal data. An r of .50 is a strong relationship in social science.
Correlation calculator (Pearson and Spearman)
Spearman’s rho
A correlation computed on ranks instead of values. It is the alternative to Pearson when data are not normal or are ordinal, like Likert scores.
Correlation calculator (Pearson and Spearman)
Student’s t-test
Compares the means of two independent groups (for example, men and women) or two measurements of the same people (before and after). It assumes normal data.
Student’s t-test calculator
Mann-Whitney U
Compares two independent groups using ranks. It is the non-parametric alternative to Student’s t when data are not normal.
Mann-Whitney U test calculator
Wilcoxon signed-rank test
Compares two measurements of the same people (before and after) using the ranks of the differences. It is the alternative to the paired t-test.
Wilcoxon signed-rank test calculator
ANOVA
Compares the means of three or more groups at once. If it is significant, pairwise comparisons tell you which groups differ.
One-way ANOVA calculator
Kruskal-Wallis
Compares three or more independent groups using ranks. It is the non-parametric alternative to one-way ANOVA.
Kruskal-Wallis test calculator
Friedman test
Compares three or more measurements of the same people using ranks. It is the non-parametric alternative to repeated-measures ANOVA.
Friedman test calculator
Chi-square test
Tests whether two categorical variables are associated by comparing what a cross table shows with what independence would predict.
Chi-square calculator
Linear regression
Estimates how much one variable changes when one or more others change. R² tells you how much variation the model explains, and each coefficient the weight of each variable.
Simple and multiple linear regression calculator
Effect size
How large a difference or relationship is, regardless of sample size. It complements the p-value and APA style asks you to report it.
Effect size calculator
Cohen’s d
Effect size for the difference between two means, in standard deviations. 0.2 is small, 0.5 medium and 0.8 large.
Effect size calculator
Score levels (norms)
The cutoffs that turn a score into levels (low, medium, high). They come from equal ranges, percentiles or the instrument author’s cutoffs, and you need to say which one you used.
Likert scale score ranges

Experience metrics

NPS (Net Promoter Score)
Asks from 0 to 10 how likely people are to recommend the company. NPS = % promoters (9 and 10) − % detractors (0 to 6). It ranges from −100 to 100.
NPS calculator
eNPS
NPS applied to employees: how likely they are to recommend the company as a place to work. It is computed like NPS.
eNPS calculator
CSAT
Customer satisfaction with a purchase or interaction, on a 1 to 5 scale. It is reported as the share who answered 4 or 5.
CSAT calculator
CES (Customer Effort Score)
Measures how much effort it took the customer to get something done, for example on a 1 to 7 scale. Less effort tends to predict more loyalty.
CES calculator

Not sure which test fits?

The thesis checklist tells you which test to use for your goal and normality result, and what your methodology is missing.

Open the checklist