Survey and statistics glossary
Short definitions of the terms you run into when you build a survey or a thesis: sampling, validity, reliability, statistical tests and experience metrics. Each term links to the free tool that computes it.
Design and sampling
- Population
- The whole group you want to draw conclusions about: every customer of a store, every teacher in a school network. If you survey all of them it is a census and you need no sample.
- Sample
- The part of the population that answers the survey. It represents the population well when it is chosen at random and is large enough for the margin of error you accept.
- Sample size calculator
- Sample size
- How many responses you need. It depends on population size, confidence level and margin of error. For a large population at 95% confidence and a 5% margin, it is about 385 responses.
- Sample size calculator
- Margin of error
- How far a percentage from your survey may be from the true population value. If 60% are satisfied with a ±5% margin, the true value is very likely between 55% and 65%.
- Margin of error calculator
- Confidence level
- How sure you want to be that the true value falls within your margin of error. The usual is 95%: if you repeated the survey 100 times, about 95 intervals would contain the true value.
- Sample size calculator
- Probability sampling
- Every person in the population has a known chance of being chosen, by random draw. It is what lets you use the sample size formula and generalize the results.
- Non-probability sampling
- You survey whoever is easy to reach (convenience) or whoever the researcher picks (purposive). It is the most common in theses, but results do not generalize to the whole population; size is better justified with a power analysis.
- Sample size for correlation and regression
- Power analysis
- Works out how many people you need to detect an effect of a given size with a statistical test, if that effect exists. The usual is 80% power and α = .05.
- Sample size for correlation and regression
- Response rate
- Completed responses divided by people invited. A low rate does not invalidate the survey, but it raises the risk that only the happiest or the angriest answered.
- Response rate calculator
- Likert scale
- A set of statements answered with levels of agreement, for example from 1 (strongly disagree) to 5 (strongly agree). The scale score is the sum or average of the items.
- Likert scale generator
- Response bias
- When the way a question is asked pushes toward an answer: leading questions, two questions in one, or the urge to give the socially acceptable answer. Neutral, simple questions reduce it.
- Survey question checker
- Operationalization
- Turning an abstract variable (for example, workplace climate) into something measurable: dimensions, indicators and questionnaire items, with their response scale.
- Operationalization of variables generator
- Pilot test
- Running the questionnaire with a small group (20 to 30 people like your sample) before the study. It finds confusing questions and gives a first reliability estimate.
- Cronbach’s alpha calculator
Validity
- Validity
- How well an instrument measures what it claims to measure. It is checked in parts: content validity (with judges), construct validity (with factor analysis) and criterion validity (against another measure).
- Thesis methodology checklist
- Content validity
- Whether the items cover what you want to measure and leave out what does not belong. It is assessed with expert judges and summarized in a coefficient per item.
- Aiken’s V calculator
- Expert judgment
- Between 3 and 10 specialists rate each item for clarity, pertinence and relevance. Their ratings are summarized with Aiken’s V, Lawshe’s ratio or the CVC; signed forms alone do not show how much they agreed.
- Aiken’s V calculator
- Aiken’s V
- A 0 to 1 coefficient summarizing how much judges agree an item is adequate. Items are usually kept with V ≥ .70 or, more strictly, when the lower bound of its interval exceeds .70.
- Aiken’s V calculator
- Content validity ratio (Lawshe)
- Each judge says whether an item is essential. CVR = (essential − N/2) / (N/2). The minimum value to keep the item depends on the number of judges.
- Lawshe's content validity ratio (CVR) calculator
- Construct validity
- Whether items group into the dimensions the theory proposes. It is tested with exploratory or confirmatory factor analysis of your own data.
- Exploratory factor analysis calculator
- Exploratory factor analysis
- A technique that finds which items move together and groups them into factors. You report explained variance and each item’s loading on its factor (loadings of .40 or more are usually accepted).
- Exploratory factor analysis calculator
- KMO and Bartlett’s test
- They tell you whether factor analysis makes sense with your data. KMO ranges from 0 to 1 and you want .70 or more; Bartlett should be significant (p < .05).
- KMO and Bartlett’s test calculator
Reliability
- Reliability
- How consistent a measurement is: whether a scale’s items point the same way and whether the score would repeat if measured again. It is computed with your own sample or pilot data.
- Cronbach’s alpha calculator
- Cronbach’s alpha
- The most used internal consistency coefficient for Likert scales. It ranges from 0 to 1: .70 or more is acceptable and .80 or more good. Very close to 1 may mean redundant items.
- Cronbach’s alpha calculator
- McDonald’s omega
- A reliability coefficient based on factor loadings, so it does not assume all items weigh the same. It reads like alpha and more and more journals prefer it.
- McDonald’s omega calculator
- KR-20 (Kuder-Richardson)
- Reliability for two-option items: right or wrong, yes or no. It is the dichotomous case of Cronbach’s alpha and is common in knowledge tests.
- KR-20 calculator (Kuder-Richardson)
- Test-retest and intraclass correlation
- Giving the same instrument twice to the same people and checking whether scores match. It is summarized with the intraclass correlation coefficient (ICC): .75 or more is good.
- Intraclass correlation coefficient (ICC) calculator
- Cohen’s kappa
- Agreement between two raters sorting the same things into categories, discounting the agreement expected by chance. Above .60 is considered substantial.
- Cohen’s and Fleiss’ kappa calculator
Analysis and tests
- Null hypothesis
- The statement that there is no difference or relationship. A statistical test measures how unusual your data would be if the null hypothesis were true.
- Compare two percentages
- p-value
- The probability of getting a result like yours, or more extreme, if there were really no difference. With p < .05 the null hypothesis is rejected. It does not say how large the effect is.
- Compare two percentages
- Statistical significance
- A result is significant when its p-value is below the chosen level (almost always .05). It means it is unlikely to be chance, not that it matters in practice.
- Compare two percentages
- Normality test
- Checks whether data follow a normal distribution, which decides between a parametric and a non-parametric test. Shapiro-Wilk works better with small samples; Kolmogorov-Smirnov (Lilliefors) is used with large ones.
- Normality test (Shapiro-Wilk and Kolmogorov-Smirnov)
- Parametric and non-parametric tests
- Parametric tests (Student’s t, ANOVA, Pearson) assume normal data and compare means. Non-parametric tests (Mann-Whitney, Kruskal-Wallis, Spearman) compare ranks and work with non-normal or ordinal data.
- Thesis methodology checklist
- Pearson correlation
- Measures the linear relationship between two numeric variables, from −1 to 1. It assumes normal data. An r of .50 is a strong relationship in social science.
- Correlation calculator (Pearson and Spearman)
- Spearman’s rho
- A correlation computed on ranks instead of values. It is the alternative to Pearson when data are not normal or are ordinal, like Likert scores.
- Correlation calculator (Pearson and Spearman)
- Student’s t-test
- Compares the means of two independent groups (for example, men and women) or two measurements of the same people (before and after). It assumes normal data.
- Student’s t-test calculator
- Mann-Whitney U
- Compares two independent groups using ranks. It is the non-parametric alternative to Student’s t when data are not normal.
- Mann-Whitney U test calculator
- Wilcoxon signed-rank test
- Compares two measurements of the same people (before and after) using the ranks of the differences. It is the alternative to the paired t-test.
- Wilcoxon signed-rank test calculator
- ANOVA
- Compares the means of three or more groups at once. If it is significant, pairwise comparisons tell you which groups differ.
- One-way ANOVA calculator
- Kruskal-Wallis
- Compares three or more independent groups using ranks. It is the non-parametric alternative to one-way ANOVA.
- Kruskal-Wallis test calculator
- Friedman test
- Compares three or more measurements of the same people using ranks. It is the non-parametric alternative to repeated-measures ANOVA.
- Friedman test calculator
- Chi-square test
- Tests whether two categorical variables are associated by comparing what a cross table shows with what independence would predict.
- Chi-square calculator
- Linear regression
- Estimates how much one variable changes when one or more others change. R² tells you how much variation the model explains, and each coefficient the weight of each variable.
- Simple and multiple linear regression calculator
- Effect size
- How large a difference or relationship is, regardless of sample size. It complements the p-value and APA style asks you to report it.
- Effect size calculator
- Cohen’s d
- Effect size for the difference between two means, in standard deviations. 0.2 is small, 0.5 medium and 0.8 large.
- Effect size calculator
- Score levels (norms)
- The cutoffs that turn a score into levels (low, medium, high). They come from equal ranges, percentiles or the instrument author’s cutoffs, and you need to say which one you used.
- Likert scale score ranges
Experience metrics
- NPS (Net Promoter Score)
- Asks from 0 to 10 how likely people are to recommend the company. NPS = % promoters (9 and 10) − % detractors (0 to 6). It ranges from −100 to 100.
- NPS calculator
- eNPS
- NPS applied to employees: how likely they are to recommend the company as a place to work. It is computed like NPS.
- eNPS calculator
- CSAT
- Customer satisfaction with a purchase or interaction, on a 1 to 5 scale. It is reported as the share who answered 4 or 5.
- CSAT calculator
- CES (Customer Effort Score)
- Measures how much effort it took the customer to get something done, for example on a 1 to 7 scale. Less effort tends to predict more loyalty.
- CES calculator
Not sure which test fits?
The thesis checklist tells you which test to use for your goal and normality result, and what your methodology is missing.
Open the checklist