Survey research for your thesis: the complete guide, from questionnaire to results
A thesis based on a survey has an order. If you skip a step, the committee notices: a questionnaire without content validity, a sample size with no justification, Pearson on data that is not normal, a hypothesis that does not answer the objective. The good news is that each step has a clear way to be done and reported.
This guide walks through that order from start to finish, for quantitative undergraduate and master’s theses in psychology, education, business, nursing, engineering and related fields. For each step I explain what to do, which values are accepted and which free calculator handles it. They all run in the browser with nothing to install, and most build the APA table for you to paste into Word.
Your university and your advisor have the final word. The criteria in this guide are the most common, but some programs ask for different cut-offs or tests. Always check your department’s thesis guidelines.
Step 1: the problem, the variables and the research matrix
Everything starts with one sentence: the research question. "What is the relationship between workplace climate and teacher performance in the schools of…?" From it come the general objective (determine that relationship) and the general hypothesis (there is a significant relationship between…). Then the specific questions, objectives and hypotheses, one for each dimension you will analyze.
The research consistency matrix puts all of that in one table, and it is the first thing a committee checks: whether each question has its objective, each objective its hypothesis, and each hypothesis can be tested with the data you will collect. If something does not fit here, it will not fit in the results either.
- The verb in the objective decides the test: "determine the relationship" calls for a correlation; "compare" calls for a test of differences; "determine the influence" or "the effect" in a non-experimental design usually calls for regression, and is worth wording carefully.
- Each specific hypothesis must be testable with a specific statistical test.
- The design (non-experimental, cross-sectional, correlational, descriptive, explanatory) must match what you will actually do.
Build the table and download it as Word:
Research consistency matrix generatorStep 2: operationalize the variables
Operationalizing means turning each variable into something measurable: variable, conceptual definition (what it is according to an author), operational definition (how you will measure it), dimensions, indicators, items and measurement scale.
- Each dimension needs at least 3 items, ideally 4 to 8. With fewer, the dimension’s reliability comes out low.
- Indicators are the observable part of each dimension; items are the questions that measure those indicators.
- The measurement level is stated here: ordinal for Likert, nominal for categories, ratio for numbers.
Build the table with dimensions, indicators and items:
Operationalization of variables table generatorStep 3: use an existing instrument or build your own?
If a validated questionnaire measures your variable, use it: it already has evidence of validity and reliability, and your results can be compared with other studies. Three things to watch:
- Cite the original author and the adaptation you use, if any.
- Check whether the instrument is free to use or needs permission.
- If you adapt it to your context (change wording, drop items, translate it), run expert judgment and a pilot test again. An instrument validated in another country or population is not automatically valid for yours.
If none fits, build your own from the operationalization. It takes more work, but it is perfectly valid if you follow steps 4 to 7.
Step 4: write the items
Most survey-based theses use a Likert scale: statements with 5 response options (Strongly disagree to Strongly agree, or Never to Always). Rules for every item:
- One idea per item. "The teacher explains clearly and uses examples" is two items.
- Language your population understands. A questionnaire for teachers is not the same as one for high school students.
- No double negatives or words that lead the answer.
- The same scale on every item of a variable, so they can be added up.
- If you include reverse-worded items, remember to reverse their scores before adding. Many low Cronbach’s alphas come from forgetting this.
Response options for each type of scale, ready to copy:
Likert scale generatorPaste your items and check the most common wording mistakes:
Survey question checkerThe scale in detail:
What a Likert scale is and how to use itStep 5: content validity, expert judgment
Content validity answers: do the items measure what they claim to measure? It is shown with expert judges who rate each item on criteria such as relevance, pertinence and clarity. The usual number is 3 to 10 experts; 5 to 7 is a good number. Choose people with academic degrees or experience in the topic and, if possible, at least one in methodology.
From the judges’ ratings you calculate a coefficient. Which one depends on how they rated and on what your university asks for:
- Aiken’s V: the most used in Latin America. Judges rate on a scale (for example, 1 to 4). Items with V of 0.70 or more are accepted; with a strict criterion, 0.80, ideally with the lower bound of the confidence interval above the cut-off too.
- Lawshe’s content validity ratio (CVR): judges say whether the item is essential or not. The minimum value depends on the number of judges.
- Hernández-Nieto’s content validity coefficient (CVC): corrects for possible judge bias. Accepted from 0.80.
- Content validity index (CVI): proportion of judges who rate the item as relevant. From 0.78 per item.
- Binomial test: when judges answer yes or no, to see whether agreement is above chance.
- Kendall’s W: agreement among judges when they rank or score items.
Items that do not pass are revised or removed, and that is reported. Keep the judges’ signed forms: they go in the appendices.
Content validity calculators:
Aiken’s V calculatorWhich to choose and how to report it:
Content validity: Aiken, Lawshe, CVC or binomialStep 6: pilot test and reliability
Reliability answers: does the instrument measure consistently? It is calculated with a pilot test: you give the revised questionnaire to a group similar to your sample, but not part of it. Usually 20 to 30 people, or about 10% of the sample.
- Cronbach’s alpha: the most used coefficient for Likert scales. 0.70 is acceptable and 0.80 is good. Above 0.95 may signal redundant items. Report it for the full instrument and for each dimension.
- McDonald’s omega: increasingly requested, because it does not assume all items weigh the same. Read with the same cut-offs.
- KR-20: the alpha equivalent for two-option items (right or wrong, yes or no), for example in knowledge tests.
- Split-half reliability (Spearman-Brown): splits the items in two halves and corrects the correlation between them.
- Test-retest with the intraclass correlation coefficient (ICC): you give the same questionnaire twice to the same people a few weeks apart.
- Cohen’s or Fleiss’ kappa: agreement between raters when two or more people score the same thing (for example, a checklist).
If alpha comes out low, first check for reverse-worded items that were not reversed, then look at the "alpha if item deleted" column: an item whose removal raises alpha a lot is usually poorly worded or measures something else.
Reliability calculators:
Cronbach’s alpha calculatorHow to run the pilot test:
Pilot test and Cronbach’s alpha, step by stepFor the other methods:
Test-retest and split-half reliabilityStep 7: construct validity (if your thesis needs it)
Construct validity checks whether the items group into the dimensions you proposed. It is done with factor analysis and is not always required at undergraduate level, but it adds a lot when you build your own instrument. It needs plenty of responses: at least 5 per item, ideally 10, or more than 200.
- First, KMO and Bartlett’s test: they tell you whether your data is suitable for factor analysis. KMO of 0.60 or more (0.70 is better) and Bartlett with p below 0.05.
- Then exploratory factor analysis: how many factors come out and which items load on each. A loading of 0.40 or more is considered adequate.
- If items do not group as expected, discuss it: two dimensions may measure the same thing, or an item may measure something else.
Both calculators:
KMO and Bartlett’s test calculatorThe full explanation:
KMO and Bartlett’s test: how to read themStep 8: population, sample and sampling
The population is everyone who meets your criteria (the 320 teachers in the school network, the 1,200 active customers). The sample is the part you survey. If the population is small (under 100 or 150), the strongest option is a census: survey everyone.
Sample size
To estimate percentages in a known population, use the finite population formula: n = N × Z² × p × q / (e² × (N − 1) + Z² × p × q), with 95% confidence (Z = 1.96), a 5% margin of error and p = q = 0.5 if you do not know the proportion. For 320 teachers, that gives 175.
For correlational or regression theses, the strongest criterion is a power analysis: how many people you need to detect a correlation of a given size. For a moderate correlation (r = 0.30) with 80% power and 5% significance, you need about 84 people. If the formula gives you fewer than that, use the larger number.
Both calculators:
Sample size calculatorSampling
- Probability sampling (simple random, stratified, cluster): lets you generalize to the population. Ideal, but you need the full list.
- Non-probability sampling (convenience, purposive, snowball): easier, but does not allow generalizing. Valid if you acknowledge it as a limitation.
- Stratified: if your population has groups of different sizes (sites, shifts, departments), split the sample in proportion to each group.
Calculate the sample in valid responses, not in questionnaires sent. If you expect half to answer, invite twice as many.
The formula explained:
Survey sample size, with examplesStep 9: run the survey
- Informed consent at the start: who you are, what the study is for, that it is voluntary, anonymous or confidential, and that they can withdraw. With minors, parental permission too.
- Written permission from the institution where you collect data. It goes in the appendices.
- If online, prevent duplicate responses and make required questions required, so you do not lose cases to missing data.
- If on paper, number the questionnaires and enter them with double checking.
- Note start and end dates, and how many answered out of how many were invited.
The text that goes at the start of the questionnaire:
Informed consent in surveys, with an exampleAnd the ethical principles an ethics board looks for:
Survey ethics: 8 rulesTo run the questionnaire online and export data ready to analyze:
Thesis surveys in SurvIQStep 10: descriptive analysis and levels
Before testing hypotheses, describe your data: the sample profile (age, sex, grade, department) as frequencies and percentages, and each variable and dimension in levels.
Levels (low, medium, high) are built from score ranges, often called norms. For example, if a dimension has 6 items scored 1 to 5, the total goes from 6 to 30, and you split it into three equal ranges or by percentiles. Then you can say "45% of teachers perceive a medium-level climate", which is how most theses present it.
Calculate the range for each level:
Likert scale score rangesThe results chapter, step by step:
Thesis results: norms, normality and correlationStep 11: normality test
Before choosing the hypothesis test, check whether your scores follow a normal distribution. That decides between a parametric test (which assumes it) and a non-parametric one (which does not).
- Shapiro-Wilk: recommended for samples under 50, and increasingly used for larger ones too.
- Kolmogorov-Smirnov with the Lilliefors correction: what many guides ask for with 50 or more.
- If p is above 0.05, normality is not rejected: you can use parametric tests. If p is below 0.05, the data is not normal: use non-parametric tests.
- Test each variable or dimension you will relate or compare, and within each group if you compare groups.
With summed Likert scales, normality very often fails. That is not a problem: you use the non-parametric test and report it.
Paste your scores and get both tests:
Shapiro-Wilk and Kolmogorov-Smirnov normality testStep 12: choose the statistical test
The test depends on your objective, the type of variables and normality. This table covers about 90% of theses:
- Relationship between two numeric or ordinal variables: Pearson correlation if normal; Spearman’s rho if not, or if ordinal.
- Relationship between two categorical variables (for example, climate level and performance level as low, medium, high): chi-square test of independence, with Cramér’s V for effect size.
- Comparing two independent groups (men and women, two sites): Student’s t (or Welch’s if variances differ) if normal; Mann-Whitney U if not.
- Comparing three or more groups (three shifts, four schools): one-way ANOVA if normal; Kruskal-Wallis if not. If significant, pairwise comparisons show which groups differ.
- Comparing the same group before and after (pretest and posttest): paired t-test if the differences are normal; Wilcoxon if not.
- Explaining or predicting one variable from several at once: multiple linear regression.
A frequent clarification: in a correlational design, a significant correlation does not prove one variable causes the other. Write "is related to", not "influences", unless your design allows it.
The calculator for each test, with an APA table:
Correlation calculator (Pearson and Spearman)Step 13: reading and reporting results in APA
Each hypothesis is reported with the same structure: null and alternative hypotheses, the test used, the table and the decision.
- State H0 (no relationship or difference) and H1 (there is one).
- State the significance level: α = .05.
- Present the table with the statistic, degrees of freedom where relevant, the p value and the sample size.
- Decide: if p is below .05, reject H0 and accept H1. Otherwise, do not reject H0.
- Interpret direction and strength, not just significance. A correlation of .15 can be significant with 400 people and still be very weak.
Example: "A moderate positive relationship was found between workplace climate and teacher performance (rho = .52, p < .001, n = 175). The null hypothesis is therefore rejected." For correlation strength, a common guide is: below .30 weak, .30 to .49 moderate and .50 or above strong.
In the discussion, compare your results with previous studies: where they agree, where they differ and why that might be. That is where it shows whether you understood what you found.
For the descriptive part and open answers, if your thesis has them:
How to analyze survey resultsThe most common committee comments
- The objective says "determine the influence" but the analysis is a correlation.
- The research matrix does not match what was analyzed in the results.
- There is no content validity, or "validated by experts" is reported without coefficients.
- Cronbach’s alpha was calculated on the final sample instead of a pilot, or only for the full instrument and not per dimension.
- The sample size is not justified with a formula or criterion.
- Pearson is used when the normality test gave p below .05.
- "p = .000" is reported. Write p < .001.
- Only significance is interpreted, not the strength of the relationship.
- Reverse-worded items were not reversed before adding.
- Informed consent or the institution’s permission is missing from the appendices.
The full path as a list
- Question, objectives and hypotheses: research consistency matrix.
- Variables, dimensions, indicators and items: operationalization table.
- Existing or new instrument; items written on the same scale.
- Expert judgment: Aiken’s V, Lawshe, CVC, CVI or binomial.
- Pilot test: Cronbach’s alpha or omega, per dimension.
- If needed, construct validity: KMO, Bartlett and factor analysis.
- Population and sample: finite population formula or power analysis.
- Data collection with informed consent and institutional permission.
- Descriptives and levels from score ranges.
- Normality: Shapiro-Wilk or Kolmogorov-Smirnov.
- Hypothesis test based on objective and normality.
- APA reporting, interpreting strength and direction, and a discussion against previous studies.
Go step by step and each chapter rests on the one before it, so committee comments shrink to formatting details. And if you get stuck on a calculation, every tool in this guide is in the thesis tools section, free and with no sign-up.