Back to blog
·6 min read

How to write survey questions without bias

Two surveys can ask essentially the same underlying question and get completely different results, just because of how the question is worded. Wording bias is one of the most common and least discussed problems in survey design, because it doesn't show up looking at a single question. It shows up when you compare results to what you actually expected to find.

Leading questions

A leading question suggests the "correct" answer within its own wording. "How much did you enjoy our excellent service?" already assumes the service was excellent. The neutral version is simply "How would you rate our service?", with no adjectives pushing the answer in a direction.

Double-barreled questions

Asking two things at once in a single question forces someone to answer with one number something that actually has two separate answers. "How satisfied are you with the speed and friendliness of support?" mixes two dimensions that can have opposite results. Split them into two separate questions.

Social desirability

People tend to answer what they believe is expected of them, especially on sensitive topics like ethics, compliance, or habits. Asking directly "do you always follow the safety protocol?" will inflate the yes answers. Reframing toward specific, recent behavior, like "the last time you did X, did you follow the full protocol?", reduces this effect.

Order effects

The order questions appear in, and the order of options within a question, both influence the answer. Asking about something negative first can put someone in a mindset that affects how they answer what comes next. If you're going to compare results across surveys, keep the same order between versions so this doesn't introduce extra noise into the comparison.

Absolute language

Words like "always", "never", or "everyone" in a question push people toward disagreement, because almost no real situation is that extreme. "Do you always get useful feedback from your manager?" will get almost no yeses, even when the feedback is good most of the time. "I get useful feedback from my manager as often as I need" measures the same thing without forcing an extreme.