Writing survey questions without bias
Why wording matters
A survey reaches many people cheaply, but it cannot ask follow-up questions. If a question is unclear or nudges people toward an answer, the results look precise yet mislead. Good survey design removes those nudges before the survey is sent.
Common wording problems
Leading
“How great was your experience?” assumes it was great. Ask “How would you rate your experience?”
Loaded
Emotional or judging words push answers. Replace “wasteful ads” with “ads”.
Double-barrelled
“Is the app fast and easy to use?” asks two things. Split it into two questions.
Jargon
Use the words your respondents use. Avoid internal names and abbreviations.
Design the answer options
- Exhaustive: cover every realistic answer, and add “Other” or “None of these” when needed.
- Mutually exclusive: ranges must not overlap. Use “18–24, 25–34”, not “18–25, 25–34”.
- Balanced scales: offer the same number of positive and negative options, such as from “Very dissatisfied” to “Very satisfied”.
A neutral midpoint (for example “Neither agree nor disagree”) lets people who have no opinion say so. Removing it forces a choice, which can be useful, but decide deliberately and keep the choice consistent across the survey.
Closed questions are easy to analyse. Open questions reveal reasons you did not anticipate but take longer to read. A common pattern is closed questions first, with one optional open question at the end.
Choosing closed or open questions
The question type should follow the decision the survey supports. Pick the simplest type that gives a usable answer.
Single choice
One answer from a list. Good for categories such as role or platform.
Multiple choice
Several answers allowed. Say so clearly: “Select all that apply”.
Rating scale
Measures agreement or satisfaction. Label every point or at least both ends.
Open text
Captures reasons in people’s own words. Use sparingly because it takes effort to answer and to analyse.
Make required questions the exception. Forced answers push people to guess or leave.
Order, length and testing
Earlier questions can influence later ones. If you ask about a specific problem before asking for overall satisfaction, the problem is fresh in mind and may lower the score. Put general questions before specific ones and keep related topics together.
- Cut every question that does not support a decision. Short surveys are completed more often.
- Pilot the survey with a few people from the target group.
- Ask them to explain each question in their own words; fix any misreading.
- Send only after the wording is stable.
If two people read a question differently, the data will mix two different answers.
This guide was drafted with AI assistance from public standards and references. For the exact wording of a standard, use the sources listed above.