Introduction
Designing a survey can seem like a simple task, however there are many common pitfalls that can cause your survey to become ineffective and inaccurate. One of the most common pitfalls in survey design is ambiguous questions. Ambiguous questions are what happen when a survey designer doesn’t think about how the respondents will interpret their survey. To get this most out of surveys and market research, you must always put the respondent first.
What is an ambiguous survey questions
An ambiguous question is defined as one where there is no specific query. You could also describe an ambiguous question as a confusing question. An ambiguous question is one that a respondent will struggle to answer, not because they are a bad respondent but because it is a bad question. Generally, ambiguous questions are too broad and leave room for interpretation by the respondents. Examples of an ambiguous question include a question that could have more than one meaning, asking for several responses, or not clearly defining the subject/object.
Example of an ambiguous question
“Please rate the speed and quality of our customer service.”
This question is an ambiguous question because the respondent is being asked to rate two things - speed and quality. It is very possible that the respondents received speedy customer service but the actual quality of the service was poor, or vice versa. However, because of the framing of the question, a respondent will struggle to respond to this question, especially if there is no “other” or text input option. The most likely outcome is that an inaccurate or at least incomplete response will be given and this means that your survey is gathering misleading data.
How to avoid ambiguous survey questions?
Follow your objectives
If the researcher doesn’t know exactly what you are asking your respondents, then your respondents definitely won’t know how to answer your question. Therefore, you need to set survey objectives before even thinking about your survey designs. Keep these objectives in mind when you are setting your questions. This helps you keep your questions specific and generally avoids ambiguous questions. Learn more about setting survey objectives here.
Be specific
Again, ambiguous are usually broad questions, so to avoid that be specific. Let’s be more specific about how to be specific in your survey questions. Here are a few tips:
- Avoid long, wordy questions that could lose respondent interest
- Don’t use vague words
- Specify the context of the question
- Avoid double negatives at all costs
However, as much as you might think using jargon, abbreviations or acronyms will keep your questions short and specific, don’t use them! We can guarantee you that most of your respondents aren’t familiar with jargon that you as a researcher use. Using jargon will just confuse respondents, so avoid them!
Split your questions up into multiple parts
To resolve the issue with the example above is to split it up into multiple parts. If you are asking respondents to rank something, break it down into different areas. So in the example above, split the questions into 1) rate the speed of the service, 2) rate the quality of the service. An added benefit of this is that you get more detailed responses. Don’t be afraid to split up questions even if that means that you have more questions. More questions isn’t a bad thing in this case especially if they are short questions, such as rating questions. A couple of short questions is better than one long ambiguous question.
Test your survey
Put yourself in the shoes of a respondent and take the survey yourself. Enlist some colleagues or friends who aren’t familiar with the survey to take the survey too. Why is this important? If you, your colleagues and your friends give different types of responses to the same questions, or there are some unexpected responses, then you should probably review the questions. Also you can see if respondents will struggle with the questions. If they do, then you should definitely reword your questions.
Conclusion
Take your time with survey design. With each question, think about how a respondent will read the question and interpret it. Respondent experience should always be at the heart of survey design. Respondents must be able to understand your questions to answer them correctly. It's as simple as that. If you are getting weird or inaccurate responses, most likely it is your fault, not the respondent’s and you need to look for and rectify any ambiguous questions in your survey.