Preparing and conducting an online survey can be a tricky process, as you may need to keep guidelines in mind as you try to keep your questions short and understandable. Moreover, you need to keep your surveys as objective and neutral as possible, so that you will be able to preserve the integrity of the responses and come up with beneficial insights. To do this, you must be able to identify possible biases in your survey and distribution process.
Let’s talk about some of the types of biases in online surveys, and how we can reduce them.
Sampling bias
In an ideal survey, all your target respondents have an equal chance of receiving an invite to your online survey. However, since you are conducting your survey online, you might be restricting your survey’s availability to respondents who are more active online, especially those who have social media accounts. Your respondents’ level of online engagement varies when you consider factors like age and income. When some of your respondents become less likely to be surveyed than others, your survey might be affected by sampling bias.
To best way to reduce sampling bias is to distribute your survey to various online channels to improve its visibility among your respondents. You can share your survey through social media, email blasts, survey websites, messaging apps, or even through QR codes. If needed, you can also provide an offline push through phone or text message reminders.
Nonresponse bias
Aside from the scenario mentioned above, there is also the possibility that those who answered your online survey somehow appear to be systemically different from those who didn’t respond. For example, you might notice that your respondents are mostly men, even if your survey didn’t have restrictions or parameters on gender. This type of bias can still happen even if you have distributed your survey across many channels.
Increasing the response rate in your online survey will help improve the chance that your target respondents will be well-represented. However, you can’t fully control the response rate due to a host of factors, like your survey topic and your target population. You can help manage this by sending a pre-notification email and personalized invite to your respondents, as well as sending a survey reminder.
Response bias
While answering your survey, your respondents may also exhibit forms of bias, which happen when subconscious and conscious factors result in less-than-truthful responses. Here are the different forms of such biases:
- Acquiescence bias – Better known as yea-saying, it is a form of bias where your respondents will tend to tell you what you want to hear, as it’s human nature to be agreeable.
- Demand characteristics – This happens when your respondents become overly aware that they are part of your survey, making them double over their answers.
- Extreme response – This form of bias appears prominently in five-point scale items when respondents tend to select the extreme options. This form of bias is culture-specific, appearing frequently in individualistic societies. The opposite version of this, where people tend to answer neutral responses, happens more in collective societies.
- Desirability bias – Similar to yea-saying, this form of bias is exhibited as a sign of self-preservation. It is human tendency to appear and behave desirably and avoid undesirable traits.
Fortunately, there are many solutions to address response biases, and one sure way is to enable self-administered surveys so that your respondents will be more honest. Other ways to reduce response bias is by using neutrally worded questions, avoid leading questions and answers, allowing anonymity, and removing your brand or company identification.
Order Bias
The order of questions and answers in your online survey can also influence the perception of your respondents. For example, respondents may experience the assimilation effect, where their response to a latter question is more similar to earlier questions than it would be if it preceded it or was asked on its own. Conversely, the respondents can experience the contrast effect, where their response to a latter question is extremely more different from a previous question than it would be if it preceded it or was asked on its own.
Order bias can be reduced by minimizing the number of scale questions, grouping survey items by topic, leaving demographic questions until later in the survey, asking questions that engage respondents, and randomizing your question and answer options. Additionally, you can conduct test runs of your online survey to fine-tune it.
Not all surveys run smoothly and without a hitch - it takes repeated checks to remove all signs of possible bias that can affect your study. It is highly recommended for you to have your survey and process checked, or even programmed, by an external group to avoid biases that may exist within your study. By utilizing a quality sample provider like dataSpring, properly drafting the screener and questionnaire, and utilizing the experience of your supplier, you can help eliminate this issue.
For more information on how to design your questionnaires properly, check out our market research survey essentials, or download our online survey terms basics brochure.