Posted by Marjorie Yoro / May 25, 2016

Pros and Cons of Online Research Panels

Pros and Cons of Online Research Panels

In our last post, Understanding Research Panels, we talked about what an online research panel is and how it works. Your next step is to figure out if conducting a survey online is the right methodology for your study.

It may seem simple enough to type a questionnaire and send it out to netizens. But the truth is, there are a lot of factors to consider. Doing away with the physical contraints of traditional market research methods, like door-to-door surveys or telephone calling, doesn't mean that all your problems are gone.

Online surveys are different animals altogether. You just have to know if it's the right one for your particular research project. 

 

Pros

1. Ease of Sampling and Incentivization

Members of online research panels answer profiling questions which makes it possible for them to be identified as qualified for a study. Because they are pre-profiled, you don’t have to actually step out of your office to find the respondents you need for your study. You’ll be able to reach hundreds of people instantly, and provide appropriate incentives immediately after they finish the survey. Fieldwork that used to take months to wrap up can be reduced to days, even hours. 

2. Minimal Cost

Before, you had to print out questionnaires and provide pens to your respondents. Oftentimes, you also had to invite them for an interview at some facility or call up each one to remind them of the meeting. In the case of online research panels, all of those things are off the expense list.

The questionnaire is programmed online and can be revised real-time, without having to waste any paper. Email reminders may also be sent automatically to respondents, at the most optimal time window of their day. 

3. Automated Data Collection

After a respondent completes a survey, his answers are automatically stored in a database. No more manual data input, which means less data errors for researchers. 

You can also customize how the data comes in by setting quotas (e.g. on age). So that when a quota is full, the system can block respondents who are trying to enter the survey for that particular quota. This is a great way to keep the fieldwork in check. Some researchers prefer to first open the survey to the quotas that are hardest to fill (e.g. elders), and then open it to the easiest quotas (e.g. 20-30’s bucket) towards the end of the fieldwork. 

4. Customizable Design

Questionnaires can be programmed to fit any screen, whether the respondent is using his desktop, tablet, or mobile. It can be programmed to skip questions based on previous answers and set other complex logic.

You'll also have the option to embed media— audio or video —within the survey. This makes for a richer survey experience for the respondents.

5. Convenient for Respondents

Online surveys are very easy to use for panelists. They can decide which surveys they want to take, skipping those that are irrelevant to them. They can decide when they take it, whether it's on their way to school or their 5-minute break at work. In some cases, surveys can be designed so that respondents can pause in the middle of the survey and then continue at a later time.

Learn about Mobile Research


Cons

1. Limited to Internet Population

Naturally, those who will be able to participate in online surveys are those who have access to the internet. If you want to reach countries or rural areas with low internet accessibility, you are better off conducting traditional face-to-face interviews. But if your research is focused on developed markets or major cities, the online methodology is perfect for you. 

2. Respondent Fraud and Bias

Unlike traditional research methods where you can verify the identity of a respondent face-to-face, members of online research panels may decide to embellish their income range or job role, thereby committing fraud. This can lead to respondents answering surveys that are not intended for them. And so, data quality suffers.

At other times, even if respondents are who they say they are, they may decide to answer surveys based on what they think will yield more incentives; the survey therefore eliciting biased responses.

Rest assured that online panel companies, the good ones anyway, go to great lengths so that neither of these scenarios occur. dataSpring, for example, uses Digital Fingerprinting Technology, which is automatically embedded to the respondents' browser to prevent them from joining the same survey. 

3.  Multiple Panel Participation

This is a risk for those who have allocated samples to multiple panel companies. Data quality may be compromised when a respondent is a member of two or more panels. This is because that respondent is likely to receive the same survey. If he answers the same way, you will have duplicate data. And if he answers differently, he's biased. Either way, you’re getting data from the same person twice, and this is not something that helps with the veracity of research findings.

When looking for an online sample provider, make sure that they are feasible for all the samples you need so that you don't need to reach out to other providers. However, if the sample size is simply too big for any one supplier, you need to make sure that your partner is using their proprietary panels. And, if not, demand them to disclose all the panel sources they are using. 

4. Researcher Bias

This can happen with other survey methods, but it’s worth mentioning here as well. The way that the targeting criteria is selected and the way that the survey is designed can have a big impact on your data.

To avoid this, consult with your colleague. Get a second opinion, get a third opinion. Consult with your online research partner. They should have insights about the suitability of the target criteria and survey design for your project.    

Download Mobile 101: A Primer for Researchers

Finally...

Conducting a survey online has a lot of benefits. But, as we've seen, it also has its risks. If you decide that the pros far outweigh the cons, you should make sure to consult with your online panel partner for their measures to mitigate these risks.

A good start is to ask them for their ESOMAR28. Learn as much as you can about how they manage their panels and ensure the quality of their data.

 

Related Readings:

Setting Research Objectives: Scope and Clarity
Online Panels: The Black Sheep of Market Research?
Are the Best Days of Online Research Panels Behind Us?

 

New Call-to-action

Topics: Online Market Research, Online Research Panels

Contact us anytime 24/7. One of our Springers will be in touch with you within 24-48 hours to follow up on your request.

CONTACT US