Survey Bias definition
Survey Bias or response bias is a general term for a range of cognitive biases that influence participants to reply to an inaccurate or dishonest response.
Types of bias
- Researcher Bias
The viewpoint of the researcher has a way of creeping into the survey. All research designers are human and have their own views and opinions. Even the most practiced and professional researchers can have subtle biases in the way they word questions or interpret results.
- Poor match of the sample to the population
It is almost never the case that the sampling frame you use is a perfect match to the population you are trying to understand, so this error is present in most studies. You can sometimes recover from asking the wrong questions, but you can never recover from asking them of the wrong people.
Most people like to focus on questionnaire development when a new project is awarded. The reality is the sampling and weighting plan is every bit consequential to the success of the project, and rarely gets the attention it deserves. We can tell when we have a client that really knows what they are doing if they begin the project by focusing on sampling issues and not jumping to questionnaire design.
- Lack of randomness/response bias
Many surveys proceed without random samples. In fact, it is rare that a survey being done today can accurately claim to be using a random sample. Remember those statistics courses you took in college and graduate school? The one thing they have in common is pretty much everything they taught you statistically is only relevant if you have a random sample. And, odds are that you don’t.
A big source of “non-randomness” in a sample is survey bias. 10% is considered a good response rate from an online panel. When we report the results of these studies, we are assuming that the vast majority of people who didn’t respond would have responded in the same way as those who did. Often, this is a reasonable assumption. But, sometimes it is not.
- Failure to quota sample or weight data
Even if we sample randomly, it is typical for some subgroups to be more willing to cooperate than others. For example, females are typically less likely to refuse a survey invitation than males, and minorities are less likely to participate. So, a good researcher will quota sample and weight data to compensate for this. In short, if you know something about your population before you survey them, you should use this knowledge to your advantage.
- Social desirability bias
This is a form of survey bias in responses caused by respondents' desire, either conscious or unconscious, to gain prestige or appear in a different social role.