Survey Incentives: Response Rates and Data Quality

Offering incentives for a survey respondent’s time is one way to increase customer survey response rates. In fact, the use of a monetary or gift incentive has been proven to significantly increase response rates in many customer surveys. The question is not whether incentives will increase response rate percentages, as this remains true throughout many examples. Instead, what kind of effect does the incentive have on data quality? Are the survey results positively or negatively biased because respondents are given an incentive to participate? How much incentive do I need to include with my survey for it to be effective?

Several studies have indicated that the use of incentives reduces to some extent item non-response and “bad answers”, such as “don’t know” or “no answer”. It was also noted in a study published by Public Opinion Quarterly that respondents who received incentives have lengthier answers to open-ended questions. There are logical reasons for these findings. When you offer someone an incentive, they will view completing the survey like returning a favor and will feel obligated to do a good job. This is known as the “norm of reciprocity”. Respondents who receive an incentive are also more likely to say the survey subject matter was interesting, and this causes them to place a greater value on their task. Though it seems likely that offering an incentive would bring apathetic participants to the study, research has proven otherwise. The data quality with an incentive, therefore, can actually be considered higher than if the incentive was not offered, as respondents have put more thought into answering the survey questions. There is also evidence that providing incentives will increase respondents’ willingness to participate in future studies because they complete the customer survey feeling positively about the overall experience.

There is also the question of demographic bias. Is a certain group of people more likely to respond to a particular incentive, thus biasing the results toward that demographic group? For example, it is reasonable to assume that offering a monetary incentive would cause greater response from a low-income demographic group. It also makes sense that offering a beauty product would generate biased results because more women would respond than men. A research study conducted at Penn State University indicates, however, that the kind of incentive offered does not affect the demographic sampling of the results. When presented with an incentive, people generally feel obligated to return the favor regardless of the type of gift they received. There is no indication of a group-specific effect of incentives.

How much of an incentive is needed for maximum effectiveness? An incentive is not usually warranted for an employee survey, as most employees want to participate and want to give honest feedback to improve the working environment for themselves and their co-workers. For small business populations (less than 500) participating in a customer survey, the incentive needs to be at least $10 and possibly up to $50 per respondent because a very high response rate percentage (40%+) may be required to achieve a representative sample. However, the availability of a larger business population (greater than 500) of potential respondents lessens the need to achieve a very high response rate and thus it is easier to achieve a representative sample of respondents using a smaller incentive. For private individuals and households, studies have shown that even a very small incentive can increase response rate percentages considerably. Regardless of the size of the incentive, people generally feel obligated to participate in customer surveys if an effort is made to show appreciation for their time. According to information presented at a recent Society for Industrial & Organizational Psychology’s Annual Conference, even finding a $1 bill in a paper survey has been shown to surprise the participant, generating more interest in the survey, and higher response rate percentages.

There are many occasions when it might be a good idea to add an incentive for your survey respondents. Do you have a lengthy survey that may take up more than 20 minutes of the participant’s time? Are you conducting a survey of a small population in which you need a higher-than-normal response rate percentage to achieve a representative sample? The logical concerns about the effects of incentives on data quality have been disproved by countless studies, leaving only the fact that an incentive is a practical, credible option to increase response rate percentages for your customer surveys without sacrificing the validity of your results.

If you would like to know more about how NBRI can help you, please contact us at 800-756-6168.