How to Conduct a Survey – Deploying the Survey

From the 3 previous “How To” white papers you should understand the basics of creating a survey. You should have objectives for your research and a list of questions that you want to ask. This type of research often drives mission critical decisions so it is important to get the previous steps right.

Now let’s begin our discussion on how to gather survey data.

Paper, online, telephone, or in-person interviews?

Now is the time to consider your deployment options. Each method has its advantages and disadvantages. Often, deciding upon the method of deployment is easy since you may be constrained by such obstacles as the number of email addresses you have available to you, the languages required, cost, your company’s infrastructure, etc. For instance, translating a survey into multiple languages and distributing it to your customers via email is probably more cost effective than hiring people to telephone each customer in their own language to administer the survey.

Every situation is different so you must decide what will work best for your organization. Maybe it’s a paper survey or an online survey or a combination of different methodologies.

Paper Surveys

Paper is the most universal of all survey deployment methodologies – globally. Paper surveys have the benefit of reaching anyone who can read and write.

Paper surveys can be delivered by hand, mailed to participants, or simply made available for “pick up” in a convenient location. The surveys are then filled out by the respondents, and collected again. If mailed, you will need to consider the cost of postage (both ways) which can be a significant part of the expense of your project if you are thinking about deploying thousands of surveys.

Don’t forget that at some point you will need to collect the survey responses from all of the completed paper surveys. This can be done via manual data entry or via a scanner in order to get your data into an electronic format that lends itself to analysis.

One drawback to paper surveys is that they can produce a low response rate for customer surveys, typically 10 – 15%. For employee surveys, the response rate for paper surveys is usually 75% or higher.

If all you have are mailing addresses for your potential survey respondents, then a paper deployment is the way to go.

Online Surveys

One of the benefits of an online deployment is that the survey data is automatically collected in a digital format for easy manipulation and analysis. Online surveys can also produce higher response rates than paper surveys, typically up to 30% for customer surveys. Once again, response rates for an employee survey via an online deployment are usually 75% or higher.

An online deployment starts with the sending of invitation emails, followed by reminder emails to those who have not yet participated. These emails will usually contain a clickable link that will log the respondent into the survey so that they can answer your survey questions.

Once a respondent is logged into your survey they experience the familiar look and feel of a Windows or Apple desktop computer, lap top, or smart phone. And, as opposed to a paper survey, online surveys allow for enhanced features such as branching to a sub-set of questions for certain respondents.

An online deployment can be launched very quickly and you can have your data within a few days.

All things considered, the cost of an online deployment is lower per response than any of the other deployment methodologies. All you need are email addresses for your target population.

Telephone Surveys

Telephone surveys offer the advantage that the survey respondent has to exert very little effort. By simply taking a few minutes over the telephone to answer the questions, he/she is done. This may be why telephone surveys can produce the highest response rates, up to 40% on customer surveys. Employee surveys are almost never deployed via telephone, but when they are, the response rates, once again, are usually 75% or higher.

Except for the fact that a telephone deployment is initiated by an interviewer placing a telephone call instead of an invitation email being sent, telephone surveys operate the same as an online survey.

One disadvantage of telephone surveys can be the cost. Human beings must place telephone calls to all of the potential survey respondents. This can be an expensive undertaking. And, it takes time to call large numbers of potential respondents, which can delay your data collection. But, if all you have are telephone numbers for your potential respondents, then a telephone deployment may be the only option available to collect your data. Telephone deployments offer the same high quality data collection as all of the other methodologies.

In-person interviews

In-person interviews can be the most costly of all survey deployment methodologies unless you have “free” labor and free travel expenses at your disposal. Even with “free” labor, there is still the issue of training your interviewers or obtaining the services of trained interviewers who can collect your data without biasing the responses one way or the other. Sending an interviewer to each customer or employee to gather your data could easily triple your cost without delivering a single advantage over any of the other deployment methodologies. In-person interviews are generally not recommended.

Anonymity and Confidentiality

Anonymity and Confidentiality are very important aspects of any research study. Every survey respondent must feel that they can submit complete and honest answers to your survey questions without the fear of repercussions or the hope of gaining a benefit. If your respondents feel that they could be identified or singled out in any way (good or bad), it will affect their responses. This is particularly true with regard to employee surveys. Employees may fear repercussions from their immediate supervisor and shape their answers accordingly. This would result in invalid data. Many organizations choose to hire a third party firm to conduct their surveys to insure anonymity and confidentiality for their survey respondents.

Representative sample

Because each employee or customer is not going to participate in your survey, you are necessarily dealing with a sample (subset) of your target population. Therefore, you must obtain enough survey responses to be sure that your sample truly represents the thinking of the whole population (“representative sample”) or your data will be invalid. Decisions based upon invalid data could hurt instead of help your organization.

There are well established statistical principles that can guide you in this area and allow you to know if and when you have reached a representative sample. First, all samples have confidence levels (or intervals) and sampling error. The confidence level is the probability that the sample mirrors the whole population. The sampling error is the probability that there are differences in the sample from the whole population.

Generally speaking, it is best to achieve a confidence level of at least 95% with a sampling error of no more than 5%. One important thing to remember is that reaching a representative sample is never based upon obtaining a response from a certain percentage of your target population. Run as fast as you can away from anyone who tells you otherwise. A couple of examples may help to drive this point home.

For example “A,” let’s pretend that you are deploying an employee survey for a company with 100 employees. In order to reach a representative sample at a 95% confidence level with a 5% sampling error, you must obtain at least 80 responses. Note that this is an 80% response rate.

For example “B,” let’s pretend that you are deploying a customer survey to 1,000 of your customers. In order to reach a representative sample at a 95% confidence level with a 5% sampling error, you must obtain at least 278 responses. Note that this is a 27.8% response rate.

Incentives

Offering incentives for a survey respondent’s time is one way to increase survey response rates without biasing respondents’ answers. How much of an incentive is needed for maximum effectiveness? An incentive is not usually warranted for an employee survey, as most employees want to participate and want to give honest feedback to improve the working environment for themselves and their co-workers. For customer populations of less than 500, the incentive needs to be at least $10 and possibly up to $50 per respondent because a very high response rate percentage (40%+) may be required to reach a representative sample. For larger customer populations (greater than 500), the need to achieve a very high response rate lessens, and so it is easier to reach a representative sample using a smaller incentive or no incentive at all.