The Importance of Sample Size

Deploying a survey can be an intricate process. One commonly overlooked aspect of surveying is: how do you know when you have collected enough survey responses to insure your data is valid?

In other words, how do you know that the data you have collected can be confidently used to make important business decisions? Determining the minimum number of survey responses requires 3 pieces of information:

The size of your population (examples are your total number of employees or customers)
The confidence level you require (90%, 95% or 99% are common choices)
The sampling error level you are willing to tolerate (5% is a common choice)

Confidence level describes the reliability of the data you have collected with your survey. A survey response from 100% of your employees or 100% of your customers or from any other population is almost always impossible to achieve. Hence, the data that you do collect is deemed a sample. That being the case, the term confidence level is used to describe the reliability of your sample. The higher the confidence level you require, the more people you must survey.

Sampling error, also known as the confidence interval, is the term used to describe the uncertainty that is associated with your data. You would like this number to be as low as possible.

Lastly, you should be familiar with what is known as a representative sample, which is the number of responses needed from your population to reach valid data at the stated confidence level and sampling error. Generally speaking, it is best to achieve a confidence level of at least 95% with a sampling error of no more than 5%.

One important caveat to remember is that reaching a representative sample is never based upon obtaining a response from a certain percentage of your target population.

As an example, let’s imagine you’re conducting a survey of 100 employees. You must obtain at least 80 responses, or an 80% response rate, to insure that your data is valid at a 95% confidence level with a 5% sampling error.

For comparison, pretend you are surveying 5,000 customers. You only need 357 responses, or a 7.14% response rate, to have valid data at a 95% confidence level with a 5% sampling error. As you can see, you don’t need all of your targets to complete the survey to have valid data and reaching a representative sample has very little to do with surveying a certain percentage of a population.

One last concept is sub-populations. Sub-populations are an important part of reaching a representative sample. Assume your 5,000 customers are equally split across North America, Canada, South America, Europe, and Asia. This means that you have 5 sub-populations of 1,000 customers each. It is important to reach a representative sample of responses on each of the 5 sub-populations if you wish to make business decisions that pertain only to one of the sub-populations. A population of 1,000 requires 278 survey responses to reach a representative sample at a 95% confidence interval with a 5% sampling error. Therefore, you will need 5 X 278 or 1,390 responses if you want valid data at the sub-population level.

We’ve condensed this knowledge into a sample size calculator that can be used to determine the required sample size for your surveys. Just fill in the blanks, collect the number of responses indicated and you will have reliable data. Contact NBRI today if you have any questions about surveys and how they can help your business grow.