Questionnaire Context, Order, and Meaning

When designing questionnaires, the order of items on the questionnaire creates a context, or meaning, for the entire questionnaire. Items placed early in the questionnaire affect the way in which people respond to later questions.

“Each item will be interpreted by respondents within the context of the questionnaire, previous questions, and the wording of each item,” notes Gregory G. Holyk, University of Illinois at Chicago, in the Encyclopedia of Survey Research Methods.

To ensure positive use of context in questionnaires, it is important that you clearly define the purpose of the questionnaire. Start by writing drafts of items to be included on the questionnaire – items that directly relate to the purpose of the questionnaire. The next step is to put items into a meaningful format (e.g. open-ended, multiple choice, etc.) and order. The order in which items appear can bias people’s thinking and, thus, their responses to your questions.

After introducing the survey’s general purpose and content, the first question should be easy and nonthreatening; it sets the tone. Difficult or sensitive questions are best asked later, once rapport is established.

Items should flow in a logical order, with similar content grouped together. Providing brief directions or a transition statement between sections of the questionnaire helps prepare respondents for a change of pace and lets them know that a different mindset is necessary. It also helps retain interest and attention.

In what order should general/specific questions asked? Some researchers contend that if specific questions are asked first, the attention given to them will influence the responses to general questions. Starting with more specific questions about various aspects of the issues can give respondents some background on which to base their replies to subsequent, general question(s).

Research suggests that context effects are most noticeable in attitudinal surveys. A between-question context effect occurs when the questionnaire contains items about a particular subject, like health care, and then suddenly changes to a totally different subject, like the outcome of a presidential election. A within-question context effect occurs when words frame an issue from an unusual perspective. Example: Holyk proposes that the use of ‘anti-abortion’ instead of ‘pro-life’ affects attitudes toward abortion. The way in which questionnaire items are worded affects responses and ultimately survey findings.

Response alternatives are the ‘multiple choice answers’ from which respondents make their selection. In crafting the questionnaire, you must decide whether or not it’s important that respondents search their memory and recall a response or simply be able to recognize a response from a list. If you provide a list of these choices, or response alternatives, know that you are making assumptions about the range of opinions or behaviors held by the population being researched or introducing your own bias.

The technique of using skip directions (i.e., if yes, go to question 6; if no, go to question 7) can become confusing and time consuming, so should be used sparingly. Enough white space must be left to record answers to open-ended questions, usually about three lines worth of space. The page must not be cluttered, since clutter can lead to errors in responding.

Having taken context effects into account in designing the questionnaire, the questionnaire must be pre-tested. Item wording, format, and order must be revised, as necessary before pre-testing the questionnaire again. A successful questionnaire is one that flows smoothly without leading questions or bias.

If you don’t have the in-house expertise or time to attend to the details of questionnaire design or flow, contact NBRI at 800-756-6168.