Quantitative Data Collection: Develop a stellar questionnaire by taking basic design concepts to the next level

This is the second of a two-part series on quantitative market research data collection. It explores survey-design considerations. The November eTip! addressed issues related to choosing an appropriate platform for quantitative research questionnaires.

Designing a questionnaire to yield information that helps you address key business decisions is central to quantitative market research. So, while your data collection platform¹ will drive many survey-design decisions, you can maximize response rates and data quality by adapting the following design concepts to your study.

Length

The temptation to roll everything, plus the kitchen sink, into a single study is common, but catch-all studies² rarely yield useful results. Respondent fatigue and drop out will skew data if your questionnaire is too long, so keep it as concise and focused as possible. Ideally, your survey should take no longer than 30 to 45 minutes to complete.

Evaluate each potential question carefully, and ask yourself how each bit of data will be used to improve your strategic planning, market position, etc. Leave out all unnecessary content. Then, consider the mix of question types; what’s the ratio of open- to closed-end questions?

While open-end questions are useful when you cannot easily capture all possible answers to a question, closed-end questions are easier to answer and analyze. They minimize the time it takes a respondent to complete the questionnaire. Information gathered from related exploratory research can be used to develop fixed-alternatives for survey questions.

Format

The presentation of elements in your questionnaire can impact research results significantly. Minimize respondent bias by implementing the following format suggestions:

• Instructions
  To establish a steady flow and minimize the need for respondents to back up or request help, provide instructions as they are needed instead of grouping them at the beginning of the survey.
• Question Sequence
  To gauge awareness of an idea or brand, be careful about positioning questions that include the concept or product. Once mentioned, a brand will remain in a respondent’s mind as (s)he answers related questions. Also consider the content of the invitation and screener questions to avoid influencing respondent data.
• Answer Choice Sequence
  For closed-end questions related to preference or recall on self-administered questionnaires, people may show bias toward answer options that appear at the beginning of a list, especially if a survey or list of answers is lengthy. In contrast, bias in phone interviews is toward answer choices that respondents hear last. Phone and web-based surveys can present answer choices in a random order to minimize these types of bias.
• Scales
  To gather meaningful ratings for product attributes that are likely to be high across all brands, use a comparative scale instead of an odd- or even-numbered scale. Below is an example of a comparative scale in a grid format:
• 

Pre-Test

Pre-testing is an important step of the research process that is sometimes eliminated due to cost and time constraints. Depending on your available resources, one of the following three pre-testing options can be used to test your research instrument before fielding it: