Questionnaire Method in Research: QUESTIONNAIRE DESIGN AND SCALE DEVELOPMENT

Questionnaire Method in Research

What Is a Questionnaire?

A questionnaire is a research instrument consisting of a series of questions to gather information from respondents. Think of it as a written interview, which can be carried out face to face, by telephone, computer, or post.

“Questionnaires provide a relatively cheap, quick, and efficient way of obtaining large amounts of information from a large sample of people.”

Data can be collected relatively quickly because the researcher doesn't need to be present during the completion of the questionnaires, making it ideal for large populations when interviews would be impractical.

However, a problem with questionnaires is that respondents may lie due to social desirability bias. Many people want to present a positive image, which might lead to dishonest answers.

The overriding objective is to translate the researcher’s information needs into a set of specific questions that respondents are willing and able to answer. While this may seem straightforward, questions may yield very different and unanticipated responses. For example, how would you answer the following question: "Which State is larger, California or Texas?" Would you answer based on population or area?

Why Is a Questionnaire Important?

A questionnaire is the main means of collecting quantitative primary data. A questionnaire enables quantitative data to be collected in a standardized way so that the data are internally consistent and coherent for analysis. Imagine how difficult it would be to analyze the data of a national survey conducted by 40 different interviewers if the questions had not been asked in a standard way, that is, if the interviewers had asked different questions using different wording and order. A questionnaire ensures standardization and comparability of the data across interviewers, increases speed and accuracy of recording, and facilitates data processing.

QUESTIONNAIRE DESIGN AND SCALE DEVELOPMENT

Questionnaire Design Process

No scientific principles guarantee an optimal or ideal questionnaire. Questionnaire design is as much an art as it is a science. The creativity, skill, and experience of the researcher play a major role in the end design. However, several guidelines are available to assist researchers in the questionnaire development process and to help them avoid major mistakes. The guidelines to support questionnaire design are shown as a series of steps.

What Information Is Needed?

The first step in questionnaire design is to specify the information needed. A continual review of the earlier stages of the research project, particularly the specific components of the problem, the research questions, and the hypotheses, will help keep the questionnaire focused. Questionnaires should also be designed with the target respondents in mind, taking into account their educational level and experience.

How Should Individual Questions Be Framed?

The researcher must determine what should be included in each question. This involves a determination of whether a question is necessary and whether more than one question is needed to obtain the information in an unambiguous way.

Is the Question Necessary?
Before including a question, the researcher should ask, "How will I use these data?" Questions that may be nice to know but that don’t directly address the research problem should be eliminated.

Are Several Questions Needed Instead of One?
In some cases, two questions are better than one. However, asking two questions in one is not the solution. Double-barreled questions should be split so each piece of information is clear.

Are the Respondents Able to Answer the Question?

Respondents may not always be able to answer the questions posed to them. Researchers can help them overcome this limitation by keeping in mind the reasons people typically cannot answer a question: they may not be informed or they may not remember. Where appropriate, “don't know” can be added.

Are the Respondents Willing to Answer the Question?

Even if respondents are able to answer a particular question, they may be unwilling to do so—maybe because it requires too much effort or is too sensitive. Sensitive questions are better placed at the end of the questionnaire, where rapport has already been established.

What Should Be the Structure of the Question?

A question may be unstructured (open-ended) or structured. Structured questions specify the set of responses as well as their format (multiple-choice, dichotomous, or scale).

Unstructured Questions

Unstructured questions are open-ended questions that respondents answer in their own words. They are useful in exploratory research and as opening questions, but can be difficult to record and analyze.

Structured Questions

Structured questions include multiple-choice questions and various scales. In multiple-choice questions, the researcher provides a set of possible answers. The response options should be mutually exclusive, collectively exhaustive, and carefully ordered to reduce bias.

What Type of Scales Should Be Used?

An itemized rating scale (like the Likert scale) is frequently used. The endpoints of a Likert scale are typically “strongly disagree” and “strongly agree.” Respondents indicate their degree of agreement with various statements.

Likert Scale

Named after Rensis Likert, this scale is one of the most widely used itemized scales. Respondents check one of five (or seven, or nine) response categories, typically anchored by “strongly disagree” and “strongly agree.”

Likert Scale Decisions

When constructing Likert scales, researchers must decide:

1. The number of scale categories to use.
2. Balanced vs. unbalanced scale.
3. Odd or even number of categories.
4. Forced vs. nonforced choice.

How Should the Question Be Worded?

Poor wording can confuse respondents, leading to nonresponse or response error. Guidelines include:

• Define the issue (who, what, when, where).
• Use ordinary words that match the respondent's vocabulary.
• Avoid ambiguous words like “often,” “sometimes,” etc.
• Avoid leading questions that clue the respondent to a particular answer.
• Balance dual statements by using both positive and negative statements.

What Is the Proper Order of Questions?

Questions should be arranged in a logical order, organized around topic areas. Start with interesting, simple, and nonthreatening questions. More sensitive or difficult questions appear later. Move from general to specific (funnel approach), unless respondents do not have a frame of reference, in which case an inverted funnel approach can be used.

How Should the Questionnaire Be Pretested?

Pretesting involves testing the questionnaire on a small sample of respondents (15-30) to identify and eliminate potential problems. All aspects of the questionnaire—including wording, order, layout—should be tested. Pretest respondents should be similar to the target sample.

Design of Questionnaires for Experimentation

Questionnaires are also used in experiments to measure variables of interest before and/or after exposure to a treatment. The same design principles apply.

Design of Observational Forms

Observational forms are used primarily to record respondent reactions (e.g., to new products or ads). Since no questions are asked, the researcher need not worry about psychological impacts of wording, but should still ensure accurate and efficient data recording.

Summary

To collect quantitative primary data, a researcher must design a questionnaire or an observational form. Designing a questionnaire is both an art and a science. We can provide guidelines, but no one optimal design fits every research need. The process includes specifying the information needed, deciding on individual questions (and dealing with respondents’ ability and willingness), choosing question structure, and carefully wording questions. Once drafted, the questionnaire must be pretested on a small sample to identify problems. Proper questionnaire design is critical to ensure quality data and meaningful results.