Organizing the Questions

1. Organizing the Questions Questionnaire Sections Common to (Nearly All) Surveys What questions should the questionnaire begin with? Grouping Questions into Sections (and Subsections) Questionnaire Length and Respondent Burden

2. Questionnaire Sections Common to (Nearly All) Surveys Introduction Respondent Selection Substantive Questions Background Questions Post-Interview Questions

3. “The survey introduction serves multiple purposes. It provides a compact preface to the survey, telling the participants the subject, purpose, sponsorship, and a few other details. It gives the prospective respondent sufficient information about the study to satisfy the needs of informed consent. Most of all, it elicits participation.” (Czaja & Blair [2005], p. 88)

4. Introduction Provides Information About the Study (subject, purpose, etc.) Allows for Informed Consent “Elicits Participation” (recruitment) Phrasing Needs to Be Time Efficient & Concise (i.e., for both written & oral) Want Respondents to Feel that the Survey Project is important, serious, etc. and therefore worthy of their time Relationship Between Overall Rates of Participation, Possible Response Bias, Low Survey Validity and/or Reliability (Czaja & Blair [2005], p. 88-89)

5. Respondents have these kinds of questions about surveys… “What is the study about?” “Who is conducting it?” “Who is the sponsor?” “Why is the study important?” “What will be done with the study results?” “Why is the study pertinent to me?” “Why was I selected?” This is NOT a comprehensive list; many additional questions may be asked about the project, especially if the questions are of a sensitive nature. (Czaja & Blair [2005], p. 90)

6. Substantive Questions “This is the heart of the questionnaire, accounting for the majority of the data and, hence, the majority of our effort and costs.” (Czaja & Blair [2005], p.86) Content-related questions, questions of substance, etc. Questions most directly related to our research questions & scientific hypotheses

7. Example: Agricultural Injury Survey Substantive Questions START: p. 3 http://www.cdc.gov/niosh/childag/pdfs/appendixfull.pdf

8. Background Questions Demographic questions E.g., sex, gender, sexual orientation, age, marital status, race, nationality, ethnicity, income, employment status, place of residence, etc. Reasons for Inclusion… Related to Our Research Question and/or Scientific Hypothesis Help Determine the Representativeness of Our Sample Useful for Stratification of the Data File and/or for Developing Weights, Coefficients, etc. During Data Analysis (Czaja & Blair [2005], p. 86)

9. Example: CDC Mass Casualties Event Survey Demographics; pp. 1-2 http://www.bt.cdc.gov/masscasualties/pdf/mhsurvey-instrument.pdf

10. Post-Interview Questions Intended for Respondents, Interviewers, or Both Will frequently ask about… Difficulty and/or Lack of Clarity of Items on the Questionnaire Reason/s Why Certain Items Were Not Answered (or Why the Respondent Hesitated Before Answering) Person/s to Contact (e.g., should respondents move, become very ill, or otherwise drop out of contact) (Czaja & Blair [2005], pp. 86-87)

11. Example: HINTS 2005(Health Information National Trends Survey) Debriefing Questions; p. 53 http://hints.cancer.gov/instrument.jsp

12. Importance of First Impressions…* “…we do have a lot of evidence that most refusals occur at the introduction or during the very first questions…it may be helpful to think of the respondent as making an initial tentative decision to participate (or at least to keep listening)…” (Czaja & Blair [2005], p. 94)

13. 1st Question Should Be… Relevant to the Central Topic Easy to Answer Interesting Applicable To and Answerable By Most Respondents Closed Format (Czaja & Blair [2005], pp. 94-95)

14. Avoid Asking Sensitive Questions Early If they must be asked then… “Ask as few sensitive questions as absolutely necessary to establish the respondent’s membership in the target population.” “Make clear to respondents why it is necessary to ask such questions.” “If costs permit, consider inserting an opening nonsensitive ‘buffer’ question or two before the screening item(s) to establish some rapport with the respondent.” (Czaja & Blair [2005], p. 96)

15. Sensitive Issues What strategies can we employ? Fowler (2002; pp.98-100) offers this advise: “Minimize a sense of judgement; maximize the importance of accuracy.” “Use self-administered data collection procedures.” “Confidentiality and anonymity.”

16. Question Order or Groupings Some Relevant Issues Include… Avoidance of Survey Termination Internal Logic Smooth Progression or Flow Through the Instrument Respondent Sense of the Ease & Speed of Completion (Czaja & Blair [2005], pp. 95-96)

17. Questionnaire Length and Respondent Burden Research on Specifics of Respondent Motivation is Lacking Respondents Need Some Reason/s to Participate in a Survey E.g., Financial Compensation; Interesting Study; Important Topic; Potential for Action Taken Following the Project (Dillman [2000]; Groves [1989]) Survey Length (or its perception) is Very Important (Czaja & Blair [2005], pp. 99-100)

18. Guidelines for Formatting Such That Survey Length:Difficulty Appear Low “Limit instruments to six to eight pages.” “Precode response categories by assigning a number to each possible answer for the respondent to circle.” “Space the categories so that it is easy to circle one response without touching an adjoining one; arrange the categories vertically, one under another, rather than horizontally spread across the page.” (Czaja & Blair [2005], pp. 100-101)

19. Guidelines for Formatting Such That Survey Length:Difficulty Appear Low “Provide simple instructions of no more than two sentences describing how to answer questions; for example, ‘Please circle the number of your answer unless otherwise instructed.’” “Use a different typeface for questions, response categories, and transitions or section headings.” “Whenever possible, use arrows to indicate skip instructions.” (Czaja & Blair [2005], pp. 100-101)

20. Example: NAMCS 2008(National Ambulatory Medical Care Survey) Extremely Short – More of a FORM than a Survey http://www.cdc.gov/nchs/data/ahcd/NAMCS_30_2008.pdf