1 / 32

Introduction to Questionnaire Design

Introduction to Questionnaire Design. Spring 2005 Seminar Series Survey Research Laboratory University of Illinois at Chicago www.srl.uic.edu. Questionnaire design encompasses :. How to write questions How to draft and organize a questionnaire. The Art of Asking Questions.

Albert_Lan
Télécharger la présentation

Introduction to Questionnaire Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Introduction to Questionnaire Design Spring 2005 Seminar Series Survey Research Laboratory University of Illinois at Chicago www.srl.uic.edu

  2. Questionnaire design encompasses: How to write questions How to draft and organize a questionnaire

  3. The Art of Asking Questions You must ask the right question Respondents must understand your question Respondents must know the answer Respondents must be willing and able to tell you the answer

  4. Three Simple Rules for Writing a Good Questionnaire • Think through your research questions and objectives before you write questions • Prepare an analysis plan before you write questions • Ask yourself, in relation to points #1 and #2 above, if each question on your list is necessary? Even if the data would be ‘interesting’ it has to ultimately be used in analysis to make the cut!

  5. Types of Survey Questions 1) Those that ask about behaviors or facts • Non-threatening behavior questions • Threatening behavior questions • Demographics 2) Those that ask about psychological states or attitudes 3) Those that ask about knowledge

  6. What Is A Good Question? One that yields a truthful, accurate answer One that asks for one answer on one dimension One that accommodates all possible contingencies of response One that uses specific, simple language One that has mutually exclusive response options One that produces variability in response One that minimizes social desirability One that is pretested

  7. Bad question: In the past 6 months, what major appliances has your household purchased new from the store? Better question: Now I’m going to read a list of household appliances. As I read each one, please tell me whether or not your household has purchased this type of appliance new from the store during the past 6 months. How about… a refrigerator? a kitchen range or oven? a microwave? Ask questions one at a time

  8. Bad Question: Compared to one year ago, are you paying more, less, or about the same for your auto and life insurance? Better Question: Compared to one year ago, are you now paying more, less, or about the same for … a. auto insurance? b. life insurance? Ask questions one at a time

  9. Specify Specify who, what, when, where and how. For example, whose income? What’s included? Over what period of time? Example: “In 2002, what was your total household income, before taxes? Please count income from all members of your household, including wages from employment, disability, social security, and public aid.”

  10. Specify through cues Example People drink beer in many places – for example, at home, at restaurants, at bars, sporting events, at friends’ homes, etc. During the past 30 days, did you drink any beer?

  11. Ambiguous: How would you compare how close you are to family in your current neighborhood to how close you were in your old neighborhood? Would you say your family is closer here, further here, or the same? More Clear: Compared to your last neighborhood, do you now live closer to your family, are you further from your family, or are you about the same distance? Use words with singular meanings

  12. Social Desirability Respondents will try to represent themselves to the interviewer in a way that reflects positively on them As questions become more threatening, respondents are more likely to overstate or understate behavior, even when the best question wording is used

  13. Minimizing Social Desirability For socially desirable behavior, it is better to ask whether respondents have ever engaged in the behavior before asking whether they currently engage in the behavior For socially undesirable behavior, it is better to ask about current behavior first, rather than ask about their usual or typical behavior Train interviewers to maintain a professional attitude Self-administered computer-assisted procedures can reduce question threat and improve reporting on sensitive questions Longer questions reduce sensitivity when obtaining information on frequencies of socially undesirable behavior

  14. Open vs. Closed Questions General rule: closed questions are usually better • Easier for the respondent • Less coding later • Better to have respondent do categorizing • Categories help define the question

  15. Disadvantages of Closed Questions • Categories may be leading to respondents • May make it too easy to answer without thinking • May limit spontaneity • Not best when • asking for frequency of sensitive behaviors • there are numerous possible responses

  16. General Principles for Response Options • Response categories should be consistent with the question • Categories must be exhaustive, including every possible answer • Categories must be mutually exclusive (no overlap) • If appropriate, include a “don’t know” category

  17. Constructing Response Scales • Respondents can generally only remember a maximum of 5 responses unless visual cues are used • Using graphic images such as thermometers and ladders and using card sorting for compex ratings is effective • Number of points in scale should be determined by how you intend to use the data • With scales with few points, every scale can be labeled; in longer scales, only the endpoints are labeled

  18. Ordering Response Categories Usually better to list responses from the lower level to the higher level Associate greater response levels w/ greater numbers Start with end of a scale that is least socially desirable

  19. Should you use a middle category? • Common practice is to omit it to push respondents (Rs) toward one end or the other, on the theory that few individuals are truly in the middle on a particular issue • Evidence from empirical studies shows that use of an explicit middle alternative will often be taken by Rs in a forced choice situation if offered; at the same time, it does not affect the ratio of ‘pro’ to ‘con’ responses or the size of the don’t know category • Our usual recommendation is to include it unless there are persuasive reasons to exclude

  20. Part 2:How to draft and organize your questionnaire

  21. Ordering the Questions • Segment by topic • Ask about related topics together • Salient questions take precedence over less salient ones • Ask recall backwards in time • Use transitions when changing topics – give a sense of progress through the questionnaire • Leave objectionable questions (e.g., income) for the end • Put demographic questions at the end

  22. How to Start the Questionnaire • Start with easy questions that all respondents can answer with little effort • First questions should also be non-threatening • Don’t start with knowledge or awareness questions • First questions should be directly related to the topic as described in the introduction or advance/cover letter

  23. Survey Intro/Cover Letter Introduction should indicate: • who is conducting the survey • the topics to be covered in the survey • an assurance of confidentiality • any IRB stipulations • whether you offer how long it will take depends on mode, topic, population

  24. Physical Format of the Self-Administered Questionnaire • Careful formatting is necessary to decrease errors and increase motivation • Respondent’s needs must always take priority, followed by interviewer and data processors

  25. Physical Format Checklist • Number all questions sequentially • Use large, clear type; don’t crowd • ‘White space:’ Place more blank space between questions than between subcomponents of questions • List answer categories vertically instead of horizontally • Avoid double/triple ‘banking’ of response choices • Be consistent with direction of response categories • Be consistent with placement of response categories

  26. Physical Format, Continued • Don’t split questions across pages. If necessary (e.g., question requires 1.5 pages), restate question and response categories on next page • Put special instructions on questionnaire as needed, next to question • Distinguish directions from questions • Precode the questionnaire (vs. check boxes)

  27. Mail questionnaires • Include a cover letter and contact information if the respondent needs help • Use a booklet format • Easier to turn pages • Prevents lost pages • Permits double-page formats • Looks more professional • Include a title, graphic, name/address of sponsor on cover

  28. Testing the Questionnaire • Preferable to test the questionnaires with people like those in your main study study population • Test in same mode to be used for main study • Consider cognitive pretesting

  29. Questionnaire Design Steps 1.Decide what information is needed 2. Search for existing questions 3. Focus groups 4. Draft new questions/revise existing ones 5. Sequence the questions 6. Get peer evaluation 7. Revise and test on self/co-workers

  30. Questionnaire Design Steps 8. Think-aloud interviews 9. Revise/eliminate questions 10. Prepare interviewer instructions for pilot test 11. Pilot test (10-20 cases) 12. Revise eliminate questions based on respondent & interviewer comments

  31. Questionnaire Design Steps 13. Pilot test again, if necessary 14. Prepare final interviewer instructions 15. Be prepared to modify questionnaires if interviewer training raises problems 16. After interviewing is complete, debrief interviewers for potential problems 17. Use experience from one study for future planning

  32. Key References To Get Started Dillman, Don. Mail and Internet Surveys: The Tailored Design Method. New York: John, Wiley & Sons, Inc, 2000. Bradburn, N, Sudman, S. and Wansink, B.. Asking Questions: The Definitive Guide to Questionnaire design – for Market Research, Political Polls, and Social and Health Questionnaires. San Francisco: Jossey Bass, 2004. See also past issues of Public Opinion Quarterly

More Related