1 / 15

Survey Design: Wording and Question Order Issues

Explore the two main issues in survey design - wording and question order - and their impact on respondent understanding and response. Learn how to minimize confusion and bias in questionnaire design.

arnoldot
Télécharger la présentation

Survey Design: Wording and Question Order Issues

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. STAT 472 Survey Design Wording Issues Questionnaire design issues

  2. Wording Issues • There are two wording issues: • Discussed earlier, is to use simple vocabulary and grammar to minimize confusion • The effects of specific words or phrases on the respondent Example: what is the difference between forbid and not allow? Both terms have the same meaning, but many people are willing to not allow something than to forbid(emotional reaction) it. Less well educated respondents are most influenced by minor wording differences.

  3. Example: Respondents were asked whether they thought television news was impartial. large numbers of respondents had ignored the word impartial. Less than 50% interpreted the word with its proper meaning Over 25% ignored it or had no idea of its meaning Others gave it unusual meanings 10% thought it was directly opposite to its true meaning Researchers need to be cautious, because some wording effects (e.g., the difference between forbid and not allow) remain the same for decades, while other effects may appear.

  4. Questionnaire Design Issues • Length of Survey • Question Order • Nonresponse • Format and Layout • Total Design Method

  5. Length of Survey or Questionnaire • Researchers prefer long questionnaires or interviews because they are more cost effective • There is no absolute proper length. The length depends on the survey format and on the respondents characteristics • In telephone interview: 10-20 minutes • In mail questionnaire: 3 to 4 pages (for highly educated and a salient topic: 15 pages) • In face-to-face interview: 1 hour (in special situations, 3-5 hours)

  6. Question order or Sequences • There are two question sequence issues • How to organize the items in the overall questionnaire • Context effects of answering specific questions before others.

  7. Organization of Questionnaire You should sequence questions to minimize the discomfort and confusion of respondents. A questionnaire has opening, middle, and ending questions. After an introduction explaining the survey, make opening questions pleasant, interesting, and easy to answer to help a respondent feel comfortable about the questionnaire. Avoid asking many boring background questions or threatening questions first Organize questions into common topics. Mixing questions on different topics causes confusion. Orient respondents by placing questions (e.g., "Now I would like to ask you questions about housing"). Make question topics flow smoothly and logically, and organize them to assist respondents' memory or comfort levels. Do not end with highly threatening questions, and always end with a "thank you”.

  8. Order Effects It may influence respondent answers, it’s appear for people who lack strong views, less educated, and for older or those with memory loss. Example: For a woman having an abortion rises if the question is preceded by a question about abortion being acceptable when a fetus has serious defects, but not when the question is by itself or before a question about fetus defects. Previous questions can influence later ones in two ways: • through their content (i.e., the issue) • through the respondent's response

  9. For example, a student respondent is asked, "Do you support or favor an educational contribution for students?" Answers vary depending on the topic of the preceding question. If it comes after, "How much tuition does the average U.S. student pay?” respondents interpret “contribution“ to mean support for what students will pay. If it comes after "How much does the Swedish government pay to students?” respondents interpret it to mean a contribution that the government will pay.

  10. Responses can be also influenced by previous answers, because a respondent having already answered one part will assume no overlap. For example, a respondent is asked, "How is your wife?" The next question is, "How is your family?" Most respondents will assume that the second question means family members other than the wife because they already gave an answer about the wife.

  11. Question Order Effects

  12. Use a funnelsequence of questions - that is, ask more general questions before specific ones (e.g., ask about health in general before asking about specific diseases) • Or, divide the number of respondents in half and give half of the questions in one order and the other half in the alternative order. then examine the results to see whether question order mattered. Context Effects. Researchers found powerfulcontext effects in surveys. As a practical matter,two things can be done regarding context effects If question order effects are found, which order tells you what the respondents really think? The answer is that you cannot know for sure.

  13. Example: A telephone survey on two topics were studied: Crime and attitudes toward a new anti-drunk-driving law A random half of the respondents heard questions aboutthe drunk-driving law first; the other half heard about crime first (to see whether there was any context effect-a difference by topic order). They found that respondents who were asked about the drunk-driving law first expressed lessfearabout crime than did those who were asked about crimefirst. Respondents answer all questions based on a context of preceding questions and the interview setting. A researcher needs to remember that the more ambiguous question's meaning, the stronger the context effects, because respondents will draw on the context to interpret and understand the question, previous questions on the same topic and heard just before a question can have a large context effect.

  14. Order effects • Questionnaire : • Opening  middle  closeting questions -Start with Introduction explaining the survey -opening questing an easy question to be answered. - Avoid asking boring or threatening questions

  15. Mixing questions on different topics causes confusion • Arrange questions on the same topic together, and introduce the section with a short introductory. • Don’t end with highly threatening questions, and always end with a “Thank you” • Use Funnel sequence of questions: Ask more general questions before more specific ones,

More Related