SURVEY RESEARCH

1. SURVEY RESEARCH

2. Survey Research • a.k.a. “paper-pencil” measures or “self-report” measures • represents the dominant paradigm for social science research in the last 30 years • advantages: • expedience in gathering data • yields interval level data

3. Do subjects know their own minds? • problems: • problem of “mindfulness,” verbal reports of mental processes are suspect • problem of “non-attitudes,” people tend to venture an opinion whether they have one or not • unreliability of self-reports data: even factual information can be misreported

4. Will subjects tell you if they know? • intentional misreporting of information. • social desirability bias • reasons for non-disclosure • embarrassment • fear of negative sanctions • shame, public humiliation • privacy • sometimes helpful to include a “lie scale”

5. phrasing of questions is critical avoid vague, nebulous questions Bad example: “What do you think about abortion?” Bad example: “How many children do you have?” questions must be clear, unambiguous Good example: “Which of the following statements best represents your attitude toward elective abortions in the first trimester?” Conduct a “readability analysis” avoid lengthy questions; keep questions short, succinct Bad example: Sara Palin would be the first female Vice President Secretary of the United States. Do you think her gender wuold be a problem in negotiating with the Arab world, which is a male-dominated culture? Good example: Would Hillary Clinton make a good Supreme Court justice? Questionnaire Designand Construction

6. avoid double-barreled questions (compound questions) Bad example: “Do you favor stricter hand gun controls and mandatory minimum sentences for carjackers?” avoid loaded language (push polling) Bad example: “Don’t you think that...?” “Isn’t it true that...?” Bad example: Emotionally charged words: “ gang member,” “welfare mother,” “extremist groups,” “spin doctor,” etc. avoid slang, jargon, abbreviations and acronyms Bad example: “Should states regulate PETA and the ALF?” Bad example: “Do you think hip hop is wack?” avoid or minimize negative wording Bad example: Is the Bush administration right in not establishing a firm deadline for withdrawing from Iraq? questionnaire design and construction--continued

7. open-ended versus close-ended questions • Schuman, Ludwig, & Krosnick (1986): 60% of respondents selected one of four options in a close-ended format, but only 2.4% mentioned any of the same four responses in an open-ended format. • open-ended: allows subjects more leeway, flexibility • “What is your primary ethnic/cultural background?” • requires a content analysis of responses • close-ended or “forced-choice” ties respondents’ hands somewhat • easy to code the data • always include an “other_____________” category

8. phrasing of questions is critical • even slight variations in wording can alter respondents’ answers. • “occupied territory” versus “contested territory” • “assisted suicide” versus “mercy killing” • “fetus” versus “unborn child” • make questions concrete, come down the “ladder of abstraction” • Use negatively worded questions or statements sparingly • sometimes necessary to include reverse-valenced items to identify a “response set”

9. …more on phrasing questions • Avoid evaluative language • Phrasing of questions should not imply approval, disapproval • Follow-ups should not suggest surprise, liking, disliking, etc. • Bad example: Do you think the Democratic dominated Congress should lift the harsh restrictions on stem cell research? • Bad example: Despite its poor track record in crisis intervention, do you think the United Nations should intervene in Darfur, Sudan?

10. respondents’ abilities • Respondents must be capable and willing to answer the questions • May be unaware of topic/issue • May be unwilling to disclose • May lie, exaggerate, under-report or over-report • example: If you ask people what nonverbal cues they leak when they lie, people may not know. • example: if you ask people what they would do in a hostage crisis, how would they know?

11. additional considerations • order effects: sequencing of questions can be a factor • “halo effect”: carry-over from one item to the next • use multiple versions of questionnaire • “response set”: checking the same response all the way down a questionnaire • use “reverse scaling” to detect this

12. additional considerations • gender of researcher/respondent can influence responses • ethnicity of researcher/respondent can influence responses • medium can influence responses • face-to-face interview, telephone survey, mail survey • anonymous versus non-anonymous questionnaire • fatigue: minimize length of surveys

13. Characteristics of typesof surveys