Survey Research
E N D
Presentation Transcript
12 Survey Research
Survey Research • Nonexperimental method using interviews or questionnaires to assess attitudes, activities, opinions, or beliefs • Surveys often used to • assess changes in attitudes over time • test theoretical models • describe and predict behavior • To insure high external validity, random samples should be used
Steps in Conducting Survey Research • plan and design the survey research study • determine what issues you want to survey • determine whether a cross-sectional or longitudinal design will be used • identify the target population and select the sample(s) • construct and refine the survey instrument • collect the survey data • enter and “clean” the data • locate and eliminate errors where possible • analyze the survey data • interpret and report the results
Cross-Sectional Designs • Cross-sectional studies • collecting data in a single, brief time period • typically from multiple groups in survey research • examples • Whisman (2007) • “Marital Distress and DSM-IV Psychiatric Disorders in a Population-Based National Survey” • a national survey research study with a representative sample of English-speaking adults (18 years or older) in the United States • found that marital distress was associated with anxiety, mood, and substance disorders • the association between marital distress and depression was stronger when one moved from younger to older age groups • Plous (1996) • surveyed APA members to determine members’ attitudes toward the use of animals in research • the majority of respondents approved the use of animals, but wanted to eliminate or minimize the pain experienced by research animals and the number of animals euthanized
Longitudinal Designs • Longitudinal studies • collecting data from the same participants at more than one point in time • can be time consuming and expensive • in survey research, longitudinal studies can be called panel studies • type of longitudinal design in which the same individuals are surveyed multiple times over time • example • Moskowitz and Wrubel (2005) • wanted to gain a more in-depth understanding of the meaning of having contracted HIV • participants included 57 gay men, ranging in ages from 24 to 48, who tested positive for HIV • researchers conducted bimonthly interviews over the course of 2 years to identify how these individuals appraised their HIV related changes over time
Trend Studies • Independent samples are taken successively from a population over time and the same questions are asked • i.e., same survey questions are asked of different samples over time • example • General Social Survey • conducted by the National Opinion Research Center (at the University of Chicago) • each year, a different sample of U.S. citizens who are 18 years or older are asked questions about many social, psychological, and demographic variables
Survey Data Collection Method • Interview • verbal self-report data are collected from interviewees by an interviewer • types • face-to-face or personal interview • advantages • ability to clear up ambiguities and higher completion rate • disadvantage • expense and participants may be uncomfortable discussing private issues • telephone interview • less expensive than face to face and comparable data • can utilize random digit dialing for random samples
Survey Data Collection Method • Questionnaire • self-report data collection instrument filled out by research participants • Mail questionnaires • advantage • low cost • disadvantage • low return rate, typically 20-30% • Group-administered questionnaire • advantage • quick and efficient • disadvantage • cannot be used if participants are spread out across locations
Survey Data Collection Method • Electronic survey • e-mail and Web-based • advantages of electronic surveys • low cost • instant access to wide audience • data in form easy for analysis • flexible in layout – especially web-based survey • disadvantages of electronic surveys • privacy and anonymity may not be upheld • sample may not be representative of population because of volunteer sampling
Constructing and Refining a Survey Instrument • Principle 1. Write items to match the research objectives • construct items that cover the different areas and content needed to fulfill your objectives • conduct an extensive review of the literature to make sure you have identified all areas that you need to cover • write items and construct a questionnaire that will have the psychometric properties of providing reliable and valid data • content and construct validity are especially relevant
Constructing and Refining a Survey Instrument • Principle 2. Write items that are appropriate for the respondents to be surveyed • who will be completing the questionnaire? • you need to consider, empathetically, how your participants will view what you write • don’t use stilted or pretentious language • consider reading level and the demographic and cultural characteristics of your participants • write items that are understandable and meaningful to participants • use natural and familiar language
Constructing and Refining a Survey Instrument • Principle 3. Write short, simple questions • survey questionnaire items should be short, clear, and precise • use simple language and avoid jargon • write items that are unambiguous and easy to answer
Constructing and Refining a Survey Instrument • Principle 4. Avoid loaded or leading questions • loaded term • a word that produces an emotionally charged reaction • example – “liberal” • may have political connotations even when used in the looking question • “I like a liberal amount of peanut butter on my sandwich” • leading question • suggests to the respondent how they should respond • example
Constructing and Refining a Survey Instrument • Principle 5. Avoid double-barreled questions • double-barreled questions ask about two or more issues in a single question • example • “Do you agree that President Obama should focus his primary attention on the economy and foreign affairs?” • Principle 6. Avoid double negatives • double negative • a sentence construction that contains two negatives • example • Do you agree or disagree with the following statement? • psychology professors should not be allowed to conduct research during their office hours
Constructing and Refining a Survey Instrument • Principle 7. Determine whether closed-ended or open-ended questions are needed • open-ended question • a question that allows participants to respond in their own words • example • “What do you do most often when you feel depressed?” • open-ended better if researcher is unsure what respondent is thinking or variable is ill-defined • commonly used in exploratory or qualitative research • responses to open-ended questions must be coded and categorized
Constructing and Refining a Survey Instrument • Principle 7. Determine whether closed-ended or open-ended questions are needed • closed-ended question • a question where participants must select their answer from a set of predetermined response categories • closed-ended are easier to code and provide more standardized data • example
Constructing and Refining a Survey Instrument • Principle 7. Determine whether closed-ended or open-ended questions are needed • mixed-question format • a combination of both open- and closed-ended questions • example
Constructing and Refining a Survey Instrument not mutually exclusive mutually exclusive • Principle 8. Construct mutually exclusive and exhaustive categories • mutually exclusive • the categories do not overlap
Constructing and Refining a Survey Instrument • Principle 8. Construct mutually exclusive and exhaustive categories • exhaustive • categories include all possible responses
Constructing and Refining a Survey Instrument • Principle 9. Consider the different types of closed-ended response categories • rating scales • dichotomous • two choices (e.g., yes and no) • multichotomous • more than two choices (usually preferred) • ability to measure direction and strength of attitude • distance between each descriptor should be the same • anchors • descriptors placed on points on a rating scale
Examples dichotomous multichotomous
Constructing and Refining a Survey Instrument • Principle 9. Consider the different types of closed-ended response categories • binary forced choice • participant chooses one of a pair of attitudinal objects • can reduce response set • can be difficult for item analysis • typically not recommended
Binary Forced Choice Example • Example • Narcissistic Personality Inventory (NPI) (Foster & Campbell, 2007) • used to measure “normal” narcissism in personality and social psychological research
Constructing and Refining a Survey Instrument • Principle 9. Consider the different types of closed-ended response categories • rankings • participants asked to put their responses in ascending or descending order • can be open or closed ended • typically rank 3-5 objects • example
Constructing and Refining a Survey Instrument • Principle 9. Consider the different types of closed-ended response categories • checklists • participants asked to check all response categories that apply • example
Constructing and Refining a Survey Instrument • Principle 10. Use multiple items to measure complex or abstract constructs • variables like gender, weight, or ethnicity can be easy to measure • complex or abstract constructs such as self-esteem, intelligence, or locus of control can be harder • multiple items needed to measure these constructs • semantic differential • scaling method in which participants rate an object on a series of bipolar rating scales
Semantic Differential Example • “Occupation and Social Experience: Factors Influencing Attitude Towards People with Schizophrenia” (Ishige & Hayashi, 2005) • measured the participants’ attitudes using 20 bipolar adjectives • adjective pairs used • safe vs. harmful, bad vs. good, fierce vs. gentle, shallow vs. deep, active vs. inert, lonely vs. jolly, simple vs. complicated, dirty vs. clean, distant vs. near
Constructing and Refining a Survey Instrument • Principle 10. Use multiple items to measure complex or abstract constructs • Likert scaling • a multi-item scale is used to measure a single construct by summing each participant’s responses to the items on the scale • questions can be positively or negatively worded • statistically analyzed using coefficient alpha
Likert Scale Example • Five items each positively and negatively worded
Constructing and Refining a Survey Instrument • Principle 11. Make sure the questionnaire is easy to use from beginning to end • ordering of questions • if using positive and negative questions, ask positive questions first • ask interesting questions first to capture participants’ attention • demographic questions last • questionnaire length • questionnaire optimal length unknown • mail questionnaires should be short • telephone interviews should be less than 15 minutes • face-to-face interviews can be longer
Constructing and Refining a Survey Instrument • Principle 11. Make sure the questionnaire is easy to use from beginning to end • contingency questions • an item directing the participant to different follow-up questions depending on the initial response • to many can be confusing for participant • example
Constructing and Refining a Survey Instrument • Principle 11. Make sure the questionnaire is easy to use from beginning to end • response bias • social desirability bias • occurs when participants respond in a way to make themselves look good • can minimize by insuring anonymity • if using binary forced choice questions, make each choice equally desirable • response set • tendency to respond in a specific way • participant may not want to pick extremes and always choose middle choice • solution: use even number of response categories on rating scale • including multiple question types helps to reduce response set, but can also reduce reliability
Constructing and Refining a Survey Instrument • Principle 12. Pilot test the questionnaire until it is perfected • to identify and fix problems • to practice protocols • clear up ambiguity • use think aloud technique
Selecting Your Survey Sample From the Population • If primary goal is to explore relationship between variables rather than generalization, convenience sample is acceptable • If generalization to population is needed, a random sampling method should be used
Preparing and Analyzing Survey Data • Check for errors • examples • participants answers 7 on a Likert scales that is 1-4 • participant does not answer a question, data missing • Analyze quantitative data with statistical analysis
