1 / 18

Designing and Using a Survey

Designing and Using a Survey. February 7, 2011. Objectives. By the end of this meeting, participants should be able to: Explain how surveys accomplish the goals of: measuring attitudes, measuring change over time, making group comparisons, and analyzing the causes of behavior.

iona
Télécharger la présentation

Designing and Using a Survey

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Designing and Using a Survey February 7, 2011

  2. Objectives By the end of this meeting, participants should be able to: Explain how surveys accomplish the goals of: measuring attitudes, measuring change over time, making group comparisons, and analyzing the causes of behavior.

  3. Goals of Surveys • Measure the frequency of attitudes, beliefs and behaviors • Measure change over time • Examine differences between groups (race, class, gender, etc.) • Analyze causes of attitudes, beliefs, and behaviors

  4. Measuring Behavior • Time can be awkward for respondents • Forward telescoping • Backward telescoping • Sensitive subjects • Bogus pipeline technique (ethical concerns?) • Random lists • Randomized response technique (some sensitive, some not) • Importance of others’ beliefs

  5. Predicting Behavior • Election Predictions • Problems with third parties • Problems with really close elections • Problem with social desirability for turnout • Undecided • Sensitive topics

  6. 2008 Presidential Example- NC Final Results Obama 2,142,651 49.70% McCain 2,128,474 49.38% Others 39,726 0.92% Total 4,310,851 Source: Office of the Clerk of the US House of Representatives

  7. How did the individual polls do? FOX News/Rasmussen McCain MofE Reuters/Zogby McCain MofE American Research Group Obama MofE SurveyUSA McCain MofE Mason-Dixon McCain MofE Research 2000 Obama MofE Politico/InsiderAdvantage Tie MofE

  8. Recall: Margin of Error • So for example, FOX/Rasmussen’s final poll for North Carolina showed 50% for McCain and 49% for Obama • Sample: 1000 likely voters • Margin of error = ±t {p(1−p)/(n−1)} 1/2 {1 − f}1/2 • But since the population being sampled is large we can ignore the final {1 − f}1/2 • ±1.96 {50(50)/(1000-1 )} 1/2 • = ±3.1% • In other words, Rasmussen/FOX was 95% confident that the final vote for McCain would be between 46.9% and 53.1%. Right on the mark, pretty much. • Yet, if they had to make a prediction of the winner, they’d guess wrong. (Some polls had MO wrong, too.)

  9. Measuring Attitudes • One of the most common goals of surveys • Need to be concerned about measuring non-attitudes (Converse) • Need to consider attitude strength, respondents may not care equally about all issues (environment, gun control, etc.) • How do they feel about the issue personally? • How knowledgeable are they about this issue? • How certain are they of their opinion? • How much thought they have given to the issue? • Should you offer counterarguments?

  10. Measuring attitudes • Attitudes especially weakly held attitudes will change over time (first impression versus considered opinions) • Need to watch disagreements over the meaning of words • Example: ideology and Stimson’s work • Need to watch for changing frames • Verbal (example: welfare, next page) • Contextual (example: sexual harassment)

  11. Example of Verbal Differences- GSS

  12. Measuring Change over Time • Measurement of personal attitude change is frequently unreliable • Comparisons of cross sectional surveys are more common but pose their own issues • Similar interviewing methods? • Similar sample sizes? • Similar question wordings? • Does the change exceed sampling variation?

  13. Measuring Change over Time • Frequently can be atheoretical • Can only measure gross change, not change at the individual level • Insta-polls can allow for immediate reactions to stimuli but suffers worries about representativeness and long term impact

  14. Measuring Change over Time • Preferred method: panel studies • Repeated studies of the same individuals over time • Limitations • Long term studies are prohibitively expensive • Atrophy of the original sample (and worries about those that remain) • Errors in interviewing

  15. Sub-group Comparisons • Researchers frequently want to compare the attitudes of various sub-groups of the population • One difficulty is the smaller and less accessible groups may be harder to randomly sample • Double sample or over sampling • Pyramiding or combining multiple surveys

  16. What Leads to Change in Attitudes over Time? • Surveys generally struggle with this type of question • Most people’s answers for why their opinions changed are post-hoc rationalizations • Individuals may not be sure of their reasons for opinion change • Similar stimulus may affect people differently

  17. For February 9: Work on research proposals, you will have time for group work next meeting. • For February 11: Download the program R from http://www.r-project.org/ and bring your laptop to class. You also may want to print-out “Getting Our Feet Wet with R” from the website. • For February 14: • Research proposals due for each group. • Read WKB Chapter 8 • On p. 189 answer either question 1, 2, or 3. • Construct a causal model relevant for YOUR research question. (Turn-in the C.8 items individually.)

More Related