1 / 17

Response bias

Response bias. People respond differently to how they believe Deliberate bias “Do you agree that abortion, the murder of innocent beings, should be outlawed?” Unintentional bias “Do you or do you not use drugs?” People often want to please the interviewer

wayne-neal
Télécharger la présentation

Response bias

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Response bias • People respond differently to how they believe • Deliberate bias • “Do you agree that abortion, the murder of innocent beings, should be outlawed?” • Unintentional bias • “Do you or do you not use drugs?” • People often want to please the interviewer • Affected by sex, attire, race, behavior of interviewer • Wording, Ordering, Complexity of Questions

  2. Wording the question • Do you agree? Two questions from The New York Times (April, 1982) • (1) “A freeze in nuclear weapons should be opposed because it would do nothing to reduce the danger of thousands of nuclear weapons already in place and would leave the Soviet Union in a position of nuclear superiority.” • (2) “A freeze in nuclear weapons should be favored because it would begin a much-needed process to stop everyone in the world from building nuclear weapons now and reduce the possibility of nuclear war in the future.” • Results: 58% agreed with (1). 56% agreed with (2), and 27% agreed with both!

  3. Open versus closed questions • “What do you think is the most important problem facing the country today?” • “Which of the following do you think is the most important problem facing the country today---the energy shortage, the quality of public schools, legalized abortion, or pollution---or, if you prefer, you may name a different problem as most important.” • From “Problems in the use of survey questions to measure public opinion,” Science, Volume 236 (1987)

  4. Open versus closed questions • Results of 171 responses to open question and 178 responses to closed question

  5. Another type of response bias • “Some people say that the 1975 Public Affairs Act should be repealed. Do you agree or disagree that it should be repealed.” Washington Post, Feb. 1995 • Results: For repeal: 24%, Against repeal: 19%, No opinion: 57% • No such thing as the Public Affairs Act!

  6. Other forms of bias • Selection bias • Method of selection produces unrepresentative sample • Shoppers at a mall: “How do you feel about raising the sales tax?” • Non-response bias • Telephone survey during work hours • Voluntary response bias • “Should the drinking age be lowered to 18?”

  7. You critique it • Before 2000 election: What to do with large government surplus • (1) “Should the money be used for a tax cut, or should it be used to fund new government programs?” • (2) “Should the money be used for a tax cut, or should it be spent on programs for education, the environment, health care, crime-fighting, and military defense?” • (1): 60% for tax cut; (2): 22% for tax cut

  8. Telephone polling • SRS assumes each household has known and equal chance of selection • Not true • Multiple lines per household • Households without telephones • Number of adults per household • Some remedies • Pick from {oldest male, youngest male, oldest female, youngest female} • “Random” adult • Last birthday

  9. Weighting the sample “The sample first was weighted to take into account unequal probabilities of selection from sampling: Weighting accounts for the number of telephones going into the household, and household size. It then was weighted for age, gender, and education to take care of minor fluctuations in the sample, and align it with the findings of the 2000 Census of the adult population. It is assumed to be representative of all Minnesota households with telephones, within the margin of sampling error.” – How the Poll was Conducted, Minneapolis Star Tribune, May 2, 2003

  10. Weighting to adjust • Suppose women are twice as likely to answer the phone as men • Weight survey results by multiplying women’s responses by ½. • Responses: 150 men, 300 women • Count as: 150 men and 150 women • In practice, very complicated

  11. Other sampling schemesStratified sampling • Goal: Random sample of 240 Carleton students • To insure discipline representation divide into strata according to population • Arts and Literature 20% • Humanities 15% • Social Sciences 30% • Mathematics and Natural Sciences 35% • Choose 240 x .20 = 48 A and L students at random 240 x .15 = 36 Humanities 240 x .30 = 72 Social science 240 x .35 = 84 Math and natural

  12. Stratified sampling • Advantages: Sample will be representative for the strata; Can gain precision of estimate • Disadvantages: Logistically difficult; must know about the population; May not be possible • Note that a stratified sample is not a simple random sample • E.g., every possible group of 240 students is not equally likely to be selected

  13. Cluster sampling – an example • Warehouse contains 10,000 window frames stored on pallets • Goal: Estimate how many frames have wood rot • Sample about 500 window frames • Pallets numbered 1 to 400 • Each pallet contains 20 to 30 window frames • Cluster sample • Sample pallets, not windows. Choose, say 20. • Include in sample all the windows on each pallet

  14. Cluster sampling • Door-to-door surveys • City blocks are the clusters • Airlines get customer opinions • Individual flights are the clusters • Advantage: Much easier to implement depending on context • Disadvantage: Greater sampling variability; less statistical accuracy

  15. Multistage cluster sampling • Stage 1: Regions (West, Midwest, Northeast, South)  Demographic groupings  Towns • Stage 2: Towns  Wards • Stage 3: Wards  Precincts • Stage 4: Precincts  Households • Individuals interviewed, but no discretion (e.g., “speak to oldest woman at home 18 or older, or if no woman is at home, the youngest man 18 or older.” • Offers advantages of quota sampling without disadvantage of selection bias

  16. Modern telephone surveys • Multistage cluster design • 4 time zones, 3 areas of population density = 12 strata • Random digit dialing (RDD) SRS of telephone numbers • Telephone directory biased toward middle-class • Computer excludes businesses • Those without phones creates small bias • Interviewing done on evenings, weekends • Will call back up to 3 times if no answer • Refusal rate 20%; Cost 1/3 as much as personal interviews

  17. Do you believe the poll/survey?“Our graduates earn on average $60,000 per year” • Who carried out survey? • What is the population? • How was sample selected? • How large was the sample? • What was the response rate? • How were subjects contacted? • When was the survey conducted? • What are the exact questions asked?

More Related