Bias in Survey

1. Bias in Survey

2. What is Bias In survey sampling, bias refers to the tendency of a sample statistic to systematically over- or under-estimate a population parameter.

3. Type Of Bias Bias Due to Unrepresentative Samples Bias Due to Measurement Error Sampling Error and Survey Bias

4. Type of Bias in Survey Bias Due to Unrepresentative Samples A good sample is representative. This means that each sample point represents the attributes of a known number of population elements. The bias that results from an unrepresentative sample is called selection bias.

5. Bias Due to Unrepresentative Samples Undercoverage. Undercoverage occurs when some members of the population are inadequately represented in the sample. A classic example of undercoverage is the Literary Digest voter survey, which predicted that Alfred Landon would beat Franklin Roosevelt in the 1936 presidential election. The survey sample suffered from undercoverage of low-income voters, who tended to be Democrats.

6. Bias Due to Unrepresentative Samples Nonresponse bias. Sometimes, individuals chosen for the sample are unwilling or unable to participate in the survey. Nonresponse bias is the bias that results when respondents differ in meaningful ways from nonrespondents.

7. Example The Literary Digest survey illustrates this problem. Respondents tended to be Landon supporters; and nonrespondents, Roosevelt supporters. Since only 25% of the sampled voters actually completed the mail-in survey, survey results overestimated voter support for Alfred Landon.

8. Bias Due to Unrepresentative Samples Voluntary response bias. Voluntary response bias occurs when sample members are self-selected volunteers, as in voluntary samples..

9. Example An example would be call-in radio shows that solicit audience participation in surveys on controversial topics (abortion, affirmative action, gun control, etc.). The resulting sample tends to overrepresent individuals who have strong opinions

10. Bias Due to Unrepresentative Samples Random sampling is a procedure for sampling from a population in which (a) the selection of a sample unit is based on chance and (b) every element of the population has a known, non-zero probability of being selected. Random sampling helps produce representative samples by eliminating voluntary response bias and guarding against undercoverage bias. All probability sampling methods rely on random sampling.

11. Type of Bias Bias Due to Measurement Error A poor measurement process can also lead to bias. In survey research, the measurement process includes the environment in which the survey is conducted, the way that questions are asked, and the state of the survey respondent.

12. Bias Due to Measurement Error Response bias refers to the bias that results from problems in the measurement process

13. Bias Due to Measurement Error Leading questions. The wording of the question may be loaded in some way to unduly favor one response over another..

14. Example For example, a satisfaction survey may ask the respondent to indicate where she is satisfied, dissatisfied, or very dissatified. By giving the respondent one response option to express satisfaction and two response options to express dissatisfaction, this survey question is biased toward getting a dissatisfied response

15. Bias Due to Measurement Error Social desirability. Most people like to present themselves in a favorable light, so they will be reluctant to admit to unsavory attitudes or illegal activities in a survey, particularly if survey results are not confidential. Instead, their responses may be biased toward what they believe is socially desirable.

16. Type OF Bias Sampling Error and Survey Bias A survey produces a sample statistic, which is used to estimate a population parameter. If you repeated a survey many times, using different samples each time, you might get a different sample statistic with each replication. And each of the different sample statistics would be an estimate for the same population parameter. .

17. Sampling Error and Survey Bias If the statistic is unbiased, the average of all the statistics from all possible samples will equal the true population parameter; even though any individual statistic may differ from the population parameter. The variability among statistics from different samples is called sampling error.

18. Sampling Error and Survey Bias Increasing the sample size tends to reduce the sampling error; that is, it makes the sample statistic less variable. However, increasing sample size does not affect survey bias.

19. Sampling Error and Survey Bias A large sample size cannot correct for the methodological problems (undercoverage, nonresponse bias, etc.) that produce survey bias. The Literary Digest example discussed above illustrates this point. The sample size was very large - over 2 million surveys were completed; but the large sample size could not overcome problems with the sample - undercoverage and nonresponse bias

20. Type of Bias Interviewer bias: Error that results from conscious or unconscious bias in the interviewer’s interaction with respondent The bias is created when respondents give untrue or inaccurate answer due to the interviewer’s dress, age, sex, facial expression, body language, or tone of voice

21. Type of Bias Measurement Instrument questionnaire Bias Error that results from the design of the questionnaire or measurement instruments

22. Question as Measure:

23. What is a Good Question? In different fields, and increasingly in medical science, important measurements are based on a question-and-answer process

24. Critical Standards for a good Question and Answer Process • Produces answer that provide meaningful information about what we are trying to describe • Purpose of measurement usually is to produce comparable information about many people or event (it is important that the measurement process, when applied repeatedly, produce consistent results.

25. What Is A Good Question • A good question is the one that produces answers that are reliable and valid measure of something we want to describe.

26. Validity and Reliability The principles of validity and reliability are fundamental cornerstones of the scientific method.

27. Validity and Reliability • What is Reliability? • The idea behind reliability is that any significant results must be more than a one-off finding and be inherently repeatable. • Other researchers must be able to perform exactly the same experimentor Survey, under the same conditions and generate the same results. • Reliabilityis the degree to which an assessment tool produces stable and consistent results.

28. Validity and Reliability • What is Validity? • Validity encompasses the entire experimental concept and establishes whether the results obtained meet all of the requirements of the scientific research method. • Validityrefers to how well a test measures what it is purported to measure.

29. Validity and Reliability • Reliability is a necessary ingredient for determining the overall validity of a scientific survey and enhancing the strength of the results.

30. Characteristics Of Questions And Answers That Affect Measurement • Question need to be consistently understand All people answering should understand it in a consistent way and in the same way that is consistent with what the researcher expected it to mean

31. Characteristics Of Questions And Answers That Affect Measurement • Question need to be consistently administrated or communicated to respondents If Questions are presented to people in written form , all respondents should be able to read the questions

32. Characteristics Of Questions And Answers That Affect Measurement • What constitutes an adequate answer should be consistently communicated Example: When did you move to Irbid City? Possible answers: In 1999 When I was 10 After I left college Better Question is In what year did you move to Irbid?

33. Characteristics Of Questions And Answers That Affect Measurement • Unless measuring knowledge is the goal of the question, all respondents should have access to the information needed to answer the question accurately

34. Characteristics Of Questions And Answers That Affect Measurement • Respondents must be willing to provide the answer called for in the question. We should ask questions to which respondents are willing to give correct and valid answers

35. Question Evaluation • Good science entails attempting to minimize error and taking steps to measure the remaining error so that we know how good our data are and we can continue to improve our methods

36. Question Evaluation • Evaluate how well questions we propose to ask meet the five standard • Assessing the validity of answers that results

37. Evaluate how well questions we propose to ask meet the five standard • Focus group discussion • Cognitive interviewers, in which people’s comprehension of questions and how they go about answering questions is probed and evaluated • Field pretest under realistic conditions

38. Assessing the validity of answers that results • Analysis of resulting data to evaluate the strength of predictable relationships among answers and with other characteristics of respondents • Comparisons of data from alternatively worded questions asked of comparable sample

39. Assessing the validity of answers that results • Comparison of answers against records • Measuring the consistency of answers of the same respondents at two points in time

40. Questions