Sources of error: Information bias

1. Principles of Epidemiology for Public Health (EPID600) Sources of error: Information bias Victor J. Schoenbach,PhD home page Department of EpidemiologyGillings School of Global Public HealthUniversity of North Carolina at Chapel Hill www.unc.edu/epid600/ Sources of error: Information bias

2. Abort, Retry, Fail “Tips for safer drives: Never turn off a PC or accessories while the computer is on or the disk is active.” — USA Today [PC Magazine, 10/3/1996] Sources of error: Information bias

3. Chapter 1 THE HISTORIAN'S TASK:Insight into the future History, a record of things left behind by past generations, started in 1815. Thus we should try to view historical times as the behind of the present. Anders Henriiksson (ed), Non Campus Mentis, NY, Workman Publishing Co., 2003

4. Non Campus Mentis “History, as we know, is always bias, because human beings have to be studied by other human beings, not by independent observers of another species.” Anders Henriiksson (ed), Non Campus Mentis, NY, Workman Publishing Co., 2003, chapter 1

5. Information bias Information bias: a systematic distortion or error that arises from the procedures used for classification or measurement of the disease, the exposure, or other relevant variables. Sources of error: Information bias

6. Information bias • Classification or measurement • Differential or nondifferential bias • Direction of bias • Misclassification of covariables Sources of error: Information bias

7. Classification or measurement • Data for epidemiologic studies consist of classifications (e.g., “hypertensive” vs. “normotensive”) or measurements (e.g., 120 mmHg systolic BP). • Possible sources of measurement or classification error include instrumentation, laboratories, records, respondents; data collectors, managers, analysts, and interpreters. Sources of error: Information bias

8. Sources of measurement error Respondent (interview, questionnaire): • inability to understand, recall, articulate; • unwillingness to disclose • social desirability influences Can be influenced by wording of questions and how they are asked. Sources of error: Information bias

9. Example of misunderstanding • Medico – Não consigo encontrar o motivo das suas dores, meu caro. Só pode ser por causa da bebida. • Paciente – Não tem importãncia, doutor. Eu volto outro dia que o senhor estiver sóbrio. De Luciana V. Paiva, Osasco - SP, en Bom Humor Nosso E Dos Leitores”, Almanaque Brasil de Cultura Popular. Maio 2001;3(26) (almanaquebrasil@uol.com.br). Exemplar de quem viaja TAM.

10. How not to ask questions “Has anyone ever tried to give you the mistaken idea that sex intercourse is necessary for the health of the young man? (from a survey by the NC state health officer, circa 1926, summarized in Kinsey et al., 1948) Can you guess the right answer? Sources of error: Information bias

11. Respondent cognitive processes • Respondent cognitive processes: interpretation, recall, judgment formation, response formatting, editing • Qualitative research on response processes, e.g.: • “What types of physical activity or exercise did you perform during the past month?” • “What did you think we meant when we said ‘physical activity’?” • “Which, if any, of the following would you (also) consider to be physical activity? www.minority.unc.edu/institute/2000/materials/slides/RichardWarnecke-2000-06-08.ppt Sources of error: Information bias

12. Cognitive testing - 2 Recalling and retrieving – Retrieval probes: • Recall strategy • Recall interval • Search strategy (proximal, distal, anywhere) • Long term recall - link to events to help remember • Recall frame of reference--what kinds of things helped you remember? www.minority.unc.edu/institute/2000/materials/slides/RichardWarnecke-2000-06-08.ppt Sources of error: Information bias

13. Surveys and Questionnaires Survey validation Pretesting (wording, item sequence, time) Pilot testing (all steps - procedure, item performance) Translation validation Sources of error: Information bias Sources of error: Information bias 13

14. Sources of measurement error • Data collector: unclear or ambiguous questions, lack of a neutral demeanor, insufficiently conscientious, inaccurate transcription, fraud Sources of error: Information bias

15. Sources of measurement error • Data managers: inaccurate transcription, mis-reading, miscoding, programming errors • Data analysts: variable coding and programming errors • Data interpreters: inadequate appreciation of the characteristics of the measure or of the relations being studied Sources of error: Information bias

16. Techniques for avoiding data collection errors • Precise operational definitions of variables • Detailed measurement protocols • Repeated measurements on key variables • Training, certification, and re-certification • Data audits (of interviewers, of data centers) • Data cleaning – visual, computer • Re-running all analyses prior to publication Sources of error: Information bias

17. Validation and agreement Sensitivity and specificity – used to evaluate classifications When no validation standard, we measure agreement Measures of agreement often correct for “chance” 11/5/2009 Sources of error: Information bias 17

18. Information bias – differential or non-differential • Important question for any kind of bias – are error processes different for groups being compared • If no, “non-differential” • If yes, “differential” • Has implications for direction of bias • In general, non-differential is safer Sources of error: Information bias

19. Direction of bias • “Upward” • “Downward” • “Towards the null” • “Away from the null” Null = 0 (for differences) 1 (for ratios) Sources of error: Information bias

20. Direction of bias In simple situation, information bias is towards the null IF: 1. Dichotomous exposure and disease 2. Non-differential misclassification with both sensitivity and specificity each greater than 0.5; AND 3. Errors in one variable are independent of errors in the other Sources of error: Information bias

21. Errors in covariables • It is almost always important to control for other variables (e.g., age) • Errors in measurement of these variables hamper attempts to control for them • Direction of bias is generally unpredictable Sources of error: Information bias

22. Classic studies – 6 degrees of separation? • Classic experiment by Yale psychologist Stanley Milgram asked people in Kansas to forward a letter to a target person in Massachusetts • If did not know target person, then send it to someone they thought might know him • Milgram’s 1967 paper reported that it only took 5 jumps, on average, for letters to arrive Sources of error: Information bias

23. Selection bias in a classic study According to Judith Kleinfeld, psychologist at the University of Alaska, Fairbanks, archives reveal that only 30% of the letters actually reached their destination! (Gewolb, Josh. Random samples. Science 26 October 2001;294:777. See Kleinfeld, Judith S. Society. Jan/Feb2002; 39(2):61-66) Sources of error: Information bias

24. Dr. Kinsey and the Institute for Sex Research Alfred S. Kinsey (photograph from Wardell B. Pomeroy, Dr. Kinsey and the Institute for Sex Research)

25. Sex research in the mid-20th century Alfred S. Kinsey (photograph from Wardell B. Pomeroy, Dr. Kinsey and the Institute for Sex Research)

26. Kinsey et al. on selection and information bias (Alfred C. Kinsey, Wardell B. Pomeroy, Clyde E. Martin. Sexual behavior in the human male, Phila, W.B. Saunders, 1948) Alfred S. Kinsey (photograph from Wardell B. Pomeroy, Dr. Kinsey and the Institute for Sex Research)

27. Sources of error: Information bias

28. Real-life example: Quit for Life • Randomized trial of smoking cessation interventions • Self-reported “In the past 7 days, have you smoked a cigarette, even a puff?” • Attempted (unsuccessfully) to validate with saliva cotinine • People who did not give a time to be called for validation had very high quit rates! Sources of error: Information bias

29. Kinsey biography by Wardell Pomeroy Pomeroy, Wardell B. Dr. Kinsey and the Institute for Sex Research NY, Signet / New American Library, 1972: p136

30. Non Campus Mentis “Hindsight, after all, is caused by a lack of foresight.” Anders Henriiksson (ed), Non Campus Mentis, NY, Workman Publishing Co., 2003, chapter 1