Secondary Data Analysis:

1. Secondary Data Analysis: Opportunities and Pitfalls

2. Who am I and why am I here? • Laurel A. Copeland, BS MPH PhD • – VA Health Services Research & Development investigator; UTHSCSA Department of Psychiatry Assistant Professor - Research • copelandl@uthscsa.edu • http://czresearch.com/dropbox

3. What Can Secondary Data Analysis Do for You? • Provide preliminary data for grant proposals • Facilitate publication while you apply for grants • Explore new areas

4. Shope, Jean T; Copeland, Laurel A; Maharg, Ruth; Dielman, TE; Butchart, Amy T (1993). Health Education Quarterly 20(3): 373-390. Shope, Jean T; Copeland, Laurel A; Dielman, TE (1994). Alcoholism: Clinical and Experimental Research 18(3): 726-733. Shope, Jean T; Copeland, Laurel A; Marcoux, Beth C; Kamp, Mary E (1996). Journal of Drug Education, 26(4): 323-337. Shope, Jean T; Copeland, Laurel A; Maharg, Ruth; Dielman, TE (1996). Alcoholism: Clinical & Experimental Research, 20(5): 791-798. Copeland, Laurel A; Shope, Jean T; Waller, Patricia F (1996). Journal of School Health, 66(7): 254-260. Barry, Kristen Lawton; Fleming, Michael F; Manwell, Linda; Copeland, Laurel A (1997). Journal of Family Practice, 45(2): 151-158. Zimmerman, Marc A; Copeland, Laurel A; Shope, Jean T; Dielman, TE (1997). Journal of Youth and Adolescence, 26(2): 117-141. Barry, Kristen Lawton; Fleming, Michael F; Manwell, Linda; Copeland, Laurel A; Appel, Scott (1998). Family Medicine, 30(5):366-371. Blow, Frederic C; Barry, Kristen Lawton; BootsMiller, Bonnie J; Copeland, Laurel A; McCormick, Richard; Visnic, Stephanie (1998). Journal of Psychiatric Research, 32: 311-319. Valenstein, Marcia; Barry, Kristen Lawton; Blow, Frederic C; Copeland, Laurel A; Ullman, Esther (1998). Psychiatric Services, 49(8): 1043-1048. Shope, Jean T; Copeland, Laurel A; Kamp, Mary E; Lang, Sylvia W (1998). Journal of Drug Education, 28(3): 185-197. Kales, Helen C; Blow, Frederic C; Copeland, Laurel A; Bingham, Raymond C; Kammerer, Ericka E; Mellow, Alan F (1999). American Journal of Psychiatry, 156(4): 550-556. Blow, Frederic C; Barry, Kristen Lawton; Copeland, Laurel A; McCormick, Richard; Lehmann, Laurent; Ullman, Esther (1999). Psychiatric Services, 50(3): 390-394. Maio, Ronald F; Shope, Jean T; Blow, Frederic C; Copeland, Laurel A; Gregor, MaryAnn; Brockmann, Laurie M; Weber, Janet E; Metrou, Mary E (2000). Annals of Emergency Medicine, 35(3):252-257. Kales, Helen C; Blow, Frederic C; Bingham, Raymond C; Copeland, Laurel A; Mellow, Alan M (2000). Psychiatric Services, 51(6): 795-800. Roberts, J Scott; Blow, Frederic C; Copeland, Laurel A; Barry, Kristen Lawton; Van Stone, William (2000). Journal of Geriatric Psychiatry and Neurology, 13 Summer: 78-86. Blow, Frederic C; Walton, Maureen A; Barry, Kristen Lawton; Coyne, James C; Mudd, Sharon A; Copeland, Laurel A (2000). Journal of the American Geriatrics Society, 48(7): 769-774. Kales, Helen C; Blow, Frederic C; Bingham, Raymond C; Roberts, J Scott; Copeland, Laurel A; Mellow, Alan M (2000). American Journal of Geriatric Psychiatry, 8:301-309. Blow, Frederic C; Ullman, Esther; Barry, Kristen Lawton; Bingham, C Raymond; Copeland, Laurel A; McCormick, Richard; Van Stone, William (2000). American Journal of Orthopsychiatry, 70(3): 389-400. Valenstein, Marcia; Copeland, Laurel A; Owen, Richard; Blow, Frederic C; Visnic, Stephanie (2001). Journal of Clinical Psychiatry 62(7): 545-551. Valenstein, Marcia; Copeland, Laurel A; Owen, Richard; Blow, Frederic C; Visnic, Stephanie (2001). Psychiatric Services, 52(9): 1242-1244. Duffy, Sonia A; Terrell, Jeff; Valenstein, Marcia; Ronis, David; Copeland, Laurel A; Connors, Mary (2002). General Hospital Psychiatry, 24: 140-147. Valenstein, Marcia; Copeland, Laurel A; Blow, Frederic C; McCarthy, John F; Zeber, John E; Gillon, Leah; Bingham, C Raymond; Stavenger, Thomas (2002). Medical Care 40(8): 630-639. Hegedus, Andrea M; Copeland, Laurel A; Barry, Kristen Lawton; Blow, Frederic C. (2002). American Journal of Orthopsychiatry 72(3): 331-340. Copeland, Laurel A; Blow, Frederic C; Barry, Kristen Lawton (2003). Health Education & Behavior 30(3):305-321. Copeland, Laurel A; Zeber, John E; Valenstein, Marcia; Blow, Frederic C (2003). American Journal of Psychiatry, 160:1817-1822. Valenstein, Marcia; Blow, Frederic C; Copeland, Laurel A; McCarthy, John F; Zeber, John E; Gillon, Leah; Bingham, C Raymond; Stavenger, Thomas (2004). Schizophrenia Bulletin 30(2): 255-64. Janz, Nancy K.; Wren, Patricia A.; Copeland, Laurel A.; Lowery, Julie C.; Goldfarb, Sherry L.; Wilkins, Edwin G. (2004). Journal of Clinical Oncology 22(15):3091-3098. Lambert, Michael T.; Terrell, Jeffrey E.; Copeland, Laurel A.; Ronis, David L.; Duffy, Sonia A. (2005). Nicotine & Tobacco Research 7(2):233-241. Copeland, Laurel A; Fletcher, Carol E; Patterson, Judith (2005). Military Medicine 170(7):602-606. Copeland, Laurel A; Elshaikh, Mohamed A; Jackson, James; Penner, Lou A; Underwood III, Willie (2005). Cancer 104(7):1372-1380. O'Neill, Jessica L; Flanagan, Petra S; Zaleon, Carolyn R; Copeland, Laurel A (2005). Pharmacotherapy 25(11):1560-1565. Copeland, Laurel A; Zeber, John E; Rosenheck, Robert A; Miller, Alexander L (in press). Medical Care. Pugh, Mary Jo V; Copeland, Laurel A; Zeber, John E; Cramer, Joyce A; Amuan, Megan T; Cavazos, Jose E; Kazis, Lewis E (in press). Epilepsia. Silveira, Maria J; Copeland, Laurel A; Feudtner, Chris (2006). American Journal of Public Health v96:1243-1248. Hopp, Faith P; Woodbridge, Peter; Subramanian, Usha; Copeland, Laurel A; Smith, David; Lowery, Julie C (2006). Telemedicine and eHealth. Zeber, John E; Copeland, Laurel A; Grazier, Kyle L (2006). Military Medicine v171(7):619-626. Alexander JA, Copeland LA, Metzger ME (in press). Clinical Toxicology. Wilkins EG, Lowery JC, Copeland LA, Goldfarb SL, Wren PA, Janz NA (in press). Medical Decision Making. My CV

5. My CV without 2° Data • Shope, Jean T; Copeland, Laurel A; Maharg, Ruth; Dielman, TE; Butchart, Amy T (1993). Health Education Quarterly 20(3): 373-390. • Shope, Jean T; Copeland, Laurel A; Dielman, TE (1994). Alcoholism: Clinical and Experimental Research 18(3): 726-733. • Shope, Jean T; Copeland, Laurel A; Marcoux, Beth C; Kamp, Mary E (1996). Journal of Drug Education, 26(4): 323-337. • Shope, Jean T; Copeland, Laurel A; Maharg, Ruth; Dielman, TE (1996). Alcoholism: Clinical & Experimental Research, 20(5): 791-798. • Copeland, Laurel A; Shope, Jean T; Waller, Patricia F (1996). Journal of School Health, 66(7): 254-260. • Barry, Kristen Lawton; Fleming, Michael F; Manwell, Linda; Copeland, Laurel A (1997). Journal of Family Practice, 45(2): 151-158. • Zimmerman, Marc A; Copeland, Laurel A; Shope, Jean T; Dielman, TE (1997). Journal of Youth and Adolescence, 26(2): 117-141. • Barry, Kristen Lawton; Fleming, Michael F; Manwell, Linda; Copeland, Laurel A; Appel, Scott (1998). Family Medicine, 30(5):366-371. • Blow, Frederic C; Barry, Kristen Lawton; BootsMiller, Bonnie J; Copeland, Laurel A; McCormick, Richard; Visnic, Stephanie (1998). Journal of Psychiatric Research, 32: 311-319. • Valenstein, Marcia; Barry, Kristen Lawton; Blow, Frederic C; Copeland, Laurel A; Ullman, Esther (1998). Psychiatric Services, 49(8): 1043-1048. • Shope, Jean T; Copeland, Laurel A; Kamp, Mary E; Lang, Sylvia W (1998). Journal of Drug Education, 28(3): 185-197. • Kales, Helen C; Blow, Frederic C; Copeland, Laurel A; Bingham, Raymond C; Kammerer, Ericka E; Mellow, Alan F (1999). American Journal of Psychiatry, 156(4): 550-556. • Blow, Frederic C; Barry, Kristen Lawton; Copeland, Laurel A; McCormick, Richard; Lehmann, Laurent; Ullman, Esther (1999). Psychiatric Services, 50(3): 390-394. • Maio, Ronald F; Shope, Jean T; Blow, Frederic C; Copeland, Laurel A; Gregor, MaryAnn; Brockmann, Laurie M; Weber, Janet E; Metrou, Mary E (2000). Annals of Emergency Medicine, 35(3):252-257. • Kales, Helen C; Blow, Frederic C; Bingham, Raymond C; Copeland, Laurel A; Mellow, Alan M (2000). Psychiatric Services, 51(6): 795-800. • Roberts, J Scott; Blow, Frederic C; Copeland, Laurel A; Barry, Kristen Lawton; Van Stone, William (2000). Journal of Geriatric Psychiatry and Neurology, 13 Summer: 78-86. • Blow, Frederic C; Walton, Maureen A; Barry, Kristen Lawton; Coyne, James C; Mudd, Sharon A; Copeland, Laurel A (2000). Journal of the American Geriatrics Society, 48(7): 769-774. • Kales, Helen C; Blow, Frederic C; Bingham, Raymond C; Roberts, J Scott; Copeland, Laurel A; Mellow, Alan M (2000). American Journal of Geriatric Psychiatry, 8:301-309. • Blow, Frederic C; Ullman, Esther; Barry, Kristen Lawton; Bingham, C Raymond; Copeland, Laurel A; McCormick, Richard; Van Stone, William (2000). American Journal of Orthopsychiatry, 70(3): 389-400. • Valenstein, Marcia; Copeland, Laurel A; Owen, Richard; Blow, Frederic C; Visnic, Stephanie (2001). Journal of Clinical Psychiatry 62(7): 545-551. • Valenstein, Marcia; Copeland, Laurel A; Owen, Richard; Blow, Frederic C; Visnic, Stephanie (2001). Psychiatric Services, 52(9): 1242-1244. • Duffy, Sonia A; Terrell, Jeff; Valenstein, Marcia; Ronis, David; Copeland, Laurel A; Connors, Mary (2002). General Hospital Psychiatry, 24: 140-147. • Valenstein, Marcia; Copeland, Laurel A; Blow, Frederic C; McCarthy, John F; Zeber, John E; Gillon, Leah; Bingham, C Raymond; Stavenger, Thomas (2002). Medical Care 40(8): 630-639. • Hegedus, Andrea M; Copeland, Laurel A; Barry, Kristen Lawton; Blow, Frederic C. (2002). American Journal of Orthopsychiatry 72(3): 331-340. • Copeland, Laurel A; Blow, Frederic C; Barry, Kristen Lawton (2003). Health Education & Behavior 30(3):305-321. • Copeland, Laurel A; Zeber, John E; Valenstein, Marcia; Blow, Frederic C (2003). American Journal of Psychiatry, 160:1817-1822. • Valenstein, Marcia; Blow, Frederic C; Copeland, Laurel A; McCarthy, John F; Zeber, John E; Gillon, Leah; Bingham, C Raymond; Stavenger, Thomas (2004). Schizophrenia Bulletin 30(2): 255-64. • Janz, Nancy K.; Wren, Patricia A.; Copeland, Laurel A.; Lowery, Julie C.; Goldfarb, Sherry L.; Wilkins, Edwin G. (2004). Journal of Clinical Oncology 22(15):3091-3098. • Lambert, Michael T.; Terrell, Jeffrey E.; Copeland, Laurel A.; Ronis, David L.; Duffy, Sonia A. (2005). Nicotine & Tobacco Research 7(2):233-241. • Copeland, Laurel A; Fletcher, Carol E; Patterson, Judith (2005). Military Medicine 170(7):602-606. • Copeland, Laurel A; Elshaikh, Mohamed A; Jackson, James; Penner, Lou A; Underwood III, Willie (2005). Cancer 104(7):1372-1380. • O'Neill, Jessica L; Flanagan, Petra S; Zaleon, Carolyn R; Copeland, Laurel A (2005). Pharmacotherapy 25(11):1560-1565. • Copeland, Laurel A; Zeber, John E; Rosenheck, Robert A; Miller, Alexander L (in press). Medical Care. • Pugh, Mary Jo V; Copeland, Laurel A; Zeber, John E; Cramer, Joyce A; Amuan, Megan T; Cavazos, Jose E; Kazis, Lewis E (in press). Epilepsia. • Silveira, Maria J; Copeland, Laurel A; Feudtner, Chris (in press). American Journal of Public Health. • Hopp, Faith P; Woodbridge, Peter; Subramanian, Usha; Copeland, Laurel A; Smith, David; Lowery, Julie C (in press). Telemedicine and eHealth. • Zeber, John E; Copeland, Laurel A; Grazier, Kyle L (in press). Military Medicine. • Alexander, Jeffrey A; Copeland, Laurel A; Metzger, Maureen E (in press). Clinical Toxicology. • Wilkins EG, Lowery JC, Copeland LA, Goldfarb SL, Wren PA, Janz NK (in press). Medical Decision Making.

6. Is it scientifically valid to use data for purposes for which it was not originally collected? • Yes, because…. • No, because… • Maybe…please use caution

7. Today and next week • Uses of secondary data • Common biases encountered with secondary data • Methods of adjustment • Sources of secondary data • Possible effects of HIPAA on this type of research

8. Different Uses • Health Care Delivery • Quality assessment • Geographic variation; under-/over-utilization • Adverse events • Outcomes of a particular treatment • Cost • Natural history • incidence, prevalence • prognosis • Association not causation • Other • Regulatory

9. Some Examples To show you the range of uses To show you the work is publishable

10. Regulatory enforcement • TENET HEALTHCARE - 2002 • Medicare Report Jan-Oct 2004 included these findings: • – At Alvarado Hospital, prosecutors accused Tenet of running covert kickback arrangements • – Document requests were received from federal prosecutors in L.A. • – False claims were uncovered and the company agreed to pay $22.5M to settle allegations • – Redding Medical Center was determined to be performing unnecessary cardiac surgery • – Tenet sold some holdings to Hospital Partners of America to satisfy part of the 2003 settlement with federal officials • – Whistleblowers were to receive over $8M

12. Whose data set is it anyway? Sharing raw data from randomized trials Andrew J Vickers Trials. 2006 May 16;7:15 http://www.trialsjournal.com/content/pdf/1745-6215-7-15.pdf

13. Risk of death in elderly users of conventional vs. atypical antipsychotic medications Wang PS, Schneeweiss S, Avorn J, Fischer MA, Mogun H, Solomon DH, Brookhart MA. N Engl J Med. 2005 Dec 1;353(22):2335-41 http://content.nejm.org/cgi/reprint/353/22/2335.pdf

14. Cancer incidence in Kentucky, Pennsylvania, and West Virginia: disparities in Appalachia Lengerich EJ, Tucker TC, Powell RK, Colsher P, Lehman E, Ward AJ, Siedlecki JC, Wyatt SW (2005). J Rural Health 21(1):39-47. http://www.blackwell-synergy.com/toc/jrh/21/1

15. Does Medicare coverage of colonoscopy reduce racial/ethnic disparities in cancer screening among the elderly? Shih YC, Zhao L, Elting LS (2006). Health Aff (Millwood). Jul-Aug; 25(4):1153-62.http://content.healthaffairs.org/cgi/content/full/25/4/1153

16. Can you think of examples of secondary data analysis that you have read in the medical literature? • Any that have changed your practice or your research direction?

17. Methods of Learning • Lecture • In class epidemiologic exercise • Journal article evaluation – readings are online (http://czresearch.com/dropbox/) • Optional brief proposal for your own secondary data analysis

18. Exercise #1 Policy Decision on FundingOf Regional Trauma Center (handout)

19. Secondary Data Exercise #1 A Regional Trauma Center (which encourages the surrounding hospitals to refer patients with serious injuries to it for expert care) is seeking additional funds from next year’s health budget for more equipment and staff. A local politician (who would rather spend such money on a new hospital named after his father) criticizes this request for additional resources by claming that Regional Trauma Centers do not, in fact, save lives, and he submits the following data to back up his claim. You are asked for your opinion on these data. How should you respond?

20. Secondary Data Exercise #1 A Regional Trauma Center (which encourages the surrounding hospitals to refer patients with serious injuries to it for expert care) is seeking additional funds from next year’s health budget for more equipment and staff. A local politician (who would rather spend such money on a new hospital named after his father) criticizes this request for additional resources by claming that Regional Trauma Centers do not, in fact, save lives, and he submits the following data to back up his claim. By examining the data stratified by severity, the value of the regional trauma center becomes clear.

21. Bias • Systematic error in measurement or a systematic difference (other than the one of interest) between groups • Selection • For cohorts, assembly, migration, contamination, and referral bias • Measurement • Confounding

22. Bias: Anticipate and Control • Restriction (may lose generalizability) • Matching • Stratification • Standardization • Multivariate Adjustment • Assuming the worst (sensitivity analyses) • Discussing possible effects on your results

23. Example: Hospital Mortality Report Cards • Originally unadjusted • Hospitals without trauma centers, doing primarily elective surgery, etc., looked really good • Made hospitals who took care of the sickest of the sick look bad

24. Quality Assessment • Data Quality: Garbage in, garbage out • Risk Adjustment: To remove the confounding effect of different institutions providing care to patients with dissimilar severity of illness and case complexity

25. Interpret data carefully

26. Data Quality • If no reliable (accurate and adequate) data are available, questions about risk adjustment are moot • Inconsistent practices in assigning standard ICD9 codes & names to diseases exist—but this has improved over time (Do you know who assigns the codes?) http://www.eicd.com/ • Lack of specificity of ICD9 system in some diagnostic areas, especially with regard to severity

27. Administrative Data Limitations • Need patient-specific identifiers to link episodes • Need multi-year data when outcomes are infrequent • Limited generalizability if restricted by type of institution or hospital, by type of payor, or by location/region • May lack important clinical variables known to be related to outcomes (especially clinical tests or qualitative evaluations of severity)

28. Risk Adjustment • Controls for those patient characteristics that are related to the outcomes of interest • Removes the confounding effect of different institutions providing care to patients with dissimilar severity of illness and case complexity • Addresses regional variations • Inadequate case-mix adjustment can lead to misclassification of outlier status

29. Risk Adjustment • Primary data collection vs. administrative data • Disease-specific vs. generic • Commercial vs. developed for your study • Predictors vary by outcomes being predicted

30. Essential Elements of Risk Adjustment • Outcome-specific • Contains specification of the principal diagnosis • Contains demographics as proxies for preexisting physiological reserve • Measure count of comorbidities and all the most important comorbidities to assume their own empirically derived coefficients

31. Classification of Disease States • ICD-9: too many specific codes (n~10,000) • Clinical Classifications for Health Policy Research (CCHPR): good for chronic illness and longitudinal care [http://www.ahrq.gov/data/hcup/his.htm] • Primary diagnosis: good for studies that focus on a single episode of care

32. Risk Adjustment: Charlson • Advantages: • Commonly used case-mix classification system in the health care industry • System with which most clinicians and reviewers are familiar

33. Risk Adjustment: Charlson • Disadvantages • Principal diagnosis not differentiated • Original work did not specify ICD-9 codes that went into the disease categories • Developed on inpatients predicting mortality; may not be well suited to outpatients at low risk of death • Not good for longitudinal care / chronic illness

34. Demographic Factors to Consider in Risk Adjustment • Age (e.g., age-adjusted Charlson) • Proxies of Social Support • Marital status • Race • Gender • SES (occupation, employment status, education) • Proxies of Socioeconomic Status • Health insurance status • Home address zip code average income

35. Race and Gender • Don’t adjust for automatically • Ideally adjust for variation in the patients’ physiological reserve and disease burden but not for variation in care rendered to patients

36. Propensity Scores • Useful when dataset is small, to conserve power • Need a good proxy to develop a propensity score • Ask: propensity for what? (Tx) • Published schema may include predictors you want to study separately • Best for non-randomized studies of treatment effect where you want to adjust for the factors that may have influenced the treatment choices

37. Your Assignment • Read other two articles • If you want extra feedback, e-mail me a 1-page proposal (abstract) for a database analysis related to your area of interest any time in next few years copelandl@uthscsa.edu