A Primer on Outcomes Research

1. Vidya Sharma MBBS, MPH. A Primer on Outcomes Research The Children’s Mercy Hospital

2. Objectives • Understand the principles of outcomes research. • Define outcome measures and variables • Understand the advantages/limitations of the outcomes research.

3. The Promise of Health ServicesResearch “Health services research can save more lives in the next decade then can bench science, work on the human genome, stem cell therapy and cancer research..” Atul Gawande MD

4. Health Services Research inPediatrics • A core body of pediatrics relevant health services literature already exists • Outcomes research • Patient satisfaction research • Expert practice • Utilization research • Health disparities

5. Outcomes Research • What is outcomes research? • Outcomes research studies the end results of medical care • End results include effects that people experience and care about, such as change in the ability to function • For individuals with chronic conditions end results include quality of life as well as mortality • The effects of the health care process on the health of patients and populations

6. Outcomes Research • What works? • For which patients? • At what cost? • From whose perspective?

7. Case • 5 year old fell from a grocery cart • Dazed 60 seconds • Lethargic 20 minutes • Exam –no focal findings • CT frontal fracture no i/c bleed • Observed 6 hours and d/c with follow up with PCP

8. Clinical Research • Duration of altered mental status • Vomiting • Cost of care • Rates of return to ED • Disposition: discharge, admit, death

9. Outcomes Research • Clinical effectiveness: evidenced based treatment • Quality of care: adherence to clinical guidelines • Quality of life: return to normal activities • Patient satisfaction: satisfaction with discharge instructions, seen in a timely manner • Cost effectiveness: comparisons of admission vs observation, CT or not • Organization of care: pre-hospital, referral

10. Clinical effectiveness Quality of care Quality of life Patient satisfaction Organization of care Death Disease Disability/Discomfort Dissatisfaction Dollars Domains of Outcomes Research

11. Types of Outcomes Research • Evaluating effectiveness of medical/ surgical procedures • Measuring impact of insurance/ reimbursement policies on outcomes of care • Development and use of tools to measure health status • Analyze the best way to disseminate results to encourage physician/consumer behavior change

12. What makes Outcomes Research different? • The question it asks • The settings it studies • The method it uses • The health status it measures

13. What makes Outcomes Research different? • The questions it asks • all aspects of the health care delivery system • clinical encounter • organization of the health care system • financing of the health care system • regulation of the health care system

14. What makes Outcomes Research different? • The question it asks • The settings it studies • The method it uses • The health status it measures

15. What makes Outcomes Research different? • The setting it studies • contrast with randomized controlled studies • efficacy studies • research in real-life settings • effectiveness studies

16. Definitions • Efficacy • Probability of benefit from an intervention under standardized or ideal conclusions • Randomized control trials • Effectiveness Studies • Results achieved in actual practice with different patients and providers • Research in real life settings

17. What makes Outcomes Research different? • The question it asks • The settings it studies • The method it uses • The health status it measures

18. What makes Outcomes Research different? • The methods it uses • use existing computerized databases • patient questionnaires • meta-analysis • complement standard strategies

19. Types of Projects Using Databases • Health Services Research • Data is used to examine access, use, costs, delivery, and organization of health services to better understand the structure and effects of health services for populations • Intervention Projects • A therapy or practice is compared with similar populations which do not use therapy

20. Types of Studies • Adherence to clinical practice guidelines • Costs of Care • Resource utilization of special groups of patients • Use of specific therapies • Pharmaceutical use

21. Observational Studies • Cohort study • Patients with exposure X differ from patients without exposure X on an outcome • Case Control Study • Do cases differ from controls on some exposure • Historical Controls Study • Outcome of interest compared with historical records

22. RCT error on side of not finding an effect generalizability limited to patient group studied populations are equivalent small differences can be detected small numbers of patients Observational error on side of finding an effect ordinary medical practice adjust for differences by multivariate analysis large differences can be detected large numbers of patients Contrasting RCTs with Observational studies

23. Observational Studies • Association is not always causation • Causation is implied when: • Strength(large risk, rules out other factors) • Consistency (replicated by different researchers and in different conditions) • Specificity (exposure related to a specific disease than a wide range) • Temporality (exposure preceded disease) Hill B. The Environment and Disease, Association or Causation? 1995.Proc R Soc Med;58:295.

24. Observational Studies • Causation is implied when: • Biological Gradient (dose related response) • Plausibility (credible scientific explanation) • Coherence (association consistent with natural history of the disease) • Experimental evidence (physical intervention shows results associated with the association) • Analogy (similar result that we can draw a relationship to) http://www.childrens-mercy.org/stats/ask/causation.asp

25. Health Services Research Goodman DM, Mendez E, Throop C, Ogata ES: Adult survivors of pediatric illness: the impact on pediatric hospitals. Pediatrics 2002;110:583-589.

26. Adult Survivors • Observational study using PHIS, covering the years 1994-1999 • Described the demographics of adult in-patients in pediatric facilities • Used census data to estimate national impact • Used HCFA data to estimate financial impact across continuum of care

27. 18-64 Years Old 21-64 Years Old All Ages Number of discharge episodes 5051 2127 103,733 Number of patients 3863 1785 86,188 Average LOS 7.48 8.49 6.19 Adult Survivors

28. 18-64 Years Old 21-64 Years Old All Ages Avg charge/ discharge $26,640 $31,114 $20,444 Medicaid/ Medicare 32%/7.4% 27%/13.6% 35%/0.8% Adult disch/ total disch 4.9% 2.1% Adult charges/ total charges 6.3% 3.1% Adult Survivors

29. Diagnostic Group 18-64 years old (% of pts) 21-64 years old (% of pts) Mental retardation/cerebral palsy 10.2 10.7 Cystic fibrosis 21.8 34.5 Malignancy 7.6 5.6 Congenital heart disease 9.9 11.0 Adult Survivors

30. Adult Survivors

31. Intervention Study Todd J, Bertoch D, Dolan S: Use of a large national database for comparative evaluation of the effect of a bronchiolitis/viral pneumonia clinical care guideline on patient outcome and resource utilization. Arch Pediatr Adolesc Med 2002;156:1086-1090

32. Bronchiolitis/Viral pneumonia • Clinical guideline developed and implemented • Effect of guideline determined by internal comparison across time and external comparison using PHIS

33. Bronchiolitis/Viral pneumonia

34. Bronchiolitis/Viral pneumonia

35. Bronchiolitis/Viral pneumonia

36. Care for Cystic Fibrosis

37. Best or Worst Patient Outcomes? • Best and worst Cystic Fibrosis centers offered the virtually the same treatments to patients!! • Striking differences in the way that care was delivered within centers

38. Results • The best Cystic Fibrosis Center had leadership from someone who: • Aggressively pursued consistency of care • Individualized his approach to each patient • Engaged patients in their own care process

39. Potential Pitfalls • Original goals for which the data was collected may not coincide with subsequent use • Dataset may not have been designed or maintained to maximize consistency or data quality

40. Potential Pitfalls • More data collected than needed • Infrequently used data elements may be unreliably coded • Codes may change over time • Geographic variation in coding

41. Issues with Databases • Required data elements • Are they present? • Are they properly coded? • Principal diagnosis only • All diagnoses

42. Issues with Databases • Sampling • what population? • which hospitals? • what data? • Record Structure • Discharge episode • Clinical data

43. Approach to using secondary Data • Clearly define your question: • P population • I intervention • C comparison groups • O outcome

44. Planning a Database Project • It always takes longer than you think • Would you read it if someone else wrote it? • Being better is more important than being the first • Will the results be worthwhile no matter what you find? • The probability of success is inversely related to the number of collaborators. Goldman l, et al. J of Gen Int Med. 1986;1 (Suppl):S25-S30.

45. What makes Outcomes Research different? • The question it asks • The settings it studies • The method it uses • The health status it measures

46. What makes Outcomes Research different? • The health staus it measures • Traditional: more proximal outcomes • laboratory test results • complication rates i.e. infections • death • Outcomes: measures distal outcomes • functional status • patient well-being / quality of life • satisfaction with care

47. How is health status measured? • Questionnaires that measure aspects of health • Functional Status • Well- being / quality of life • Satisfaction with Care

48. How is health status measured? • Functional status • physical functioning i.e. walking,etc • role functioning i.e. going to school etc • social functioning i.e. playing with friends etc

49. How is health status measured? • Well-Being / quality of life • mental health i.e. general mood • health perceptions i.e. person’s view of health • quality of life • pain i.e. the extent of pain experienced

50. How is health status measured? • Satisfaction with care • access • convenience • information received • financial coverage • technical quality