Randomized Controlled Trials in Health Services Research

1. Randomized Controlled Trials in Health Services Research Morris Weinberger, PhD Senior Career Scientist, HSR&D Service Investigator, Center for Health Services Research, Durham VAMC VA Cyber-Seminar, January 12, 2009

2. Overview to Today’s Seminar • Overview of randomized controlled trials • Minimizing threats to internal validity • Targets of interventions • Analytical issues • Practical issues and advice

4. Overview of Randomized Controlled Trials • Bias • Systematic (non-random) error • Bane of research, regardless of study design • Can invalidate study results • Can occur in any phase of research

5. Overview of Randomized Controlled Trials • Most powerful research design to establish causality, including the effectiveness of interventions • Establishes causality by controlling for confounding factors • Well-designed experiments minimize bias • Not suitable for all research questions

6. Overview of Randomized Controlled Trials • Subjects randomized to treatment groups • Follow subjects prospectively • Compare subjects across treatment groups on relevant outcomes

7. Required CONSORT Figure

8. Overview to Today’s Seminar • Overview of randomized trials • Minimizing threats to internal validity

9. Validity • Internal validity: Can the observed differences between groups be attributed to the intervention? • Randomization • External validity: Are the observed differences in your study representative of patients/subjects in general? • Random sampling

10. Threats to Internal Validity • Biased assignment of patients to groups • Biased outcome assessment • Non-compliance with treatment protocol • Dropouts • Co-intervention • Contamination

11. Minimizing Threats to Internal Validity • Randomization • Blinding (Masking) • Intervention design • Study protocol

12. Randomization • Random assignment of subjects to study groups: • Produces study groups comparable with respect to measured and unmeasured characteristics • Removes investigator bias in assigning patients to groups • Increases validity of statistical tests • If allocation of subjects to groups is predictable, it may lead to bias, e.g., decision to participate

13. Minimizing Threats to Internal Validity • Randomization • Blinding (Masking)

14. Minimizing Threats to Internal Validity • Randomization • Blinding (Masking) • Intervention design

15. Considerations When Designing Interventions • What is the intervention? • Is it likely to be potent? • Is it ethical? • Is it practical and feasible in the “real world”? • Will it be acceptable to patients? • Is it effective?

16. What is the Intervention?Standardization • Who did what to whom? • What was the dose? • How often? • For how long? • Administered under what conditions? • With what dose adjustments?

17. Minimizing Threats to Internal Validity • Randomization • Blinding (Masking) • Intervention design • Study protocol

18. Study Protocol • Choice of control group • Recruitment strategies • Retention strategies • Outcome measures • Evaluating effectiveness of the intervention • Compliance with treatment protocol • Co-intervention • Contamination

19. Choosing the Control Group • Nothing • Usual Care • Placebo

20. Recruitment Strategies • Recruitment sources • Community vs. patients • Volunteer bias • Barriers to recruitment • Interest in subject • Distrust of research • Distrust of contact from unknown persons • Transportation • Informed consent • Long enrollment visits

21. Retention • Impact on external validity of the trial • Who completes the trial? • Impact on internal validity of the trial • What if there is differential dropout? • Impact on number of patients recruited • How will my sample size estimates be affected?

22. Design Considerations:Choosing Measures • Properties of the measure • Validity and reliability • Floor and ceiling effects • Sensitivity to change • Pragmatic considerations • Setting • Respondent burden • Appropriateness for subjects • Cost

23. Evaluating Effectiveness of the Intervention • Dose: Was the intervention delivered? • Contamination: Did the control group receive components of the intervention? • Co-intervention: Did the treatment group receive interventions other than what was intended? • Key: Measure what elements of the intervention were delivered to all study groups

24. Overview to Today’s Seminar • Overview of randomized trials • Minimizing threats to internal validity • Targets of interventions

25. Targets of Strategies to Improve Outcomes • Patients • Providers • System

26. Patient-Level Strategies • Randomization is at the patient level • Simplifies the statistical analysis

27. Provider-Level Strategies • Strategies to improve quality of care by intervening with providers • If outcomes are at physician level, issues are generally similar to patient-level interventions

28. Provider-Level Strategies • Often, goal is to evaluate the impact of provider interventions on patient outcomes because • Expect intervention with provider to affect patient outcomes (e.g., improving patients’ glycemic control) • Concern that providing intervention to intervention patients will affect providers behavior with control patients (i.e. contamination) • It seems like effective sample size should be greater than the number of physicians randomized

29. Provider-Level Strategies • Randomizing patients assumes balance on both measured and unmeasured variables • Analytically, assumes that patients are independently assigned to groups • When unit of randomization is not the unit of analysis: • Patients are not independent within physicians • Physicians not independent within setting (e.g., team, hospital) • Complicates sample size estimates

30. Provider-Level Strategies:Summary • Often, interventions to improve outcomes target providers, but analyze patients • Reasonable, and perhaps only plausible strategy • Must provide reviewers with clear justification (i.e., minimizes threats to internal validity that would otherwise result) • Involve biostatisticians early, as the analyses (including sample size calculations) are complex

31. Design Considerations: Biomedical vs. HSR Trials ConsiderationBiomedicalHSR • Patients • Eligibility Narrow Broad • Randomization Patient Patient, Physician, Clinic, Hospital • Intervention: - Components Single Multiple - Type Drug, device, Structure of care procedure - Uniformity High Low

32. Design Considerations: Biomedical vs. HSR Trials ConsiderationBiomedicalHSR • Intervention • Control group Standard Usual Care • Assess compliance Easy-Moderate Difficult • Data Collection • Outcomes Events, test result Patient-centered • Process of care Easy-moderate Difficult • Cost Easy-moderate Difficult • Blinding Possible Not possible

33. Overview to Today’s Seminar • Overview of randomized trials • Threats to internal validity • Targets of interventions • Analytical issues

34. Analytical Issues • Between- versus within-group comparisons • Primary analysis is between-groups • Controlling for baseline differences • Randomization does not achieve balance on key factors

35. Analytical Issues • Intention to Treat: Subjects analyzed as part of original group, regardless of compliance, dropout, or crossover • Question answered: What is benefit of treatment/intervention as given? • Per protocol: Focus on subjects who were compliant with protocol • Question answered: What is benefit for people who are compliant?

36. Analytical Issues • Sub-group analyses: Focus on subjects with certain characteristics • Question answered: Does the treatment work for certain types of patients • Disaggregating complex interventions: Can we identify the effective component? • Unit of assignment versus unit of analysis

37. Overview to Today’s Seminar • Overview of randomized trials • Threats to internal validity • Targets of interventions • Multi-site trials in health services research • Analytical issues • Practical issues and advice

38. Practical Issues and Advice • What help do I need? • Am I ready to conduct a clinical trial? • Is the intervention worth evaluating? • Is the project feasible? • What outcomes can I reasonably expect to change?

39. What Help Do I Need? • Colleagues with content and methodological expertise and/or other specialized skills • Biostatistician

40. Biostatisticians Are Your Friends • Study design • Sample size • Measuring outcomes • Data management • Statistical analysis

41. Am I Ready to Conduct a Clinical Trial? • Equipoise • Advance previous study • Good idea, flawed study • Application to another patient population • Application to another clinical venue • Relevance • Policy makers • Clinicians • Health care organizations

42. Is the Intervention Worth Evaluating? • Is the study ethical? • Is the intervention feasible and practical? • If effective, will intervention be accepted and useful? • Health care organization • Physicians • Staff • Patients

44. Is the Project Feasible?Pilot Study as Dress Rehearsal • Are there enough subjects? • Do I have a practical strategy to identify, enroll and retain subjects in the study? • Do I have time to complete the project? • Do I have resources to complete the study? • Can I measure the critical variables? • Are data collection forms reasonable? • Do I have buy-in from the organization and personnel (physicians, nurses, clerks, etc.)?

45. What Outcomes Can I Reasonably Expect to Change? • What should my outcomes be? • Mortality and/or morbidity • Clinical parameters • Health-related quality of life • Health services utilization/cost

46. Final Advice • Specify the hypotheses • Write early, write often: • methods • dummy tables • Monitor what is happening • Enrollment • Retention • Delivery of the intervention • Review CONSORT statement on reporting randomized trials