Three Step Guide in Using an Article to Assess Therapy

Three Step Guide in Using an Article to Assess Therapy

Three Step Guide in Using an Article to Assess Therapy • Are the results of the study valid? • What are the results? What measures of precision of effects were reported (CIs, p-values)? • How can I apply these results to patient care?

Assess Validity and Applicability to my practice setting 1.Is the study a randomized control trial (RCT)? Yes (go on) No (stop) 2.Were the patients properly selected for the trial and randomized with concealed assignment? Yes (go on) No (stop) 3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause) 4.Were the intervention and control groups similar at the start? (Check “Table 1” of most studies) Yes (go on) No (stop) 5.Was follow-up complete? ii. Were patients analyzed in the groups to which they were randomized (“intention-to-treat” analysis)?

Simple randomization • An almost infinite number of methods can be used to generate a simple randomisation sequence based on a random-number. For example, for equal allocation to two groups,predetermine the direction to read the table: up, down, left, right, or diagonal. Then select an arbitrary starting point—ie, first line, 7th number: 56 99 20 20 52 49 05 78 58 50 62 86 52 11 88 31 60 26 13 69 74 80 71 48 73 72 18 60 58 20 55 59 06 67 02 . . . • For equal allocation, an investigator could equate odd and even numbers to interventions A and B, respectively. • Therefore, a series of random numbers 05, 78, 58, 50, 62, 86, 52, 11, 88, 31, &c, represent allocation to intervention A,B, B, B, B, B, B, A, B, A, &c. • Alternatively, 00–49 could equate to A and 50–99 to B, or numbers 00–09 to A and 10–19 to B,ignoring all numbers greater than 19. • Any of a myriad of options suffice, provided the assignment probabilities and the investigator adhere to the predetermined scheme.

Benefits of Random Allocation (Randomization) 1.Reduces bias in those selected for treatment • guarantees treatment assignment will not be based on patients’ prognosis 2.Prevents confounding • known and unknown potential confounders are evenly distributed

Is allocation concealed?

BEST – most valid technique • Central computer randomization DOUBTFUL • Envelopes, etc Ensuring Allocation Concealment NOT RANDOMIZED • Date of birth, alternate days, etc

Do Not Confuse Allocation Concealment with Blinding • Allocation concealment seeks to prevent selection bias, protects assignment sequence before and until allocation, and can always be successfully implemented

Importance of allocation concealment • Trials that used inadequate or unclear allocation concealment, yielded up to 40% larger estimates of effect. • Moreover, the worst concealed trials yielded greater heterogeneity in results—ie, the results fluctuated extensively above and below the estimates from better studies. • Indeed, having a randomized (unpredictable) sequence should make little differencewithout adequate allocation concealment.

Do Not Confuse Allocation Concealment with Blinding(Cont’d) • Blinding seeks to prevent information bias, protects sequence after allocation, and cannot always be successfully implemented

Placebo effectTrial in patients with chronic severe itching No treatment Trimeprazine tartrate Cyproheptadine HCL Treatment vs no treatment for itching

Placebo effectTrial in patients with chronic severe itching No treatment Trimeprazine tartrate Cyproheptadine HCL Placebo Treatment vs no treatment vs placebo for itching Placebo effect - attributable to the expectation that the treatment will have an effect

Double-Blinded Single-Blinded

Blinding and Reporting • Usually reduces differential assessment of outcomes (information bias) • Authors should explicitly state who was blinded – and how. • Many investigators and readers consider a randomized trial as high quality simply because it is “double-blind,” as if double-blinding is the sine qua non of an RCT. • Trials not “double-blinded” should not automatically be deemed inferior trials.

Blinding in randomised trials: hiding who got what • Double blinding prevents bias but is less important, than adequate allocation concealment. • open studies are more likely to favour experimental interventions over the controls and that studies that are not double-blinded can exaggerate effect estimates by 17% • Furthermore, in some trials, blinding cannot be successfully implemented, whereas allocation concealment can always be successfully implemented.

Comparison of baseline data Does P>0.05 indicate comparability of treatment groups? Chan et al. Lancet 1997

Significance tests for baseline differences INAPPROPRIATE Chan et al. Lancet 1997

Baseline data Effect of azathioprine on the survival of patients with primary biliary cirrhosis Christensen et al. Gastro 1985

How complete was the follow up? How many dropouts were there? • Conventionally, a 20% drop out rate is regarded as acceptable, but this depends on the study question. • Some regard should be paid to why participants dropped out, as well as how many. • It should be noted that the drop out rate may be expected to be higher in studies conducted over a long period of time. • A higher drop out rate will normally lead to downgrading, rather than rejection of a study.

Bias: a one-sided inclinationOf the mind Over-estimation of treatment effect • Not random 40% • Not double-blind 17% • Duplicate information 20% • Small trials 30% • Poor reporting quality 25%

Intention to treat Montorri V, Guyatt G. CMAJ 2001 165(10) p1340

Intention to treat High risk? Montorri V, Guyatt G. CMAJ 2001 165(10) p1340

Summary 1.Is the study a randomized control trial (RCT)? Yes (go on) No (stop) 2.Were the patients properly selected for the trial and randomized with concealed assignment? Yes (go on) No (stop) 3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause) 4.Were the intervention and control groups similar at the start? (Check “Table 1” of most studies) Yes (go on) No (stop) 5.Was follow-up complete?

Three Step Guide in Using an Article to Assess Therapy (Part-2)

Three Step Guide in Using an Article to Assess Therapy • Are the results of the study valid? • What are the results? What measures of precision of effects were reported (CIs, p-values)? • How can I apply these results to patient care?

Three Step Guide in Using an Article to Assess Therapy • Are the results of the study valid? • What are the results?What measures of precision of effects were reported (CIs, p-values)? • How can I apply these results to patient care?

Measuring Risk:Relative Risk Relative Risk (RR) = rate in exposed = 0.04 = 0.67 rate in nonexposed 0.06 SHEP. JAMA. 1991;265:3255-3264

Communicating risk: 40% RRR=? ARR=? 20% 20% 10% 10% 5% placebo

Measuring Risk:Relative Risk 6% 4% Relative Risk (RR) = rate in exposed = 0.04 = 0.67 rate in nonexposed 0.06 SHEP. JAMA. 1991;265:3255-3264

Measuring Risk: ARR? 6% 2% 4%

Measuring Risk:Absolute Risk Reduction Absolute Risk Reduction (ARR)is the absolute difference in event rates between the experimental and control patients. Calculated by: ARR = CER - EER = 0.06 - 0.04 = 0.02 In its decimal form the ARR is not easy to use! Converted to a percentage - there is an absolute risk reduction of 2%

Measuring Risk: RRR? 6% 2% 4%

Measuring Risk:Relative Risk Reduction Relative Risk Reduction (RRR)is the proportional reduction in event rates between the experimental and control patients. Two ways to calculate: RRR = (1 - RR) = (1 - 0.67) = 0.33 OR RRR = CER - EER = 0.06 - 0.04 = 0.33 CER 0.06 Therefore, treatment reduced the stroke rate by 33% OR a RRR of 33% means that the new treatment reduced the risk of death by 33% relative to that occurring among control patients CER = control event rate EER = experimental event rate

Relative Risk Reduction: • Usually reported in studies. • Ratio of the improvement of outcome over outcome without intervention (Rx): • {Control Event Rate (CER) — Experimental Event Rate (EER)} / CER • i.e. {CER-EER}/CER • often independent of prevalence! • often similar at different ages!

Absolute Risk—> The risk our patient is facing! • How likely is our patient to die (or have the outcome of interest) without intervention? = Control Event Rate (CER) • consider this individual patient’s risk factors to estimate Patient Expected Event Rate = PEER. • Absolute Risk usually increases with age. • Improvement measured as Absolute Risk Reduction (ARR)

Usefulness of the ARR:Number Needed to Treat Number Needed to Treat (NNT)is the number of patients a clinician needs to treat in order to prevent one additional adverse outcome. NNT is for dichotomous outcomes. Calculated by: NNT = 1/ARR = 1/0.02 = 50 Therefore, you would have to treat 50 hypertensive patients to prevent one stroke.

Number Needed to Treat CER=0.06=100----------6 EER=0.04=100----------4 SO: 100-----------2 X------------1 50-----------1 Calculated by: NNT = 1/ARR = 1/0.02 = 50 Therefore, you would have to treat 50 hypertensive patients to prevent one stroke.

Why not just use RRR?PTH trial CER = 6 % Age =70 + 7 EER = 3 % ARR = CER – EER = 3% NNT = 1/ARR = 1/ 0.03 = 33 CER = 1/1000 Age = 55 + 5 EER = 1/2000 ARR=1/1000-1/2000=1/2000 NNT = 1/ARR= 2000

Three Step Guide in Using an Article to Assess Therapy