appreciate • appraise • apply Part 6 Brenda Boucher PT, PhD, CHT, OCS, FAAOMPT
Evidence Based Practice The Diagnostic Process The Intervention Process The Intervention Process
The Intervention Process Strategies in best practice
1 2 5-Step Process 3 4 5
What are best treatment strategies?
1 2 5-Step Process 3 4 5
Develop an answerable question 1 Background & Foreground
Background Questions “What is spinal stenosis?” “What are the effects of aging on osteoarthritis” Information related to disorders, pathologies, conditions etc . . .
Foreground Questions Identify evidence regarding the use of a specific intervention in the management of a particular patient
PICO “In a 65 y/o patient with lumbar spinal stenosis, what is the effectiveness of exercise and manual therapy compared to exercise alone to decrease pain ?”
Identify the evidence for treatment 2 MEDLINE McMaster University’s Health Information Research Institute for EBP PEDro PubMed Clinical Queries CINAHL Hooked on Evidence The Cochrane Library
Critically appraise the evidence 3 How does? What if? Why? PICO Oh To begin the critical appraisal . . . . . . acknowledge the inherent strength of a study by understanding its design
Critically appraise the evidence 3 Hierarchy of Evidence bias minimized
Critically appraise the evidence 3 Internal & External Validity
Internal Validity: “Was the research done right?” Intervention Outcomes What you do What you measure Selection – Competing Interventions – History – Maturation – Instrumentation - Testing
External Validity: “Does the same thing happen in other settings?” Clinical Practice Study Rsults
Critically appraise the evidence Random Allocation Concealment of Allocation Blinding Follow-up Intention to Treat 3 Internal Validity
Critically appraise the evidence Random Allocation (Internal Validity) 3 Random allocation infers: difference between groups at end of study due to treatment or chance Random allocation infers: effect of treatment responsible for differences found between groups
Critically appraise the evidence Concealment of Allocation (Internal Validity) 3
Critically appraise the evidence Blinding (Internal Validity) 3 Rater bias: possibility that an investigator’s belief in the effectiveness of treatment may unintentionally impact measurement of outcome Subject bias: potential for subjects to demonstrate an apparent effect either by placebo or simply being a part of the study
Critically appraise the evidence Follow up (Internal Validity) 3 • Drop-out can seriously distort outcomes. Possible reasons why subjects drop out: • Got better • Got worse • Sought alternate treatment • Other?
Critically appraise the evidence Intention to Treat (ITT) (Internal Validity) 3 Intention to Treat (ITT) ITT approach: substitute a value for missing data points. Most often, the last known value is carried forward to any subsequent missed points.
Critically appraise the evidence 3 Statistical Significance How different were the groups? “p < .05” Statistical significance refers to whether the effect of the treatment is bigger than can be reasonably be attributed to chance alone.
Critically appraise the evidence 3 Clinical Significance Size of the Intervention’s Effect (size of effect) Best estimate of the difference between groups, or the size of the intervention’s effect, is the average difference between groups.
Critically appraise the evidence 3 Size of the Intervention’s Effect Specific Exercise Group mean initial pain rating = 7/10 Postural Analysis Group mean initial pain rating = 7/10 mean final pain rating = 5/10 mean final pain rating = 2/10 Mean Effect of Treatment = 3 cm (5-2 = 3)
Critically appraise the evidence 3 • Is the evidence clinically useful? • Recognize the inherent strength of a study (think pyramid) • Understand common threats to Internal Validity • Determine the generalizability of a study (External Validity) • Understand the meaning of “statistical significance” (p < .05) • Appreciate the size of the intervention’s effect (“clinical significance”) “It’s unlikely that the outcomes of interest in this study are due to chance.” OR “It’s very likely the outcomes in this study are a result of the treatment of interest.”
References: • Fritz JM, Wainner RS. Examinning diagnostic tests: an evidence-based perspective. Physical Therapy. 2001;81(9):1546-1564. • Cleland JA, Noteboom JT, Whitman JM, Allison SC. A primer on selected aspects of evidence-based practice relating to questions of treatment, Part 1: asking questions, finding evidence, and determining validity. JOSPT. 2008;38(8):476-484. • Noteboom JT, Allison SC, Cleland JA, Whitman JM. . A primer on selected aspects of evidence-based practice relating to questions of treatment, Part 2: interpreting results, application to clinical practice, and self evaluation. JOSPT. 2008;38(8):485-501. • Gyatt G, Rennie D. User’s Guide to Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago: AMA Press, 2002. • Introduction to critical appraisal. http://www.otseeker.com/Info/Tutorial.aspx