Plan for Today

1. Knowledge Translation& Common Child and Youth Mental Disorders (CCYMD) Offord Centre for Child Studies, November 26, 2015

2. Plan for Today KT Refresher • KT & evidence-informed decision making • Synthesizing research evidence: the science & the art • Diffusion, dissemination or implementation? • Knowledge to Action Cycle • Implementation Science KT & CCYM Disorders: Are We Ready for Prime Time? • SR, MA & PG: Do they meet international quality standards? • Knowledge repositories & tools: Users guide • Effective Knowledge Implementation Strategies?

3. Team

4. Common Child and Youth Mental Disorders (CCYMD) Anxiety Disorders: Agoraphobia, generalized anxiety disorder, social phobia, specific phobia, panic disorder, separation anxiety disorder Mood Disorders: Major depressive disorder, dysthymia Disruptive Behaviour Disorders: Attention deficit hyperactivity disorder, oppositional defiant disorder, conduct disorder

5. What is Knowledge Translation? CIHR Definition*: • Process: Dynamic and iterative; includes synthesis, dissemination, exchange and ethically sound application of knowledge. • Three goals: improve the health of Canadians, provide more effective health services and products and strengthen the health-care system. * CIHR. Knowledge translation and commercialization. (Updated Nov 17, 2015) http://www.cihr-irsc.gc.ca/e/29529.html

6. But Many Different Terms Out There… McKibbon et al (2010). A cross-sectional study of the number & frequency of terms used to refer to knowledge translation in a body of health literature in 2006: A Tower of Babel? Implementation Science; 5:16. • 100 different terms in 581 articles • CIHR definition used widely.

7. Integrated Knowledge Translation (iKT)* • KUs involved in all stages of research*: • increase relevance • facilitate uptake • Process: takes place within complex system of interactions between researchers and knowledge-users (KUs) which may vary in intensity, complexity and level of engagement depending on the nature of the research and findings, as well as the needs of the particular knowledge user. * CIHR (2015). Guide to Knowledge Translation Planning at CIHR: Integrated and End-of-Grant Approaches. http://www.cihr-irsc.gc.ca/e/45321.html * Gagnon ML. Moving knowledge to action through dissemination and exchange. Journal of Clinical Epidemiology, 2011

8. What is Evidence-Informed Policy & Practice? • Process: • Systematic, transparent use of research evidence in health practice and policy decision making • Principles: • Research evidence provides guidance not prescription • Research evidence doesn’t make decisions – people do • Primum non nocere: Above all, do no harm

9. Evidence-informed Decision Making: 3 Pillars

10. First Step: Synthesize Research Knowledge • Major focus/energy to date • Synthesis can: • Inform research agenda: • Identify research gaps & priorities • Provide rationale for new research • Facilitate research knowledge use in health services decisions: • Strengthen health provider knowledge and intentions • Better care process decisions • Improve health outcomes • Science and art Targets of Change

11. Research Evidence Pyramid: The Science

12. Why Synthesize Body of Knowledge? • Ioannidis, PLoS 2005*: Need adequately powered, low bias evidence: “ … most research questions are addressed by many teams, and it is misleading to emphasize the statistically significant findings of any single team. What matters is the totality of the evidence...” • Ioannidis, JAMA 2005†: • 45/49 highly cited studies claim intervention effective • 16% contradicted by subsequent studies • 16% found bigger effects • 44% replicated • 24% remained largely unchallenged * Ioannidis JPA. Why most published research findings are false. PLoS. 2005. † Ioannidis JPA. Contradicted and Initially Stronger Effects in Highly Cited Clinical Research. JAMA. 2005.

13. Research Evidence Synthesis: The Art • What: Raw or digested? • Primary studies • SR/MA • PGs • Synopses & tools • Curated/quality assessed • Actionable recommendations • Who: Tailored and user friendly? • Practitioners • Patients • Managers • Policy-makers • How: Communication format/medium? • e-Resources • Reminders and updates • Face-to-face • Paper • Combination • Push, pull, push & pull

14. From Knowledge to Action Knowledge Producers Knowledge Users* * Policy-makers; Managers; Clinicians; Patients & Families

15. Knowledge To Action Cycle* * Graham ID, et al. Lost in knowledge translation: Time for a map? The Journal of Continuing Education in the Health Professions. 2006.

16. Action!

17. Knowledge To Action Cycle* * Graham ID, et al. Lost in knowledge translation: Time for a map? The Journal of Continuing Education in the Health Professions. 2006.

18. Implementation Science “Scientific study of methods to promote the systematic uptake of research findings & other evidence-based practices into routine practice to improve the quality and effectiveness of health services and care”* • Identify barriers and facilitators to practice & policy change • Create interventions to promote research uptake • Theories, models & frameworks * Eccles, MP & Mittman BS. Welcome to Implementation Science. Implementation Science, 2006. * Nilsen P. Making sense of implementation theories, models and frameworks. Implementation Science, 2015.

19. Theories, Models & Frameworks* Three aims: • Describe translation process • Understand or explain what influences implementation outcomes • Evaluate implementation * Nilsen P. Making sense of implementation theories, models and frameworks. Implementation Science, 2015.

20. Consolidated Framework for Advancing Implementation Science (CFIR) • Damschroder, 2009*: • Identified 19 theories/models/frameworks • Created CFIR • Overarching typology to promote implementation theory development and verification about what works where and why across multiple contexts *Damschroder LJ, et al. Fostering implementation of health services research findings into practice: Consolidated framework for advancing implementation science. Implementation Science, 2009.

21. CFIR: Domains & Constructs

22. Effective Knowledge Implementation Interventions • Implementation processes combined with high quality research knowledge • Grimshaw et al, 2012*: What we know so far • Professional behaviour change • Policy makers and senior health service managers * Grimshaw J, et al. Knowledge translation of research findings. Implementation Science, 2012.

23. Effective Knowledge Implementation Strategies Professional Behaviour Change Strategies • Printed educational materials • Educational meetings • Educational outreach • Local Opinion Leaders • Audit and Feedback • Computerized Reminders • Tailored Interventions Policy-makers and Senior Health Service Managers • ? but many innovative approaches developed and worthy of testing

24. KT & CCYM Disorders: Are We Ready for Prime Time? • Research Synthesis: SR/MA & PG Quality? • Dissemination: Knowledge Repositories & Tools – Coverage & Quality? • Implementation: Effective Strategies?

25. Do Systematic Reviews and Meta-analyses About CCYM Disorders Meet International Quality Standards? • Cochrane & PRISMA Systematic review methods • Research librarian created & conducted search • Two independent reviewers • Inclusion criteria: • Systematic review or meta-analysis • Prevention or treatment • Anxiety, depression, suicide related behaviors • Aged ≤18 years • English language • 2000 – 2012

26. PRISMA Flow Diagram 4194 Unique Records Identified & Screened 85 Eligible Reviews 14 20 35 16 Anxiety & Depression SRB Anxiety Depression

27. Quality Assessment Methods AMSTAR* • Assessing the Methodological Quality of Systematic Reviews • 11 items • Minimum quality defined as: • Systematic reviews: 5/9 • Meta-analyses: 6/11 • 2 raters; Disagreements resolved by consensus *SheaBJ, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology, 2009.

28. How Many Eligible Reviews Meet Minimum Quality Standards?

29. AMSTAR CriteriaFailure Rate * Data based on reviews containing meta-analyses only (n=42)

30. Did Introduction of AMSTAR in 2007 Improve Review Quality? No significant difference in AMSTAR scores 2007-2009 compared to 2010-2012; t(54)= -1.08, p = 0.286.

31. Does Journal Impact Factor Predict Review Quality? Correlation between AMSTAR Score & Impact Factor: r = 0.25, p<0.05

32. Do Practice Guideline Development Methods Meet International Quality Standards?* • Cochrane and PRISMA systematic review methods • Research librarian created & conducted search • Two independent reviewers Step 1: Find CYMH PGs (published or updated 2009-2014) • Journals (4) of leading CYMH professional organizations • National Guideline Clearinghouse web-site • Websites of organizations who produce PGs: NICE, SIGN, USPSTF Step 2: Find CYMH development methods • Eligible PGs reviewed to identify cited or associated development methods *Bennett K, et al. Practitioner Review: On the trustworthiness of clinical practice guidelines: A systematic review of the quality of methods used to develop guidelines in child and youth mental health. Journal of Child Psychology and Psychiatry (Forthcoming).

33. PRISMA Flow Diagram PGs Identified Unique PGs Screened PGs Eligible PGs PG Development Methods Screened PG Development Methods Eligible PG Development Methods Sets

34. PG Quality Rating Methods AGREE II*: • Appraisal of Guidelines for Research and Evaluation • Validated PG quality assessment tool • 6 domains Scoring Options: *Brouwers M, et al. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. CMAJ, 2010.

35. AGREE II Ratings AACAP = American Academy of Child and Adolescent Psychiatry AAP = American Academy of Pediatrics USPSTF = U.S. Preventive Services Task Force NICE = National Institute for Health and Care Excellence SIGN = Scottish Intercollegiate Guidelines Network 0 = not mentioned 1 = optional - low confidence 2 = optional - moderate confidence 3 = mandatory - high confidence

36. Summary 68.5%

37. Conclusions • Up to 69% of PGs developed using methods that don’t align with AGREE II • Do professional specialty societies have the resources required to produce high quality PGs? • Quality of individual PGs needs to be assessed (underway) • Need strategies to: • Guide users to high quality PGs quickly • Facilitate adherence by PG developers to quality standards • Avoid harm and wasted resources

38. Dissemination: e-Resources/Technologies Repositories and Clearinghouses: • What’s out there? • How good are they? • Do they meet user needs? • Pilot study findings

39. Research Repositories: ‘Best in Class’ • Rx for Change • EvidenceUpDates • HealthEvidence.org • HealthSystemsEvidence.org • Cochrane Database of Systematic Reviews

40. ‘Best In Class’Knowledge Repositories * OQAQ = Overview Quality Assessment Questionnaire (Oxman & Guyatt, Journal of Clinical Epidemiology, 1991) † # of hits when searching for ‘child and adolescent mental health’

41. CCYMD Repositories • SAMHSA National Registry of Evidence-based Programs and Practices (NREPP) • California Evidence-based Clearinghouse for Child Welfare (CEBC) • Child Trends LINKS database • Canadian Best Practices Portal • What Works Clearinghouse • Ontario Centre of Excellence for CYMH Evidence In-Sight and Policy Ready Papers • U-Mind

42. CCYMD Knowledge Repositories 1

43. CCYMD Knowledge Repositories 2

44. Summary • What is good enough? • Who are the users? • What are their needs?

45. Effective Knowledge Implementation Interventions: 2 SRs So Far Novins et al (2013)*: • 73 eligible articles (16 RCTS)based on 44 host studies • Inner context factors: • Strongest evidence: fidelity monitoring & supervision • Less evidence for organizational climate/culture but better sustainment and youth outcomes • Outer context factors: • Strongest evidence: training and use of special technologies Barwick et al (2012)†: • 12 intervention studies • Poor quality; self report of behavior change *Novins DK, et al. Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. JAACAP, 2013. † Barwick MA, et al. Knowledge translation efforts in child and youth mental health: A systematic review. Journal of Evidence-based Social Work, 2012.

46. What Are We Doing Re Implementation? • CIHR Dissemination Grant: • Disseminating Child and Youth Mental Health Practice Guidelines: The Development of a User-Informed, Social Media Integrated, Mobile Website • Tailored PG implementation tool

47. Questions & Thank-you kbennett@mcmaster.ca