1 / 54

Introducing systematic reviews

Introducing systematic reviews. Karin Hannes Centre for Methodology of Educational Research. Outline. Intro - daily practice: evidence-based? Systematic reviews What is it and why do we need them? How do they look like? Who is producing them? Where can we locate them?

dafydd
Télécharger la présentation

Introducing systematic reviews

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introducingsystematicreviews Karin HannesCentre forMethodology of Educational Research

  2. Outline • Intro - daily practice: evidence-based? • Systematic reviews • What is it and why do we need them? • How do they look like? • Who is producing them? • Where can we locate them? • What is there (potential) impact? • How can we contribute to them?

  3. Daily practice: Evidence-Based? • 1861: “I firmy believe that if the whole material medica as used now, could be sunk to the bottom of the sea, it would be all the better for mankind and the worse for the fishes.” (Holmes) • 1979: “Only 10 to 20 percent of all procedures currently used in medical practice have been shown to be efficacious by controlled trial...” (The US congressional office of technology assessment) • How many of our daily educational, criminological, social welfare interventions actually ARE evidence-based? • Let’s Raise our Hand for…

  4. Daily practice: Evidence-Based? 80 to 90 % basedon -Magic? -Intuïtion? -Expert opinion -???

  5. Daily practice: Evidence-Based? • “EBP is the integration of the best available research with clinical expertise in the context of patient characteristics, culture and preferences.” American Psychological Association.  (2006).  APA presidential task force on evidence based practice.  Washington, DC: Author. • We all want the best for our target group • We have always been evidence-based… • Our academic education guarantees a sufficient level of scientific knowledge, skills and attitudes Half of whatyoulearn in school willbeshown to beeitherdead wrong orout-of-datewithin 5 years of yourgraduation; the trouble is thatnobodycantellyouwhich half! (Dr. SydneyBurnwell)

  6. Daily practice: What is evidence? • Evidence of ‘effectiveness’: the extent to which an intervention, when used appropriately, achieves the intended effect. • Evidence of ‘feasibility’: the extent to which an intervention is practical and practicable, whether or not an intervention is physically, culturally or financially practical or possible within a given context. • Evidence of ‘appropriateness’ the extent to which an intervention fits with a situation, how an intervention relates to the context in which it is given. • Evidence of ‘meaningfulness’: the extent to which an intervention is positively experienced by the population and relates to the personal experience, opinions, values, beliefs and interpretations of the population.

  7. Part 1: SystematicreviewWhat is itand Why do we need THEM?

  8. What is it? A systematic review is a ‘systematic identification, evaluation and synthesis of all relevant studies regarding a specific topic, based on an explicit and pre-defined methodology’ If high quality research studies exist... Summarize them!

  9. What is it? Predefinedformat: To help reviewauthorsbesystematic To help people reading the reviews to findinformationquickly Transparancy Reliability

  10. Explosive growth. MEDLINE: Meta-analysis (Publication Type) OR “systematic review”

  11. Why do we need them? “Over two million articles are published annually in the biomedical literature in over 20,000 journals – literally a small mountain of information … a stack … would rise 500 metres” Cynthia Mulrow, in Systematic Reviews (BMJ Publishing Group, 1995) And withinthatstack, we wouldbelookingfor a needle… Reduce “bias” (garbage in-garbage out) Reduce “random error” • Provide reliable basis for making decisions • Inform and influence future research

  12. Lack rigor • Methodology not transparent • Different reviewers reach different • conclusions • Become out of date Traditional versus systematic review • Scientific rigor to minimise bias • Explicit and reproducible methodology • Regularly updated (Cochrane/Campbell)

  13. Key characteristics of a SR • A clearly stated set of objectives with pre-defined eligibility criteria for studies • An explicit, reproducible methodology

  14. Key characteristics of a SR • A clearly stated set of objectives with pre-defined eligibility criteria for studies • An explicit, reproducible methodology • A well defined, systematic search that attempts to identify all studies that would meet the eligibility criteria

  15. Key characteristics of a SR • A clearly stated set of objectives with pre-defined eligibility criteria for studies • An explicit, reproducible methodology • A systematic search that attempts to identify all studies that would meet the eligibility criteria • An assessment of the methodological quality of the findings of the included studies (assessment of risk of bias)

  16. Key characteristics of a SR • A clearly stated set of objectives with pre-defined eligibility criteria for studies • An explicit, reproducible methodology • A well defined, systematic search that attempts to identify all studies that would meet the eligibility criteria • An assessment of the methodological quality of the findings of the included studies (assessment of risk of bias ) • A systematic extraction, synthesis, and presentation of the characteristics and findings of the included studies

  17. Why do we need them? • Answers a specific healthcare question • Aims to identify and include all relevant trials • Uses pre-planned methods and strategies to limit bias and random error • May includeMETA-ANALYSIS: statistical synthesis of the results of included studies • Helps practitioners and patients make decisions about appropriate health care. • Provide a scientific rather than subjective summarisation of literature • Can reveal “new” evidence • Help decision makers to cope with the sheer volume of literature by summarising it • More reliable evidence with which to aid decision making

  18. Part 2: systematicreviewshow do they look like?

  19. What is a meta-analysis? Optional part of a systematic review • To identify a common effect among a set of studies • To improve precision of an estimate • To investigate whether the effect is constant • To answer controversies arising from conflicting studies or to generate new hypotheses Systematicreviews Meta-analyses

  20. Intervention Comparison no effect Meta-analysis Skills training versus usual curricula in the prevention of drug use in school kids. Gives a more precise estimate of effect than when derived from the individual studies included within a review 1. Summarystatistic + CI foreachstudyindividually (RR, OR, RD…) 2. Pooledintervention effect + CI is calculated as a “weighted average”

  21. Meta-analysis Skills training versus usual curricula in the prevention of drug use in school kids. Facilitates investigations of the consistency of evidence across studies, and the exploration of differences across studies. Intervention Standard no effect

  22. CC & C2: meta-analyse ‘Skill-based interventions’ prove to be an effective strategy to prevent from marihuana use in schoolkids.

  23. Part 3: systematicreviewsWho is producingthem?

  24. Astronomy, 17th century. “Combinations of data might be better than attempts to choose amongst them” Karl Pearson, 1904. “Many of the groups are far too small to allow of any definite opinion being formed at all, having regard to the probable error involved” Glass, 1976. “Primary, secondary, and meta-analysis of research”

  25. “It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials.” (1979) Archie Cochrane’s challenge

  26. Cochrane Collaboration: 1993 an international not-for-profit organisation which aims to help people make well-informed decisions about healthcare by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of health careinterventions

  27. Steering Group: Policy making body of the Cochrane Collaboration. • Elected representatives of each of the Collaboration entities Centres: Help organise and register review groups Facilitate collaboration among reviewers Provide training and consultation Establish liaisons Promote the Cochrane Collaboration Provide unique contribution Cochrane Collaboration • Review Groups: Focus on Treatment of disease or health problemsTo prepare and maintain systematic reviews • To develop a specialised trials register Collaborative Review Groups Centres Steering Group Methods Groups: Develop methods and products integral to internal functioning of the CollaborationDevelop state of the art methods for systematic review Examples: Statistical, Economics, Placebo Effects, Informatics, Qualitative Methods Groups Fields Fields: Represent a population, group, or type of care that overlaps multiple Review Group area.Examples: Primary Health Care, Health Care of Older Adults, Complementary Medicine The Consumer Network Consumer Network:Provides consumer inputHelps set priorities Helps with dissemination

  28. Campbell Collaboration (C2): 2000 an international organisation which aims to help people make well-informed decisions about public policy (crime & justice, education, social welfare) by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of socialand behaviouralinterventions

  29. Steering group: strategic and policymaking body Coordinating Groups Partners Steering Group Methods Groups Users Group Campbell Collaboration Coordinating Groups:Provide editorial services and support to authors of Campbell reviews and build links with users of systematic reviews: Crime and Justice Social Welfare Education Partners: Instituteswho have a formalagreementwith Campbell and support the same goals. Methods Groups: Improving the methodology of research synthesis, and disseminating guidelines for state-of-the-art review methods Usersgroup: Knowledgetranslation to increase the impact of Campbell reviews in policy and practice arenas, and to make the information more accessible

  30. CC & C2 Collaboration goals • To ensure high quality, up-to-date systematic reviews are available across a broad range of topics • To promote access to systematic reviews • To develop an efficient, transparent organisational structure and management system for the Collaboration • To achieve sustainability of the Collaboration

  31. Principles • Collaboration • Building on the enthusiasm of individuals • Avoiding duplication • Minimising bias • Keeping up to date • Striving for relevance • Promoting access • Ensuring quality • Continuity

  32. Part 4: systematicreviews: wherecan we locatethem?

  33. The Cochrane Library • Database of Systematic Review • Database of Abstracts of Reviews of Effectiveness • Controlled Trials Register • Database of Methodology Reviews • Methodology Register • Health Technology Assessment Database • NHS Economic Evaluation Database

  34. Still a lot of work to do!

  35. Part 5: systematicreviewsWhat is their (potential) impact?

  36. Evaluate a solution • Situation: 1 mj. people die each year due to traffic accidents • Solution: Educate the drivers • Systematic Review: 24 studies on education, no evidence that it reduces traffic accidents • Recommendations: Stimulate teleconferences and the use of safe, public transport “Driver education is big business – our results show that it is also a big con” (Prof. Roberts)

  37. Identify gaps in ‘evidence’ • Situation: anti-psychotics are subscribed for pregnant women with a psychotic disease • Systematic Review: No literature on adverse effects of anti-psychotics on the mother or the development of the (unborn) baby • RCT’s with pregnant women are considered unethical • Medicals’ referals are based on habits and opinions • Recommendation: Research on the effects of anti-psychotics in pregnant women is necessary “The continued use of antipsychotic drugs in women during pregnancy and lactation without sound evidence raises serious clinical and ethical concerns.” (Webb)

  38. Identify solutions that cause harm • Situation: Illinois law: Chicago Public Schools mandated to identify children at-risk for future criminal behaviour. • Solution: Scaired straigth programs: take them on tours of adult prison facilities • Systematic Review: Not only does a scared straight program fail to deter crime, it actually leads to more offending behaviour. “Governments need to adopt rigorous evaluation to ensure that they are not causing more harm to the citizens they intend to protect” (Mr. Petrosino).

  39. Authors of SR can make a difference! Anyone who wants to make a contribution... • www.cochranecollaboration.org • www.campbellcollaboration.org

  40. Select a topic Make a review team Register your title Part 6: systematicreviewshowcan we contribute to them?

  41. 1. Select a topic • Systematic reviews: • Commissioned • Invited • Unsolicited • Motivation to undertake a review: • Resolve conflicting evidence • Address questions when clinical practice is uncertain • Explore variations in practice • Confirm appropriateness of current practice • Highlight need for future research

  42. 1. Select a topic • Learn to ask questions (Treatment or Intervention): • P population • I intervention • C comparison • O outcome Does providinginformation(I) at school have a preventive effect on the use of drugs (O) in high school students(P)compared to sanctionswhensomeone is caughtusing drugs (C) ?

  43. 1. Select a topic “Information campagnes in the prevention of drug use” “Does informationprovision at school decreases the risk of drug use in high school students (compared to the standardapproach)? Scope of the question?

  44. 1. Select a topic • May start with a broad scope, and be divided up into narrower reviews as evidence accumulates • Cochrane “Overviews of reviews” (“umbrella reviews”) “Refinement of the review question” • Bias (“data-driven” questions) • Should be documented (“Differences between protocol and review”) • Sensitivity analysis to assess impact of changes

  45. 2. The review team Why more than 1 person? • Detection of errors (selection of eligible studies, data extraction) • Reduces risk of bias • When more than one person / team is interested in topic Who should be in the team? • Expertise in topic area • Expertise in systematic review methodology • Incorporate view of “users” (clinicians, consumers) • Address questions that are important to people • Take account of outcomes that are important to those affected • Make it accessible to people making decisions • Reflects variability in populations, settings…

  46. 3. Registering a title • Contact a Cochrane Review Group (CRG) or Campbell Coordinating Group (CCG) • Cochrane Review Group: • > 52 • Specific condition (eg incontinence, learning & developmental disorder group) • Group of diseases (eg ear nose and throat disorders group) • Organ (eg renal group, heart group) • Patient (eg neonatal group, pregnancy and childbirth group, pain and palliative care) • Campbell Coordinating Group: • 3 • Crime & Justice CG • Education CG • Social welfare CG There is a potential to co-registerreviews!

More Related