Narrative Synthesis in Systematic Reviews

1. Narrative Synthesis in Systematic Reviews An ESRC Research Methods Programme project Mark Rodgers Lisa Arai, Nicky Britten, Mark Petticrew, Jennie Popay, Helen Roberts, Amanda Sowden

2. Structure of the Presentation • What is “narrative synthesis”? • Where does narrative synthesis (NS) fit in a systematic review? • Why do we need guidance? • Developing a framework for NS • Building on the framework

3. What Is a “Narrative Synthesis”? • NS are findings summarised and explained in words • As used here, NS refers to the approach adopted to bringing together the findings from studies included in a systematic review • No clear definition of NS • “Not a meta-analysis” • Can be alternative or complementary to MA

4. Where does it fit? ‘Typical’ systematic review process: • Define review question • Literature search • Study selection • Data extraction • Study appraisal • Synthesis • Conclusions/recommendations

5. Why Do We Need Guidance? • NS lacks transparency • NS lacks reproducibility • Variations in practice • No coherent guidance currently exists

6. Developing guidance for NS • Difficult (impossible?) to produce prescriptive guidance on the conduct of NS • Most appropriate approach depends on context • A NS “toolkit” may be better than traditional “guidance” • Requires a structure

7. A framework for NS • An descriptive framework of the NS process: • Developing a theory • Developing a preliminary synthesis • Exploring relationships in the data • Assessing the robustness of the synthesis

8. Developing a theory • “Theory of change” linking resources, activities, intermediate outcomes and ultimate goals • How the intervention works, why and for whom • Has implications for review question and inclusion criteria as well as interpretation • Consider early in the review process • Can be presented in narrative and/or diagrammatic form

9. Developing a preliminary synthesis • Initial description of the results of included studies • Organise results to describe patterns, e.g. in: • size/direction of effects • factors/processes impacting on implementation • quality • Product of this process is preliminary and should be interrogated further

10. Exploring relationships in the data • Explore relationships: • Between study characteristics and their findings • Between the findings of different studies • Explore influence of heterogeneity: • Variations in outcomes, methods, populations, interventions, settings etc • Theory may help develop plausible explanations for observed differences • Investigation of context partly dependent upon reporting of included studies

11. Assessing the robustness of the synthesis • Robustness of NS depends on two related elements: • Methodological quality of included studies • ‘Trustworthiness’ of the synthesis product • Availability of key information in primary studies • Overall assessment of the strength of evidence available to support conclusions

12. Building on the framework (1) • Populated framework with range of “tools and techniques” (e.g. tabulation, idea webbing/conceptual mapping, transforming and translating data) • Allow reviewer to select tools/techniques most appropriate to the data being synthesised • Not necessarily linear or sequential process – reviewers will move iteratively between elements

13. Building on the framework (2) • Conducted two ‘test’ NS of domestic smoke alarm promotion interventions: one effects, one implementation • Qualified success - the approach can be further developed and refined • Would benefit from application to a wider range of syntheses • http://www.lancs.ac.uk/fass/projects/nssr/