1 / 13

Enhancing Systematic Review Approaches in Complex Research Areas

This exploration into systematic reviews highlights the challenges and strategies for searching in complex areas of health sciences. Dixon-Woods and Booth discuss the limitations of traditional search methods, particularly for qualitative research, emphasizing the need for innovative and flexible approaches. Key themes include the importance of iterative searching, adaptability in question formulation, and the distinction between exhaustive and rigorous search practices. The work advocates for accepting the non-linear nature of research discovery while maintaining methodological transparency to enhance trust in systematic reviews.

glenna
Télécharger la présentation

Enhancing Systematic Review Approaches in Complex Research Areas

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Searching for studies in complex areas Mary Dixon-Woods Department of Health Sciences University of Leicester Andrew Booth, University of Sheffield

  2. A “typical” definition of an SR • "A systematic review is a review in which there is a comprehensive search for relevant studies on a specific topic, and those identified are then appraised and synthesised according to a pre-determined explicit method“ Klassen, Jadad, & Moher, 1998

  3. Technical problems of searching • Effort invested in indexing systems and filters on electronic databases for quantitative research • Qualitative research still very elusive; abstracting is a problem (Shaw et al, 2004) • But searching isn’t just a technical difficulty

  4. Greenhalgh, 2005 Protocol driven: 150 (30)Electronic database search* 126 (25) Hand search (32 journals) 24 (5) "Snowballing": 252 (51)Reference tracking 218 (44) Citation tracking 34 (7) Personal knowledge: 119 (24) Sources known to research team 85 (17) Social networks of research team 29 (6) Serendipitous 5 (1) Total in final report 495

  5. CIS approach • Fixed, protocol-driven approach to searching has advantages and limitations • Emphasises more creative, organic, intuitive forms of searching • Accepts that exhaustiveness might not be possible or appropriate • Recognises that it may not always be possible to be “transparent”

  6. Treat the question as “a compass rather than an anchor” –Eakin & Mykhalovskiy (2003) Not “Big Bang” searching Iterative “Spiral” Searching Subjecting a question to continual review and refinement

  7. This requires: • Purposively searching for new insights (new databases, new disciplines, new search methods) – active sampling of the literature • “Lateral leaps” rather than “more of the same” • Actively seeking the “disconfirming case” rather than the “averaging effect” • Citation searching within divergent disciplines/schools of thought • Challenge: • “Sampling research is fundamentally not like sampling people. Unlike people, research papers have a vested interest in being different from one another, and are (in theory at least) only published if they are saying something new.”. Dixon-Woods, Bonas, Booth et al, 2006

  8. Searching not reproducible, not necessarily even transparent • More we learn about SR methods the more we recognise that reproducible searching is a myth • Different questions, search strategies, selection of abstracts, inclusion criteria, selection of articles, methods of synthesis, level of analysis, implications…..and on, and on, and on! • Replace with criterion of “reproducible” with “rigorous” • Reporting of methods is not for reproducibility but to increase our “trust” of the research

  9. Assessing comprehensiveness • The main purpose of a CIS is “interpretative” not “aggregative” • Sensitivity to evolving question/problem definition • Therefore the “point of termination” is not when you have identified ALL studies • It is when you no longer identify NEW insights (analogous to theoretical saturation) • Requires iterative, sampling-based approach

  10. A different view of systematic • “‘Systematic’ is, by definition, ‘a system, plan, or organized method’ (Oxford English Dictionary [OED]), ‘methodical in procedure or plan’ (Merriam-Webster). If a systematic review is then….about method itself, then qualitative methods include valid principles and procedures with which to progress in reviewing literature ‘systematically’”. [Jones, 2004] • “A wide range of qualitative ‘methodical’ approaches are, therefore, available to perform or carry out a review in a particular field of interest—‘systematically and meticulously’ (OED)”. [Jones, 2004]

  11. Systematic and Explicit • Simply means that we follow an (appropriate, but not specific) system and that we communicate what we have done • These are not markers of a systematic review – they are characteristics of ALL well-conducted, well-reported research

  12. References - 1 Eakin, J & Mykhalovskiy, E (2003). Reframing the evaluation of qualitative research: Reflections on a review of assessment guidelines in the health sciences. Journal of Evaluation in Clinical Practice, 9(2), 187-194. Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ. 2005 Nov 5;331(7524):1064-5. Epub 2005 Oct 17. Klassen TP, Jadad AR, Moher D. Guides for reading and interpreting systematic reviews. I. Getting started. Arch Pediatr Adolesc Med 1998;152:700-704.

  13. References - 2 Booth A (In press) “Brimful of STARLITE”: Towards standards for literature searches. JMLA (October) Dixon-Woods M, Bonas S, Booth A et al. How can systematic reviews incorporate qualitative research? A critical perspective. Qualitative Research, 6 (1), 27-44 (2006) Dixon-Woods M, Booth A, Jones D, et al. How can systematic reviews incorporate qualitative research? ESRC Methods Briefing No. 6 http://www.ccsr.ac.uk/methods/publications/documents/dixonwoods.pdf Jones K. Mission Drift In Qualitative Research, Or Moving Toward A Systematic Review Of Qualitative Studies, Moving Back To A More Systematic Narrative Review. The Qualitative report, 9(1), 95 - 112.http://www.nova.edu/ssss/QR/QR9-1/jones.pdf Shaw RL, Booth A, Sutton AJ, Miller T, Smith JA, Young B, Jones DR, Dixon-Woods M (2004). Finding qualitative research: an evaluation of search strategies. BMC Research Methodology 4:5 (16 Mar).

More Related