Zoe G. Davies Centre for Evidence-Based Conservation University of Birmingham, UK

1. Zoe G. DaviesCentre for Evidence-Based ConservationUniversity of Birmingham, UK Systematic Review Protocol Development

2. Overview • What is a protocol? • What is the value of a protocol? • Question formulation • Components of a protocol

3. What is a Protocol? • The starting point • Explicitly states the methodology to be followed during the systematic review process • Available for peer-review by subject experts

4. What is the value of a protocol? • Formalises the question under review • Helps to avoid bias • Ensures transparency

5. Question: formulation • Define the systematic review question • Key elements: • Subject (i.e., unit of study to which the intervention is to be applied) • Intervention (i.e., policy or management intervention under scrutiny)

6. Question: formulation • Outcome (i.e., any measured outcome that can be used to judge the effectiveness of the intervention) • Comparators (e.g., intervention vs. no intervention or intervention y vs. intervention z)

7. Question: making it relevant • Dependent on the purpose of the systematic review • Management, policy or research driven question • Consultation with interested stakeholders, end-users and subject experts • Striking the balance • Not too broad, not too specific

8. Question: secondary objectives • Identify factors that may influence the outcome of studies • Also referred to as: • Reasons for heterogeneity • Effect modifiers (meta-analytical terminology)

9. Case Study: hedgerow corridors • Evaluate the effectiveness of habitat corridors in promoting population viability of target species and biodiversity within fragments of remnant habitat • 20 years of debate within the ecological literature

10. Case Study: hedgerow corridors • Do hedgerows mitigate woodland habitat fragmentation? • Do hedgerows increase the population viability of target species occupying otherwise isolated fragments of woodland habitat?

11. Case Study: hedgerow corridors • Key question elements: • Subject: mammal, bird, invertebrate or amphibian populations or assemblages • Intervention: a hedgerow, or hedgerow network, connecting two or more woodland habitat fragments

12. Case Study: hedgerow corridors • Outcome: desired primary outcomes were change in population density for a target species or change in species richness within assemblages • Comparator: No comparator was necessary for inclusion (although appropriate spatial or temporal controls were a prerequisite for studies to be included in any subsequent meta-analysis)

13. Case Study: hedgerow corridors • Reasons for heterogeneity: • Physical structure of the hedgerow • Vegetation composition of the hedgerow • Nature of the non-habitat matrix • Life history stage of the target species (e.g., dispersing juveniles)

14. Protocol: background • Rational behind the systematic review • Put the review question into context • Woodland fragmentation • Habitat connectivity • Hedgerow ecology

15. Protocol: literature search strategy • State how and which information sources will be searched • Key words: • Reflect the population, intervention and outcome • Consider synonyms, alternative spellings and abbreviations (e.g., colonise and colonize) • Foreign language translations

16. Protocol: literature search strategy • Hedgerow* AND corridor* • Hedgerow* AND movement* • Hedgerow* AND dispersal • Hedgerow* AND colonisation • Hedgerow* AND colonization • Hedgerow* AND connectivity • Hedgerow* AND population* • Hedgerow* AND communit*

17. Protocol: literature search strategy • Combinations and permutations • Trade off between effort and return • Sensitivity vs. specificity • Scope searches and refine • Search generic and specific information sources

18. Protocol: literature search strategy • Online databases and libraries • Scientific literature databases (e.g., WOK and JSTOR) • Statutory and non-governmental organisation websites (e.g., Defra, EN and RSPB) • Internet search engines (e.g., Dogpile and Google Scholar) • Specialists in the field • Bibliographies • Hand searching libraries and museums

19. Protocol: study inclusion criteria • Based on the key elements of the question • State the filtering process to be used • Title • Abstract (Assessed by a second review and test for agreement) • Full text (Assessed by a second review and test for agreement)

20. Protocol: study quality assessment • Hierarchy of evidence • RCT’s • Control trials without randomisation • Site comparisons • Time series data • Used to determine study quality thresholds for included articles • Dependent on the purpose of the review

21. Protocol: data extraction • What data needs to be extracted from the accepted studies? • Pilot data extraction forms • Contact authors or organisations for retrieval of missing data • Database or spreadsheet of all information relevant to the review

22. Protocol: data synthesis • How are the studies to be pooled? • How will differences in the studies be taken into account? • Propose analyses to be undertaken • May not be possible to be specific at the protocol stage

23. Protocol: timescale • Set out key milestones (e.g., completion of searching, study selection, etc.) • Possible delays (e.g., consultation periods, inter library loans, etc.) • Some stages may overlap

24. Protocol: modifications • Amendments to the methodology (e.g., if there are no studies which meant the inclusion criteria) • Modifications must be documented and justified • Maintain transparency • Allow independent parties to judge review validity

25. Further Information • Available from our website: • www.cebc.bham.ac.uk • Medical systematic review centres: • The Cochrane Collaboration (www.cochrane-net.org) • NHS Centre for Reviews and Dissemination (www.york.ac.uk/inst/crd)