Understanding Literature Reviews: Methodologies and Significance

1. Getting Started: What’s Out inthe Literature and Is It Any Good? Susan B. Fowler, PhD, RN, CNRN, FAHA

2. Literature

3. Research Critique • Is not synonymous with “criticize” • Examines the strengths, weaknesses, meaning, and significance of the study (substantive and theoretical dimensions) • Be objective and realistic in identifying the study’s strengths and weaknesses

4. Important Points • All studies have weaknesses or flaws • Research is critiqued to broaden understanding, improve practice, and provide background for conducting a study • The critique process involves comprehension, comparison, analysis, and evaluation

5. Cochrane Systematic Reviews • Cochrane Librarywww3.interscience.wiley.com/cgi-bin/mrwhome/106568753/HOMEThe Cochrane Collaboration. Oxford: Update Software; 1996-. Updated quarterly. • Cochrane Database of Systematic Reviews (Cochrane Reviews) • 'Gold Standard' for high-quality systematic reviews • Full-text included in Cochrane Library • Cochrane Reviews includes complete reviews and protocols (reviews that are still in progress) • Cochrane Reviews abstracts are in PubMed

6. More Reviews • Database of Abstracts of Reviews of Effects (Other Reviews - DARE) • prepared by the National Health Service Centre for Reviews and Dissemination, University of York, England • Complements the Cochrane Reviews by offering a selection of quality assessed reviews in those subjects where there is currently no Cochrane review • Brief critical appraisals of previously published reviews of the effects of health care • Structured abstracts, not full-text • DARE not indexed in PubMed, but original research articles may be • Also available at no charge on the web from University of York | www.crd.york.ac.uk/crdweb/

7. Other Systematic Reviews and Meta-analyses • PubMed MEDLINE - Systematic Reviews | www.ncbi.nlm.nih.gov/entrez/query/static/clinical.shtml • access: part of PubMed's Clinical Queries (linked on PubMed Home and Advanced Search pages) • Identifies systematic reviews and other similar types of studies found in biomedical journals included in PubMed database • No evaluation of comparative quality of different reviews on a topic • Abstracts supplied by authors/journals. No separate evaluation of quality of research • Identify systematic reviews in the biomedical area - gathers together much larger collection than other evidence-based practice resources

8. Evidence Guidelines/Summaries • BMJ Clinical Evidence | www.clinicalevidence.com/ • Compendium of evidence on the effects of clinical interventions • Summarizes the current state of knowledge, including knowns and unknowns, based on thorough search • Categorizes interventions as beneficial, likely beneficial, no known benefit, harmful ... • DynaMed | www.ebscohost.com/dynamed • Best available evidence summaries for nearly 2000 topics • Clinical reference tool developed for use at the 'point-of-care' • Outline format to quickly identify key conclusions • Updated daily • Other Resources • USPSTF Guidelines | www.ahrq.gov/clinic/prevenix.htm • AHRQ Evidence Reports | www.ahrq.gov/clinic/epcix.htm • FPIN Clinical Inquiries included in Journal of Family Practice | www.jfponline.com and American Family Physician | www.aafp.org/afp/

9. RCTs, Case Cohorts, Control Studies • PubMed | (pubmed.gov) • note: Filtered by publication type, but not quality • Clinical Queries - Uses preconfigured search strategies to retrieve research-based citations on clinical topics in the areas of therapy, diagnosis, etiology, or diagnosis • Use Limits to search by specific publication type (e.g. meta-analysis, randomized controlled trial, review) • Cochrane Library | www3.interscience.wiley.com/cgi-bin/mrwhome/106568753/HOMEThe Cochrane Collaboration. Oxford: Update Software; 1996-. Updated quarterly. • Cochrane Central Register of Controlled Trials (Clinical Trials) • Bibliography of controlled trials as part of an international effort to hand search the world's journals • Includes reports published in conference proceedings and in many other sources not in PubMed or other bibliographic databases

10. Content of Research Reports • The Abstract • The Introduction (review of the literature, theoretical framework, significance of and need for the study) • The Methods Section • The Results Section • The Discussion Section • The References

11. The Beginning • The title • Could you tell what the article was about by reading the title? • The abstract • Includes a brief description of the problem • How the study was done (methodology) • The results • Conclusions

12. Next Steps • The Problem • Is it clear what the problem the authors are trying to solve? • Review of the Literature • Do the articles relate to the problem? • Is it organized, broad to specific? • Does it tell a story? • Comprehensive and current? What years are covered in the ROL? • Research Question/Aim/Purpose/Hypothesis • Do authors pose a research question or hypothesis? • Is it explicitly stated what the aim or purpose of the study is? • Design • Is the design stated? • Does the design flow from the research question or aim? • Did the authors explain why they choose this research design?

13. Design Quantitative Qualitative • Analysis of numerical data • Experimental • Quasi-experimental • Comparative • Longitudinal • Correlational • Analysis of data such as words (e.g., from interviews), pictures (e.g., video), or objects (e.g., an artifact) • Grounded theory - social processes • Phenomonology – lived experience • Ethnography – culture • Historical

14. The Journey Continues • Sample • Is the population described? • Is the sample method, or how the sample was chosen described? • Is the sample size right for the analysis? Was a power analysis done to determine sample size? • Were standards for protection of human subjects discussed? • Tool • Was the tool created by the authors or already established? • Is a copy of the tool included in the article? • Is reliability discussed? • Is validity discussed?

15. Tool, Instruments, Questionnaires, and More Validity Reliability • Extent to which the tool measures what it was intended to • Content • Criterion related • construct • Results are repeatable and consistent • Internal consistency • Test-retest • Inter-rater

16. Power • The probability that a statistical test will detect a significant difference that exists - the risk of a Type I error can be calculated using power analysis. • Level of significance • Sample size • Power - acceptable level is .80 • Effect size - the degree to which the null hypothesis is false

17. Effect Size • An estimate of how large the treatment effect is, that is how well the intervention worked in the experimental group compared to the control group (intervention studies) • The larger the effect size, the stronger are the experimental intervention’s effects. • Effect size for intervention studies: • .2 = small effect • .5 = medium effect • .8 = large effect

18. Getting Closer to the End • Methodology • Are methods of data collection sufficiently described? • Is the time frame when the study occurred described? • Data Analysis • Is information presented sufficient to answer the research question(s)? • Were statistical tests used to analyze the data? • Were values obtained from the analysis? • Was statistical significance reported? • Are the results explained? • Are tables and figures easy to understand and informative?

19. Confidence Intervals • Probability that a value will fall within a range of variables • The larger the CI the less precise the measurement of that variable • A very wide interval may indicates that more data should be collected before anything very definite can be said about the parameter • 95% CI most common

20. The End • Discussion • Is a discussion section presented? • If yes, are the results compared with the literature review? • Conclusions • Are conclusions clearly stated? • Are conclusions directly related to the results? • Do the findings add to the present nursing knowledge? • Are study limitations identified? • Did authors make recommendations for further research? • Carlson, J. (1999). J Emerg Nurs, 25, 330-332.

21. References • Relevance (extent to which the reference bears on the research question) • Primary sources (descriptions of studies written by the researchers) • Secondary sources (descriptions of studies written by someone other than the original researcher) • Opinion and anecdotal

22. Level of Significance • The researcher does not know when an error in statistical decision making has occurred. The researcher can control the risk of making a Type I Error by setting the level of significance. • Level of Significance (alpha level) is the probability of making a Type I Error or the probability of rejecting a true null hypothesis.

23. Level of Significance • Usual level set at .05 (willing to accept the fact that if the study were carried out 100 times, the decision to reject the null hypothesis would be wrong 5 times) • If researcher wants to have smaller risk of falsely rejecting null hypothesis, set at .01. • Decreasing risk of making Type I Error increases risk of Type II Error. • Findings are significant or not significant.

24. Clinical Significance • Related to practical importance of the findings • No common agreement in nursing about how to judge clinical significance • Effect size? • Difference sufficiently important to warrant changing patient care?

25. Clinical Significance • Who should judge clinical significance? • The patients and their families? • The clinician or researcher? • Society at large? • Clinical significance is ultimately a value judgment

26. Research Journals • Examples of peer reviewed / refereed journals: Nursing Research, Advances in Nursing Science, Applied Nursing Research, Clinical Nursing Research, Western Journal of Nursing Research • Example of Peer Reviewed online resource: Medscape • Peer Review – Look for statement in first few pages of journal that it is peer reviewed. Content is “reviewed” by experts in field before publication

27. Let’s Get to Work – Critique in Action!!