SHOULD SYSTEMATIC REVIEWERS SEARCH FOR RANDOMIZED CONTROLLED TRIALS PUBLISHED AS LETTERS?

1. SHOULD SYSTEMATIC REVIEWERS SEARCH FOR RANDOMIZED CONTROLLED TRIALS PUBLISHED AS LETTERS? Alla E. Iansavichene1,2,3, Margaret J. Sampson1,4, Jessie McGowan4,5, I.S.Y. Ajiferuke2 1Chalmers Research Group, Children’s Hospital of Eastern Ontario Research Institute, Ottawa, Ontario; 2FIMS, University of Western Ontario, London, Ontario; 3 London Health Sciences Centre, London, Ontario; 4 Department of Information Studies, University of Wales, Aberystwyth, UK; 5Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada Table 1. Characteristics of RCT letters (n=157) ABSTRACT Objective A systematic review search strategy often excludes certain publication types, such as letters and editorials. This was thought to improve the search precision and reduce reviewer burden by eliminating irrelevant citations. Our objective was to determine if randomized controlled trials (RCTs) would be missed by excluding letters from the search strategy. Study Design and Setting The MEDLINE database was searched for citations with publication type Randomized Controlled Trial (RCT.pt) and the publication type Letter, but not the publication type Comment. A random sample of 200 citations matching the criteria was selected and evaluated to confirm their study design. Results RCTs were found to be published as letters, and prior to 1998, they appeared disproportionately in journals in MEDLINE’s Core Clinical Journal set (formerly Abridged Index Medicus). Discussion A report by Deeks and Altman, issued in 1998 found RCTs published as letters to be incompletely reported and recommended against their publication (Deeks & Altman, 1998). While some high impact journals seem to have changed their editorial practices, RCTs are published as letters to this day. The partial uptake shown in this report is an example of incomplete knowledge translation. Many systematic reviewers may assume that this practice stopped, and inadvertently miss potentially eligible reports by excluding letters from the search. Conclusions Searchers seeking complete identification of RCTs should not exclude all letters, but rather use the Boolean operator “not” (letter.pt. not randomized controlled trial.pt.) when excluding letters. *NR stands for Not Reported Table 2. Reporting of harms in letters determined as RCTs RECOMMENDATIONS • In establishing any evidence base, study design should be given precedence over publication format. • Reviewers undertaking systematic reviews of randomized controlled trials need to use the logical construction: NOT (letter.pt. NOT randomized controlled trial.pt.) • Hopewell et al (2006) have suggested a modified version of CONSORT to optimize reporting of conference abstracts. Such a standard could usefully apply to RCTs published as letters, and when space constraints prevent adequate reporting, authors should consider publishing results as a full article instead. CONCLUSIONS Table 3. Frequency of citation since publication for letters determined as RCTs and non-RCTs (n=141) * Figure 2. Flow diagram of results of search strategy of RCT letters in OVID MEDLINE database and breakdown of letters that met the inclusion criteria. • The reports of RCTs in letter format are still being published after CONSORT guidelines were issued in late 90s. While some high impact journals seem to have changed their editorial practices, RCTs are being published as letters to this day. • RCTs published as letters appear disproportionately in general and internal medicine journals. Much of this effect is driven by Lancet, although other high impact factor journals are also quite common. Letters tagged as RCTs are more often published in Core Clinical Journals (AIM), particularly letters which were determined as RCTs. • The process of translating existing knowledge into clinical practice by publishing RCTs in letter format is inadequate, as details of the reporting data in this format such as the population, intervention, comparators and outcomes restrict complete knowledge dissemination and utilization (Deeks & Altman, 1998). The partial uptake shown in this study is an example of incomplete knowledge translation. • Since not all results of randomized controlled trials are published as full-text articles, some are published only in short report form such as conference abstracts or letters, a better reporting standard similar to CONSORT guidelines is needed for authors intending to report RCT results in letter form. *Data represents the letters that were indexed in the ISI (n=141). ** AIM (The Abridged Index Medicus) represents a search limit called “Core Clinical Journals” available in PubMed and OVID’s MEDLINE databases. *** Inter-quartile range (IQR): (25th percentile – 75th percentile) Acknowledgements: We thank J. Sholts for obtaining articles, N.J. Barrowman for generating the random sample, D. Moher for the insight regarding Lancet letters. References: • Deeks JJ, Altman DG. Inadequate reporting of controlled trials as short reports. Lancet 1998; 352(9144):1908. • Hopewell S, Clarke M, Askie L. Reporting of trials presented in conference abstracts needs to be improved. J Clin Epidemiol. 2006; 59(7):681-4.