1 / 29

Evaluating the Impact of Journal Prestige on Research Citations: An Analysis of Publication Practices

This paper by Adam Eyre-Walker explores the critical role of publication venues in determining citation impact within the scientific community. It delves into the correlation between the quality of research papers and their citations, focusing on the influence of journal prestige. By analyzing data from the Research Assessment Exercise (RAE) and employing various metrics such as journal impact factors and RAE scores, the study provides insights into how publication strategies affect scholarly recognition and the dissemination of scientific knowledge.

sally
Télécharger la présentation

Evaluating the Impact of Journal Prestige on Research Citations: An Analysis of Publication Practices

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Does It Matter Where We Publish? Adam Eyre-Walker University of Sussex

  2. Reasons • Prestige • peers • employers • Impact • scientific community • society

  3. Measuring impact • General • difficult to measure • Citations • Easy to measure • Velvet Underground effect • “only sold 10,000 copies, but everyone who bought it formed a band” (Brian Eno)

  4. Question Do papers of similar quality garner more citations in better quality journals?

  5. Ideal experiment • Papers made anonymous • authors • journal • structure • Assessed for quality by experts • Assessments confidential • Citations tracked

  6. Research Assessment Exercise (RAE) • Each department submits • 4 publications per academic (75%) • 4 indicators of Esteem (5%) • Environment • PhD student numbers • Grant income • Research structure • Facilities • Superceded by REF

  7. RAE dataset • 1170 papers published 2001 - 2007 • Scored by AEW • Scored 1 to 4 (or unclassified) • Subjects : evolutionary biology, genomics, bioinformatics, ecology, animal behaviour and organismal biology • Ranked without reference to • journal • citations

  8. RAE scores

  9. Journal quality • Impact factor • Number times papers published in the two previous years are cited in target year • Two components • Quality of papers • Effect of the journal on citations

  10. IF categories

  11. Citations • No. of citations from ISI Web of Science • October 2008

  12. Data • For each paper • Year of publication • RAE score (RS) • No. of citations • Impact factor of journal (IF category) • Otherwise anonymous

  13. How good is the assessor? • Correlation between AEW and co-scorers > 0.7 • Correlation between RS and no. of citations within each IF category and within each year

  14. How good is the assessor? • Correlation between AEW and co-scorers > 0.7 • Correlation between RS and no. of citations within each IF category and within each year • Average r = 0.27 (p<0.001) • Average rmax = 0.73

  15. RS v citation correlation r = -0.37 p = 0.005

  16. Central question • Do papers of similar quality accumulate more citations in some journals than others?

  17. RAE score 2 in 2003

  18. RAE score 2 p < 0.001

  19. RAE score 3 p < 0.001

  20. ANCOVA

  21. ANCOVA

  22. Alternative explanation • Systematically underestimate quality of papers in high ranking journals • Test • Assume no effect of journal • Consider correlation between RS and journal (IF) controlling for citations

  23. Partial correlations

  24. ANCOVA

  25. Question • Impact factors – relative contribution of • Paper quality • Journal effect

  26. Summary • Assessor scores correlated to citations • strength correlated to age • Assessor scores influenced by journal • Papers published in higher quality journals accumulate more citations independent of quality • F1000 data

  27. Question • Where will we submit?

  28. Thanks Nina Stoletzki

  29. Assessor and journal • Assume • citations measure quality • no journal effect

More Related