Open Access: Validating Metrics and Motivating Mandates

1. Open Access:Validating Metrics and Motivating Mandates Stevan Harnad, UQAM & U. Southampton Alma Swan, Key Persectives & U Southampton Arthur Sale, U. Tasmania OAR2008

2. OA Timeline and Milestones • Origin of the Universe: 14 billion years • Origin of Life on Earth: 4 billion years • Origin of our Species: 200,000 years • Origin of Language: 100-200,000 years • Origin of Writing: 10,000 years • Origin of Printing: 500 years • Origin of Learned journals: 340 years • Origin of Internet: 40 years • Origin of Web: 18 years • Origin of OAI: 9 years OAR2008

3. What is OA? Free online access to refereed research articles OAR2008

4. Gold or Green? • OA journals: Gold OA • Self-archiving non-OA journal articles: Green OA OAR2008

5. Access to what? • Published peer-reviewed journal articles • Unrefereed preprints? • Monographs? • Data? OAR2008

6. Why OA? • Maximize research uptake, usage and impact • Direct benefit: Research progress • Side-Benefits: Developing world access, student access, public access OAR2008

7. How OA? • Self-archive in Institutional Repository • Institutions and Funders Mandate Self-Archiving OAR2008

8. New impact cycles:New research builds on existing research Impact cycle begins: Research is done Researchers write pre-refereeing “Pre-Print” 12-18 Months Submitted to Journal Pre-Print reviewed by Peer Experts – “Peer-Review” Pre-Print revised by article’s Authors Refereed “Post-Print” Accepted, Certified, Published by Journal Researchers can access the Post-Print if their university has a subscription to the Journal OAR2008

9. New impact cycles:New research builds on existing research Impact cycle begins: Research is done Researchers write pre-refereeing “Pre-Print” This limited subscription-based access can be supplemented by self-archiving the Postprint in the author’s own institutional repository as follows: 12-18 Months Submitted to Journal Pre-Print reviewed by Peer Experts – “Peer-Review” Pre-Print revised by article’s Authors Refereed “Post-Print” Accepted, Certified, Published by Journal Researchers can access the Post-Print if their university has a subscription to the Journal OAR2008

10. Post-Print is self-archived in University’s Eprint Archive More impact cycles: Impact cycle begins: Research is done Researchers write pre-refereeing “Pre-Print” Submitted to Journal 12-18 Months Pre-Print reviewed by Peer Experts – “Peer-Review” Pre-Print revised by article’s Authors Refereed “Post-Print” Accepted, Certified, Published by Journal Researchers can access the Post-Print if their university has a subscription to the Journal New impact cycles: New research builds on existing research OAR2008

11. What are “metrics”? • Metrics are objective measures of research quality and quantity • The only alternative to metrics is subjective human judgment (including peer review) “Show me a philosopher who wishes to discard metaphysics and I’ll show you a metaphysician with a rival system” (Show me someone who wishes to discard metrics, and I’ll show you a metrician with rival metrics) OAR2008

12. Open Access: How? By mandating Green OA Self-Archiving Brody et al (2007) Incentivizing the Open Access Research Web: Publication-, Data-Archiving and Scientometrics. CTWatch Quarterly 3(3). http://eprints.ecs.soton.ac.uk/14418/ OA Metrics motivate OA Mandates And OA Mandates maximize OA Metrics OAR2008

13. “Online or Invisible?” (Lawrence 2001) “average of 336% more citations to online articles compared to offline articles published in the same venue” Lawrence, S. (2001) Free online availability substantially increases a paper's impact Nature 411 (6837): 521. http://www.neci.nec.com/~lawrence/papers/online-nature01/ OAR2008

14. Lawrence (2001) findings for computer science conference papers. More OA every year for all citation levels; higher with higher citation levels OAR2008

15. OAR2008

16. Contributors to the OA AdvantageEA + QA + UA + (CA) + (QB) • EA: Early Advantage • QA: Quality Advantage (Seglen 80/20 effect) • UA: Usage Advantage • (CA: Competitive Advantage) • (QB: Quality Bias) OAR2008

17. Early Access Advantage: OA is accelerating the research access/usage/citation cycle. OA articles are being cited sooner and sooner (Data from Physics Arxiv) OAR2008

18. Quality Advantage: Higher quality articles have greater OA Advantage

19. Usage Advantage + Early Advantage: OA Articles are downloaded more and early downloads lead to later citations Data from arXiv Downloads (“hits”) in the first 6 months correlate with citations 2 years later Most articles are not cited at all Brody, T., Harnad, S. and Carr, L. (2006) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST) 57(8): 1060-1072. http://eprints.ecs.soton.ac.uk/10713/

20. Time-Course and cycle of Citations (red)and Usage (hits, green)Witten, Edward (1998) String Theory and Noncommutative Geometry Adv. Theor. Math. Phys. 2 : 253 Preprint or Postprint appears. 2. It is downloaded (and sometimes read). 3. Next, citations may follow (for more important papers)… 4. This generates more downloads… 5. More citations... OAR2008

21. (Competitive Advantage): The earlier you mandate Green OA, the sooner (and bigger) your university's competitive advantage: U. Southampton School of Electronics and Computer Science was the first in the world to adopt an OA self-archiving mandate. (Competitive Advantage vanishes at 100% OA.)

22. (Quality or Self-Selection Bias ): Better authors are more likely to self-archive, and better articles are more likely to be self-archived. (Michael Kurtz considers this in itself a sufficient rationale for self-archiving!)(Quality Bias vanishes at 100% OA.) The data below are systematically ambiguous, because they could arise from either Quality Advantage or Quality Bias OAR2008

23. Some have argued that the OA Advantage might be all or mostly Just Quality (Self-Selection) Bias. So we tested this, by comparing self-selected with mandated OA: OAR2008

24. Percentage of OA articles among the ISI-indexed articles, per institution and per year:

25. Ø: non-Open Access, O: Open Access, M: Mandated, N: non-Mandated Results: Five of the seven effects are statistically significant: O > Ø, OM > ON, (ON=ØN), OM > ØM, OM > Ø (ON=Ø), OM > ØN i.e., OA is more cited than non-OA, and mandated OA is cited more, not less than non-mandated OA. (The growth of the OA Advantage with more recent years may be because mandate compliance is more immediate, whereas for earlier years the deposit was made retroactively.)

26. Green OA Mandates 1: Alma Swan’s International, Multidisciplinary Survey Predictions About Researcher Compliance: OAR2008

27. OA Mandates: Across all countries and disciplines, 95% of researchers report that they would comply with a self-archiving mandate from their funders and/or employers, and over 80% report that they would do so willingly. -- But only 15% self-archive spontaneously, if it not mandated.

28. Green OA Mandates 2: Arthur Sale’s Australian Data on Actual Researcher Compliance: OAR2008

29. University of Tasmania +Repository-Incentive -Mandate Green line: total annual output Red line: proportion self-archived Data courtesy of Arthur Sale OAR2008

30. University of Queensland+Repository +Incentive-MandateGreen line: total annual outputRed line: proportion self-archived Data courtesy of Arthur Sale

31. Queensland University of Technology+Repository +Incentive +MandateGreen line: total annual outputRed line: proportion self-archived Data courtesy of Arthur Sale OAR2008

32. Unanimous Recommendation by EUA, Jan 25 2008 791 universities in 46 countries All European Universities should create institutional repositories and should mandate that all research publications must be deposited in them immediately upon publication (and made Open Access as soon as possible thereafter) as already mandated by RCUK, ERC, and NIH, and as recommended by EURAB. In addition, the EUA recommends that these (funder) self-archiving mandates should also be extended to all research results arising from EU research programme/project funding. OAR2008

33. NOW 54 & 10!

34. The majority of journals (63%) already endorse immediate Green Open Access Self-Archiving of the postprint ROMEO/EPRINTS (Directory of Journal Policies on author OA Self-Archiving):http://romeo.eprints.org/ NOW 63% & 95% OAR2008

35. For the articles in the 37% of journals that have an embargo policy, the free EPrints institutional Repository-creating software has an ”Email Eprint Request" Button: The user who reaches the metadata for a Closed Access article puts his email in a box and clicks. This sends an automatic email to the author, with a URL on which the author clicks to automatically email the eprint to the requester. OAR2008

36. Once the ID/OA mandates are universally adopted, the embargoes will soon become obsolete, under growing OA pressure worldwide. Carr & Harnad (2005)Keystroke Economy: A Study of the Time and Effort Involved in Self-Archiving. http://eprints.ecs.soton.ac.uk/10688/ OAR2008

37. The free EPrints University Repository Software generates rich (and potentially even richer) usage metrics.It can be used for showcasing, navigating, comparing and assessing. Here is a sample of University Repository usage metrics for Southampton author Tim Berners-Lee: http://stats.eprints.ecs.soton.ac.uk/cgi-bin/irstats.cgi? OAR2008

38. Interoperable Repository Statistics OAR2008

39. Interoperable Repository Statistics

40. Some EPrints download metrics for top deposits by Southampton author Tim Berners-Lee. OAR2008

41. These Local Repository Usage metrics at the individual university level can then be complemented by CITEBASE, which provides global Citation, Download, Co-citation, Hub/Authority, growth, and other metrics: http://stats.eprints.ecs.soton.ac.uk/cgi-bin/irstats.cgi? OAR2008

42. OAR2008

43. OAR2008

46. Sample citation and download growth with time. (Downloads only start in 2005 because that is when this paper was deposited.) Early growth rate and late decay metrics for downloads and citations can also be derived.

47. Citations (C) CiteRank Co-citations Downloads (D) C/D Correlations Hub/Authority index Chronometrics: Latency/Longevity Endogamy/Exogamy Book citation index Sample of candidate OA-era metrics: • Research funding • Students • Prizes • h-index • Co-authorships • Number of articles • Number of publishing years • Semiometrics (latent semantic indexing, text overlap, etc.) OAR2008

48. These metrics can be validated in the UK Research Assessment Exercise (RAE), discipline by discipline, through multiple regression analysis: The metrics can be weighted by their ability to predict the rankings given by the evaluation by human peer panels: OAR2008

49. UK’s RAE 2008 will be a parallel panel/metric exercise, making it possible to develop a rich spectrum of candidate metrics and to validate each metric against the panel rankings, discipline by discipline, through multiple regression analysis, determining and calibrating the (“beta”) weights on each metric. Harnad, S. (2007) Open Access Scientometrics and the UK Research Assessment Exercise. Proceedings of 11th Annual Meeting of the International Society for Scientometrics and Informetrics 11(1) : 27-33, Madrid, Spain. Torres-Salinas, D. and Moed, H. F., Eds. http://eprints.ecs.soton.ac.uk/13804/ OAR2008

50. RAE 2001 Rankings for Psychology OAR2008