Enhancing Research Evaluation Through Open Access Repositories: Challenges and Opportunities
This evaluation research conducted by José Manuel Barrueco from the Universitat de València discusses the current state of Open Access Repositories (OAR) in disseminating research results and their implications for citation analysis and research evaluation. The study examines the CitEc case, illustrating its roles, successes, and challenges in providing visibility to scientific literature. It highlights the potential of citation analysis for improving the impact of research outputs, addressing issues like metadata quality, author identification, and the integration of Open Access concepts with traditional research evaluation metrics.
Enhancing Research Evaluation Through Open Access Repositories: Challenges and Opportunities
E N D
Presentation Transcript
Research evaluation requirements José Manuel Barrueco Universitat de València (SPAIN) Servei de Biblioteques i Documentació May, 2011
Background • Librarian at the University of Valencia (Spain) • Manager of the institutional repository • Co-Founder of the RePEc digital library • Running an autonomous citation index for Economics (CitEc) since 2001
Contents • The current situation • What has been done? • A case study: CitEc • The challenge • The requirements
The current situation • OAR are a new medium to distribute research results • More than 1800 registered in ROAR • Aim: increase the visibility and impact of the scientific literature • Citation analysis used for research evaluation purposes • Challenge: to apply citation analysis to contents distributed in repositories • It will demonstrate the open access usefulness • It will be an important added value for authors • It will contribute to increase OAR contents
What has been done? • Many research projects, not so many services • Usually working with OA journals, not OAR • Subject based repositories: • CiteSeer • arXiv: referes to + cited by • RePEc: referes to + cited by + author/institutions rankings • Institutional repositories • Nothing has been done yet
A case study: CitEc • It is a RePEc (Research Papers in Economics) service • RePEc describes the whole discipline, not just research outputs: • Institutions (departments, research institutes …) • Authors • Publication channels (journals, working papers series, books …) • Research outputs (articles, papers) • Every player in RePEc has a persistent identifier • RePEc in numbers: • +1,000,000 working papers + journal articles • +27,000 author contact and publication listings • +12,000 institutional contact listings • Coming from more than 1,300 data providers
A case study: CitEc • A citation dataset of: • 6,222,142 references • 2,517,198 citations • 287,216 documents processed (about 30% of the available documents) • CitEc produces open citation data which is used by other RePEc services to: • add refence linking support • Create rankings (authors, institutions … )
A case study: CitEc • Some examples: • CitEc home page: • http://citec.repec.org • CitEc data used for reference linking: • http://econpapers.repec.org/RePEc:ags:arerjl:31417 • CitEc data used for research evaluation • http://ideas.repec.org/top/
The challenge • Can citation analysis be applied to contents distributed in OAR? • YES, WE CAN!! • But …. It is not easy! • If technology is not a problem (ParsCit, Xpdf), then what … ?
The challenge • OAR are quite empty • 43% of OAR listed in BASE have less than 1,000 items • They are not limited to research outputs but any digital content (learning objects, cultural heritage … ) • The size is not a problem • We still have the subject repositories • CitEc started with no more than 25,000 documents • We may see the citation analysis as an opportunity to increase OAR contents.
The challenge • Metadata issues • No way to differenciate research outputs of other digital content in OARs • Few OARs are providing list of references for each document • E-LIS or MPRA • Limited use of persistent identifiers (DOIs) • No direct access to the documents full texts • Documents in OAR are described as independent entities instead of representing relationship (isVersionOf, isPartOf … ). Research context gets lost
The challenge • Researchers identification issues • Researchers are the core of our efforts • We need to provide them aggregated citation data from different OAR • But OAR pay little attention to the identification of researchers (authors represented as not standarized strings) • The big problem: author disambiguation!! • Here we have an opportunity for CRIS+OAR integration
The (basic) requirements • Improve the interoperability of research OAR. • Improve the amount and quality of metadata • Guidelines for providers. But, don’t we have DRIVER? • Not enought • They haven’t been widely adopted, because there are not enought incentives to implement them. • Use of persistent identifiers for researchers in CRIS+OAR • OpenID, ORCID • A working alternative: AuthorClaim by Thomas Krichel
The requirements • Develop citation service providers!! • Services like BASE should move forward to provide new added value services such as citation analysis • There is ground for new players too • The snake biting its tail!! • They should provide open citation data so that it could be reused by research managers and administrators • Open data != API • But, such services should be expensive?! • Not really, CitEc budget … 0 €
