1 / 40

ISI Research Services Group:

ISI Research Services Group:. Analytical tools for Research Evaluation. Outline. Research evaluation: Unique features of Web of Knowledge Journal Selection Quantitative tools Cui bono: What sectors use these analyses Citation Analysis – Issues and Caveats Free Stuff!!.

dympna
Télécharger la présentation

ISI Research Services Group:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ISI Research Services Group: Analytical tools for Research Evaluation

  2. Outline • Research evaluation: • Unique features of Web of Knowledge • Journal Selection • Quantitative tools • Cui bono: What sectors use these analyses • Citation Analysis – Issues and Caveats • Free Stuff!! ISI Research Services Group Nancy K. Bayers

  3. The Web of Knowledge is… • An integrated, web-based environment providing high quality content and the tools to access, analyze, and manage scientific and scholarly information • Incorporating new technologies (linking, searching, and personalization) to integrate, extend, and enhance the researchenvironment ISI Research Services Group Nancy K. Bayers

  4. Web of Knowledgesm E-Journal Full-Text A Free Resource!! Evaluated, Selected Web Content E-Patent Full-Text ISI Research Services Group Nancy K. Bayers

  5. ISI Web of Knowledge • A. Journal Selection Process • “A cost effective index index must restrict its • coverage, as nearly as possible to only those • items that researchers are likely to find useful.” • -Eugene Garfield, Ph. D. ISI Research Services Group Nancy K. Bayers

  6. ISI Web of Knowledge • … Journal Selection • 1. Basic Journal Publishing Standards • 2. Editorial Content • 3. International Diversity • 4. Citation Analysis ISI Research Services Group Nancy K. Bayers

  7. ISI Web of Knowledge … Journal Selection 1.) Publishing Standards a.) Timeliness b.) Editorial Conventions c.) English Language Bibliographic Information d.) Peer Review ISI Research Services Group Nancy K. Bayers

  8. ISI Web of Knowledge ... Journal Selection 1.) Basic Journal Publishing Standards a.) Timeliness of Publication • A journal must be published according to its stated publication schedule to be considered for coverage in the ISI database. • ISI must receive three on-time issues in sequence. ISI Research Services Group Nancy K. Bayers

  9. ISI Web of Knowledge • ... Journal Selection • 1. Basic Journal Publishing Standards • a.) Timeliness of Publication • b.) International Editorial Conventions • To optimize retrievability: • - Informative Journal Titles • - Descriptive Article Titles • - Full Address Information for Every Author • - Complete Bibliographic Information for • All Cited References ISI Research Services Group Nancy K. Bayers

  10. ISI Web of Knowledge ... Journal Selection 1.) Basic Journal Publishing Standards a.) Timeliness of Publication b.) International Editorial Conventions c.) English Language Bibliographic Information - Article Titles - Author Names and Addresses - Cited References - Abstracts and Keywords ISI Research Services Group Nancy K. Bayers

  11. ISI Web of Knowledge ... Journal Selection 1.) Basic Journal Publishing Standards a. Timeliness of Publication b. International Editorial Conventions c. English Language Bibliographic Information d. Peer Review Application of the peer review process is another indication of the journal’s standards, the overall quality of research presented, and the completeness of cited references. ISI Research Services Group Nancy K. Bayers

  12. ISI Web of Knowledge • ... Journal Selection • 2.) Editorial Content • Do we need this new journal in the database, or is the category already well covered? • Does this journal offer something new? ISI Research Services Group Nancy K. Bayers

  13. ISI Web of Knowledge • ... Journal Selection • 3.) International Diversity • Are the authors, editors, and editorial advisory • board members from around the world? • Does this journal reflect the global context • in which scientific research takes place? ISI Research Services Group Nancy K. Bayers

  14. ISI Web of Knowledge • ...3. International Diversity • Regional Journals: • To properly reflect the global context in which scientific research takes place, and to provide balanced coverage in each category, ISI also seeks to cover the best regional journals as well. ISI Research Services Group Nancy K. Bayers

  15. ISI Web of Knowledge • ... Journal Selection • 4. Citation Analysis • Expert use of citation data help ISI identify • important, influential, and useful publications. • ISI measuresuse by analyzing citation data. ISI Research Services Group Nancy K. Bayers

  16. ISI Web of Knowledge • ... Journal Selection • ... 4. Citation Analysis • New Journals • Has the past work of authors and editorial • advisory board members received citations? • Established Journals • Impact Factor ISI Research Services Group Nancy K. Bayers

  17. ISI Web of Knowledge ... Journal Selection ...4. Citation Analysis Impact Factor: The average number of times recent articles in a journal were cited in a particular year. Nature Genetics: 2000 Impact factor: 30.910 ISI Research Services Group Nancy K. Bayers

  18. ISI Web of Knowledge Journal Citation Reports (JCR) -- 2001 Science Edition Impact Factor Calculation Journal: Nature Genetics Impact Factor: 29.600 Cites in 2001 to articles published in : 2000 = 6,167 1999 = 7,538 00+99 = 13,705 Number of articles published in: 2000 = 253 1999 = 210 00+99 = 463 Calculation: Cites to recent articles 13,705 = 29.600 Number of recent articles 463 ISI Research Services Group Nancy K. Bayers

  19. Research Evaluation • Web of Knowledge: unique features • cited references -- footnotes • all author names • all author addresses • Count • papers -- output • citations -- influence or impact • Baselines ISI Research Services Group Nancy K. Bayers

  20. Cui bono? • Policymakers • Research directors & administrators • Researchers • Publishers • Information specialists ISI Research Services Group Nancy K. Bayers

  21. Words to the wise... • Citation analysis is a fair evaluative tool in those fields where publication in the journal literature is the main vehicle of communication. • Need multiple indicators. Citation analysis is only a quantitative supplement to other measures of research evaluation, including peer review • There are always exceptions to everything. ISI Research Services Group Nancy K. Bayers

  22. Citation analysis: issues and caveats • Field publication behavior differences • How to handle... • Self cites and negative cites • Reviews and methods papers • English language bias • Skewed distributions • Relative measures/baselines • Confidence levels • What citations do not reflect ISI Research Services Group Nancy K. Bayers

  23. Field citation rates: average cites 1997-2001 • Cell & Developmental Biology 15.9 • Molecular Biology 10.6 • Biochemistry & Biophysics 9.1 • Neuroscience 7.3 • Chemistry & Analysis 5.9 • Physics 4.4 • Earth Science 3.5 • Materials Science 2.1 • Mathematics 1.3 ISI Research Services Group Nancy K. Bayers

  24. Field publication rates/size Field % papers in world • Clinical Medicine 24% • Chemistry 14% • Physics 12% • Biology & Biochemistry 8% • Plant & Animal Science 6% • Materials Science 4% • Neuroscience 4% • Geosciences 3% • Mathematics 2% • Computer Science 1% ISI Research Services Group Nancy K. Bayers

  25. Self-cites and negative cites SELF CITES • Cumulative nature of individual research • a natural and acceptable procedure if not excessive • Approximately 25% in biomedicine • 16% in plant physiology • 17.5% in neurobiology NEGATIVE CITES • Only 7% ISI Research Services Group Nancy K. Bayers

  26. Reviews and methods papers • Highly cited • Influence subsequent research • Lay foundation ISI Research Services Group Nancy K. Bayers

  27. English language bias • English is the language of science • Representative journals from each region and nation • Not significant for most analyses ISI Research Services Group Nancy K. Bayers

  28. Skewed distribution in bibliometrics • 80/20 rule • 20% of papers get 80% of cites • 50% of papers are never cited ISI Research Services Group Nancy K. Bayers

  29. Relative measures--baselines • Need context -- what does X cites mean? • Normalize by size -- average citation rates • Normalize by field -- average field citation rates ISI Research Services Group Nancy K. Bayers

  30. What citations to not reflect • Clinical importance • Public policy influence • Regional or local interest ISI Research Services Group Nancy K. Bayers

  31. Confidence levels Large data sets (nations, regions, large institutions) high moderate Engineering & Applied Sciences Fundamental & Basic Sciences low moderate Small data sets (small departments, individual researchers) ISI Research Services Group Nancy K. Bayers

  32. In-Cites: www.in-cites.com ISI Research Services Group Nancy K. Bayers

  33. In-Cites: www.in-cites.comEditorial Commentary by Highly Cited Scientists ISI Research Services Group Nancy K. Bayers

  34. In-Cites: www.in-cites.comHot Paper Commentary in Medicine ISI Research Services Group Nancy K. Bayers

  35. Special Topics: www.esi-topics.com ISI Research Services Group Nancy K. Bayers

  36. Special Topics: www.esi-topics.com ISI Research Services Group Nancy K. Bayers

  37. Special Topics: www.esi-topics.com ISI Research Services Group Nancy K. Bayers

  38. Special Topics: www.esi-topics.com ISI Research Services Group Nancy K. Bayers

  39. Science Watch: www.sciencewatch.com ISI Research Services Group Nancy K. Bayers

  40. ISI Research Services Group: Thank You!! Analytical tools for Research Evaluation

More Related