Thinking about Outputs: Towards a Bibliographic Database for the Humanities and Social Sciences

1. Thinking about Outputs: Towards a Bibliographic Database for the Humanities and Social Sciences Milena Žic Fuchs Chair of the Standing Committee for the Humanities Strategic Workshop: Addressing the Shortage of Data on the Social Sciences and Humanities Strasbourg, 14 – 15th September 2010 Standing Committee for the Humanities www.esf.org/human

2. Standing Committee for the Humanities www.esf.org/human Two issues have been prominent in discussions on research outputs in the Humanities and Social Scences, especially during the last decade. • how to enhance the visibility of research outputs in the Humanities and Social Sciences • how to set up evaluation systems for the Humanities and Social Sciences – systems which would reflect the specific nature of their research outputs

3. Standing Committee for the Humanities www.esf.org/human • decades of attempts to deal with these issues in Humanities research on basis of databases, such as WoS, SCOPUS, etc. have proven unsuccessful because of the specific nature of research in the Humanities and Social Sciencesreflected in specific outputs

4. Standing Committee for the Humanities www.esf.org/human • Specific nature of Humanities and Social Sciences research reflected in a number of characteristics • publication of articles, books, etc. in national languages • importance of monographs, chapters in monographs, etc. • necessity to include “collections” such as revised editions, • collections of data

5. Standing Committee for the Humanities www.esf.org/human • in order to deal with diverse nature of Humanities research, the ESF, namely Standing Committee for the Humanities (SCH) in 2001 launches ERIH – European Reference Index for the Humanities • main aim of ERIH – to enhance the global visibility of high-quality research in the Humanities published in academic journals in various European languages across all of Europe

6. Standing Committee for the Humanities www.esf.org/human ERIH at present used as a “visibility” mechanism and an “evaluation” tool (different uses have been noted across Europe)

7. Standing Committee for the Humanities www.esf.org/human Reactions to ERIH • after publication of ERIH Initial Lists criticisms directed to ERIH from different directions • two main sources of criticisms: • those that are against any changes in Humanities research traditions and against “bibliometric” tools • those that want to take evaluation in the Humanities to a higher “bibliometric” level

8. Standing Committee for the Humanities www.esf.org/human Reactions to ERIH (cont.) • criticisms from research communities – national research communities, discipline research communities (e.g. German historians, Irish Gaelic Studies research community) • criticism from funders – the need for adequate evaluation tools

9. Standing Committee for the Humanities www.esf.org/human • In summer of 2008 a number of funding bodies • ESRC/AHRC (UK), ANR (Fr), DFG (De), NWO (Ne) • secure funds for • Towards a Bibliometric Database for the Social Sciences and Humanities: A European Scoping Project • (Ben Martin, Freeman Centre, SPRU University of Sussex, UK) • report finalized in March 2010

10. Standing Committee for the Humanities www.esf.org/human One possible solution is to go forward with a bibliometric database as outlined in the SPRU Final Report • the main aim of the SPRU Report is to facilitate the creation of a bibliometric database for the Humanities and Social Sciences • What does the SPRU Report suggest? • A pilot study is proposed, starting with a small number of countries and a small number of disciplines • Although weakened in the Final Report short time frames envisaged (approximately 12 months) • Standard setting body composed of bibliometricians and researchers, but not all disciplines represented

11. Standing Committee for the Humanities www.esf.org/human On March 31, 2010 a meeting was held in Brussels on the Future of ERIH • representatives of 26 Member Organizations • Chairs of SCH and SCSS • Marc Heppener, ESF’s Director of Science and Strategy Development

12. Standing Committee for the Humanities www.esf.org/human As the conclusion of the meeting, it was decided to set up a small working group to provide a short report with recommendations on the larger issues: Istvan Kenesei (Hungary) Pascal Perrin (Belgium) Nigel Vincent (UK) Gunnar Siversten (Norway – Chair) Sir Roderick Floud (Chair of SCSS) Milena Žic Fuchs (Chair of SCH) (Marc Heppener, Director of Science and Strategy Development, Nina Kancewicz-Hoffman, Head of Humanities Unit of ESF, Balazs Kiss, Head of Social Sciences Unit)

13. Standing Committee for the Humanities www.esf.org/human Basis for the report: Build a European bibliographic database • build an all-encompassing database which would reflect the true diversity and richness of European SSH research • visibility of European SSH research • this visibility could produce new energy in research – harnessing assets • this visibility could result in true Pan-European value

14. Standing Committee for the Humanities www.esf.org/human • this implies creating national databases of journals, monographs and other research outputs • go from national databases to a European-level database • synchronization of existing databases and setting standards for “new” ones • creation of a central coordination of national organizations for the establishment ofstandardized rules to ensure full comparabilityofnationally provided data central coordination ↔ national databases

15. Standing Committee for the Humanities www.esf.org/human Creation of such a database opens up possibilities of setting up “evaluation”/”assessment” systems on national as well as European level • National evaluation systems should stress and institutionalize the importance of high quality research outputs in national language • National evaluation systems should stress and institutionalize the importance not only of journal articles, but also books and other types of research output • National evaluation systems should take into consideration different research traditions in the disciplines of the Humanities  this diversity should be reflected at all levels of evaluation in different language research communities • Build from national to European level

16. Standing Committee for the Humanities www.esf.org/human The report has been finalized and will be disseminated in the near future. Main features of the “bibliographic approach”: • recommended strongly – close collaboration between Humanities and Social Sciences • build database which would make all scholarly literatures searchable and accessible across countries • bibliographic references would thereby mirror as fully as possible the output of European research in the Humanities and Social Sciences, and show diversity reflected in national languages, but also in national traditions within specific disciplines • on the basis of such a database move towards evaluation mechanisms

17. Standing Committee for the Humanities www.esf.org/human The envisaged approach could prevent negative effects of “evaluation tools” as documented examples show in other domains of science • In Spain, implementation of national assessment system in economics resulted in major shift to international journals indexed in either WoS or other international databases. • In Spain, in medical research, assessment had immediate effect on mass emigration of best research articles to foreign journals resulting in an increasing neglect of Spanish journals to which Spanish researchers rarely submit their best work. • In Croatia, medical journals published more and more in English – with few exceptions • Since major commercial providers (e.g. WoS) as a rule index journals and do not cover books, or chapters in books, general trend is to publish journal articles

18. www.esf.org/human Analyses of the previously mentioned shifts in publication behaviour show • “the destruction of Spanish as a language of science”  effect on language • many research groups have altered their research agendas to accomodate being published in high impact journals  change in research topics • shift towards international journals and, what is more, to journal articles in some research domains is showing shift away from books and chapters in books  change in publication behaviour

19. Standing Committee for the Humanities www.esf.org/human A bibliographic approach could to a certain extent counteract negative changes in publication cultures and possible negative effects of evaluation tools. “Back to books Researchers should be recognized for writing books to convey and develop science.” “The expansiveness of a book allows sophisticated arguments to be put forward and widely debated; new ideas that cross disciplinary boundaries can more readily be shared and worked through.” “But if this exhortation is to have any traction, the effort and skill required to write a book needs to be rewarded in the carreer recognition of scientists who devote time to mastering the art to good effect – a recognition that is commonplace in the social sciences and humanities.” Nature 463, (February 2010)

20. References www.esf.org/human • Adler, R., Ewing, J., Taylor, P. (2008) “Citation Statistics: A Report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS)”, for Joint Committee on Quantitative Assessment of Research • Carmona, S. et al. (2005) “From zero to infinity: the use of impact factors in the evaluation of economic research in Spain”, IE Working paper, WP05-22, 13-05-2005 • Ha, T.C. et al. (2006) “The Journal Impact Factor: Too Much of an Impact?”, Annals Academy of Medicine, Vol. 35, No. 12. • Hicks, D. et al. “Towards a Bibliometric Databasefor the Social Sciences and Humanities: A European Scoping Project”, School of Public Policy, Georgia Institute of Technology, USA • Moed, H. F. et al. “Options for a Comprehensive Database of Research Outputs in Social Sciences and Humanities”, Center for Science and Technology Studies, Leiden University, The Netherlands • Nature 463, (February 2010) │doi: 10.1038/463588a; Published online 3 February 2010 • Rey-Rocha, J. et al. (2001) “Some Misuses of Journal Impact Factor in Research Evaluation”, Cortex, Vol. 37, Issue 4. • Seglen, P.O. (1997) “Why the impact factor of journals should not be used for evaluating research”, British Medical Journal, 314:497.