Prof. Zdenek Nemecek Charles University, Prague

1. Research performance evaluation in the Czech Republic:current methodology and expected development Prof. ZdenekNemecek Charles University, Prague

2. Brief history and legal frame • The distribution of the government research support is governed by the law 130/2002 and its consecutive amendments • Evaluation of the research entities started in 2004 without any connection with financing • The amendment from 2005 states that the distribution of the support for the long-term conceptual development of research entities (SCD) among the support providers (ministries, Academy) should follow the evaluation of the results • Methodics 2008, 2010, 2013 • The providers are obliged to follow this distribution for their entities unless they have their own evaluation system

3. R&D budget SCD Government sources only, EU funds not included Total Grant agencies, international cooperation, etc.

4. Evaluation Methodics Keep in mind that: 1 - Methodics is dedicated for division of money among the research entities 2 – it is not supposed to be used on the level of research teams or individuals 3 - it is not intended for identification of excellence • Each research entity provides the list of their results to RIV • The results are divided into categories – journal paper, proceedings, patent, etc. • The results are further divided according topics – physics, social sciences, mathematics, chemistry, etc. • Methodics assigns a predefined number of points to each result • The money that a particular entity receives would be directly proportional to the sum of the points gained by its results

5. Topical groups The number of points gained by a particular topical group is normalized to represent a given part of the points available (3 000 000)

6. Categories of results The numbers of points assigned to publication and application results are normalized to represent a given part of the points available 82.5% 17.5%

7. Evolution of the Methodics • The first Methodics 2008 provided a reasonable evaluation of research entities • However, the production of “results for points” started immediately • The only category of results not affected by this trend were papers in good journals (WoS) • The consecutive Methodis (2010) introduced several limitations: - percentage of applied results - division of results into topics (mathematics, social sciences, etc.) with predefined number of points These limitations brought a partial improvement but the main problem – equal number of points for all results in a given category - remains

8. Evaluation Methodics 2013 • Three parts - Mainstays • Each mainstay has its own allocation of points • Mainstay I – old system for papers in journals indexed in WoS or Scopus, books and papers in the Czech journals come through peer review • Mainstay II - peer review evaluation of a limited number of results • Mainstay III – new system of evaluation of applications, the results are not weighted and the points are assigned to research entities.

9. Mainstay weight • Mainstay (Pilíř) I –publication results • Mainstay (Pilíř) II – NEW – excellent results • Mainstay (Pilíř) III – applications

10. Mainstay I This mainstay is adopted and adapted from the previous Methodics 2008 and 2010 WoS SCOPUS ERIH DomesticBooksProceedings SHVa,bJimpJscJneimpJrec BC D SHVcJimpJscJneimp BC D Technika a informatika JimpJsc BC D Zemědělství JimpJsc BC D Vědy o zemi JimpJsc BC Matematika JimpJsc BC D Fyzika Jimp Chemie Jimp Biologie Jimp Lékařství Jimp Points (10 – 305) (10 – 30) 4 (4 – 120) (8 – 60) Social Sci and Humanities Social Sci Technical Sci and IT Agriculture Sci Earth Sci Mathematics Physics Chemistry Biology Medicine 82.5% of points is distributed according to this mainstay

11. Mainstay II. • Each research unit submits one excellent result per CZK 10 mil. for evaluation • The submission is accompanied with reasoning why the result is excellent • The result is evaluated by the Expert panel of a given topical group and 20% of the best is chosen (category A) • 10% (60 000)points is allocated to this mainstay • These points will be distributed to the research entities in accord with the number of results in category A • Additional 2000 points will be allocated to the entity that received an ERC grant in course of past 5 years • Note that this mainstay will be applied from 2015

12. Estimated number of results submitted to Mainstay II Total: 999 results

13. Mainstay III. • 105 000 points will be allocated according to this Mainstay • In the first step, a given number of points will be assigned to patents, novel races or cultivars • In the next step, the rest of points will be distributed in accord with the volume of money gained by a particular entity either as grants of applied research or by research contracts with private enterprises ∑Total= α∑Grants + β∑Contracted research (α = 0.9;β = 0.1)

14. Present status of M 2013 • Data for Mainstays I and III are collected • The software tools for the data processing are almost ready • The members of Expert panels are nominated • Peer review of books and Czech journals will start in course of June • The excellent results will be collected in near future • Since the process according to M 2013 will be applied only on the data from the last year, it will influence only about 20% of the budget • Our modeling has shown that no dramatic changes in the money distribution can be expected