Enhancing R&D Resource Allocation Through Performance Information: Insights from Korea

1. Session 5. Improving the Use of Performance Information for Resource Allocation in R&D: Funding and Priority SettingFrom the Central Budget Authority’s Perspective Nowook Park Center for Performance Evaluation & Management Korea Institute of Public Finance (KIPF)

2. Contents • Performance information of R&D in Korea • Issues on performance information of R&D • Use of PI for resource allocation in R&D in Korea • Issues on the use of PI for resource allocation in R&D

3. 1. Performance information of R&D in Korea

4. R&D Performance Evaluation System in Korea • Law for R&D program performance evaluation and management was enacted in 2005 • Three evaluation systems • Program review process (Self-Assessment) • 164 programs are evaluated for 2007’s performance • Evaluation process for selected programs • Long-term program, big program, duplicative programs etc • 27 programs are evaluated for 2007’s performance • Evaluation on public research institute • 32 institutes are evaluated for 2007’s performance • Feasibility study on big R&D programs • Programs with more than $ 50 million budget

5. Performance Information of R&D Programs • Information from program reviews • Based on checklist • Contents of checklist • Program purpose • Program design • Program management • Performance plan • Results (mostly based on performance indicators) • Feedback

6. Performance Indicators of R&D • For the sake of line ministries, pools of PI for 28 program types are provided • 28 programs are based on program purpose and utilization of program outputs • Program purposes are classified into 4 categories • R&D, HR development, international cooperation and infrastructure development • Utilization of program outputs is divided into 7 categories • Knowledge promotion (basic and applied) • Industrial development (short-term, medium/long-term development) • Public and social welfare (public service, public health and social welfare, region)

7. 2. Issues on performance information of R&D

8. Issues on Performance Information of R&D • Hard to develop relevant performance information • Results are realized in long term • Sometimes even establishing milestone indicator is not easy, because the process is not certain ex ante • Hard to evaluate R&D programs based on indicators • Peer reviews are widely used method • Expertise, balance and independence are required • Need to find optimal mix of expert judgment and alternative method

9. Issues on Performance Information of R&D in Korea • PI from evaluation process is based on checklist • Evaluation on program results heavily relies on performance indicators • Cost-benefit indicators are rarely adopted in the evaluation process, even though they suggested in the PI pool • Hard to come up with relevant information • Establishing reliable peer review process is hard • Existence of strong social ties among scientists tends to hinder objective evaluation process • Influences from line ministries may exist

10. 3. Use of PI for R&D resource allocation in Korea

11. Overview of R&D Resource Allocation in Korea • Korea becomes 8th country whose R&D spending exceeds $10 billion in 2008 • Ratio of R&D spending to total government spending is 4.7% in 2008 • Based on “National R&D Program Total-Road Map”, strategic resource allocation occurred • Increased spending on small scale basic research by individuals • Increased ratio of spending on the areas where spillover effects are big • Biotechnology, Energy, Basic research • Decreased ratios of spending on the area where private sector’s capability is mature • Machinery, IT and electronics

12. Changes of R&D Budget Process in Korea (1) • National Science and Technology Council (NSTC) was established in 1998 • Centralized budget process for R&D • Previously there was no systematic budget process for R&D programs • Science and Technology Innovation Bureau within Ministry of Science and Technology was established in 2004 • Has authority of budget allocation of R&D programs • Central budget authority usually accepted the budget proposal from Science and Technology Innovation Bureau with little modification

13. Changes of R&D Budget Process in Korea (2) • Ministry of Strategy and Finance takes over the authority of budget allocation for R&D programs in 2008 • Science and Technology Innovation Bureau was abolished with the inception of New Administration • Implies strengthening the role of the central budget authority in R&D budget allocation • Expect more active use of performance information for R&D budget allocation

14. Strategic Resource Allocation of R&D • R&D is one of 14 areas in National Fiscal Management Plan (Medium Term Expenditure Framework) • For FY 2008, strategic resource allocation is based on “National R&D Program Total-Road Map” • Role of PI in strategic resource allocation appears to be limited

15. Resource allocation among R&D Programs • PI from existing R&D evaluation systems are not actively utilized in budget allocation so far • However, as the authority of resource allocation is transferred to the central budget authority, more active use of PI is expected • Evaluated as lower 10% of programs in each line ministry will be in danger of 10% budget cut

16. 4. Issues on Using PI for Resource Allocation in R&D

17. Use of PI for R&D Resource Allocation • Strategic resource allocation • Resource allocation among different areas, such as education, defense, R&D and so on • Resource allocation among R&D programs pursuing different goals • Resource allocation among R&D programs pursuing similar goals • Resource allocation among R&D programs within spending ministries/agencies

18. Issues on the Use of PI for Strategic Resource Allocation in R&D • Strategic plan and PI should be consistently integrated • PI at the national level may give rough signals for strategic resource allocation when it is combined with other considerations • It is not easy to use PI to set the priority among different areas, such as education, defense, R&D and so on • Other factors may play a bigger role

19. Issues on the Use of PI for Resource Allocation for R&D Programs • Selecting appropriate units for evaluation is important to use PI for setting the priority among R&D programs pursuing similar goals • Identifying programs pursuing similar goals is the first step • Program review process may facilitate the process • Crosscut analysis can be used for resource allocation if relevant PI is available for each program • Using PI for setting priority among programs pursuing different goals is not easy • Use of PI for management improvement is feasible • Use of PI for resource allocation is more complicated

20. Issues on the Use of PI for R&D Resource Allocation in Korea • PI has not been systematically used in strategic resource allocation • Use of PI in the national fiscal management plan need to be improved • More active use of PI for resource allocation is expected but it remains to be seen how effective it will be • So far, the focus is given to resource allocation within each spending ministry • Uniform criteria may be adopted to strengthen use of PI

21. Thank You!