Indicators to characterize public funding systems

1. Indicators to characterize public funding systems Benedetto Lepori, 28th November 2008

2. The issue • Is it possible to produce indicators to characterize/compare public funding systems including • The role of different funding agencies • The allocation methods • The streams of money • The beneficiaries • Beyond R&D statistics which is essentially focused on performers • With limited ability to map funders – performers flows • With a too simple description of funders (by “sector”) • Not taking into account the existence of intermediaries • Obvious relevance for public policies of these indicators • See for example the OECD work on steering and funding of public research where no data on allocation could be collected • Two PRIME project on the field • Exploratory project on public project funding • Analysis of CEEC funding systems and of their change

3. An overall view of public funding • Four relevant system layers • State • Funding agencies • Performers’ organisations • Research groups/individuals) • Two main allocation streams • core funding • project funding • Key issues • Interactions accross layers and funding modes • Increasing role of performers in driving funding

4. Policy layer National state Funding agencies (national) International agencies (EU, ESA, etc.) Agencies layer Core funding Project funding Organisations layer Higher Education Institutions and PROs Research groups layer Research units Internal allocation

5. Project funding analysis • Try to compare countries concerning the role and composition on project funding • Identify in each country the instruments which we can identify as project funding • As well as the main funding agencies • A basic definition + a lot of comparative work to treat in the same way the same instruments • Collect the data from different sources • Ministries reports, agencies reports, etc. • Per year and main beneficiary group • Data collection and cleaning procedures • Produce aggregations and international comparisons • As well as analyses of the evolution in last 30 years

6. Criteria • We consider project funding if • Attributed mostly for research purposes • Limited in time and scope (not recurrent) • Attributed by an external agency to the research organization • No reference to competition/modes of allocation • Issues / problems • Use for research activities (since we are not looking to performers) – difficult to assess for example for contracts • Long-term competitive schemes (centres of excellence) • Internal competitive schemes to research organizations (or vertically integrated organizations like academy of sciences)

7. Problem cases • European Space Agency contracts • Mostly to industry for development of rockets, satellites, etc. • Some funding for scientific programs • It is nearly impossible to ascertain how much should be considered as R&D • We adopted a broad delimitation • Including probably more that what would be considered as R&D funding

8. French CNRS • A very large organization with a double role • Funding its own laboratories (10%) • Funding joint laboratories with universities (thus partially outside the CNRS perimeter) • A partially competitive allocation mechanism • Labelling by CNRS gives access to its resources and there is a rather high turnover • Mostly in form of personnel, but no mandatory allocation • It is something intermediary between general and project funds • Accounting for nearly 10% of public research funding in France, thus impossible to put in a footnote • Our picture of the French system largely depends on the choice we do concerning these funds • Considering it as project gives a view of France much nearer to other Western European countries • See Theves et al. paper

9. Discussion • Overall the distinction works and is usable but • One needs much care in the comparisons especially for time series • We should try to develop a finer typology of funding instruments in the future • Intensive discussion was needed to solve dubious cases • Ensuring some comparability • In-depth knowledge of national systems was essential for this exercise • …this is not the end of the story…

10. Categories • Comparative analysis clearly needs common categories • Beyond individual list of instruments/agencies at national level • We devise three classifications • By type of agency (international, ministry, intermediaries) • By instrument type • By beneficiaries

11. Instrument types • Intuitively it is clear that there are different instruments concerning their orientation / type of research they fund • But designing a clear classification is very difficult since these concepts are largely multidimensional • We end up with a simple distinction between “academic” / thematic / innovation-oriented instruments • However, this is just a rough approximation of policy intentions, but not necessarily of the research done • Different features need to be combined to better understand the significance of instruments for the research system • Granularity is a problem since the level of aggregation is rather high

12. Data sources and data collection issues • There is no unique source of these data, but they had to be complied from: • State accounts (France) • Research ministry reports • Agency reports • Some existing databases • Surveys (for example the Swiss R&D survey) • Direct inquiry to ministries • Most data are available but with limitations/problems in many cases • In most cases it was possible to come back to the ‘70

13. Data problems • Funding to companies through loans • Estimate of cash value • Project decisions instead of allocations • One needs some kind of averaging • European Space Agency • Use national contribution as proxy (national return rule) • Contracts from ministries • Coverage is problematic outside the formal programs …even if the methdology is simple there is a lot of dirty work to get usable data…

14. Some selected results • PF as the second stream of research funding in the six considered countries • ¼ to 1/3 of funding volume • Strong increase in the role in the last years • A composite model of funding instruments • Some academic, some thematic, some innovation • Try to accomodate the different goals instead of chosing a single rationale • Differences between countries remain quite large • In the organisation of funding agencies and the type of funding • Strong dependence on history of organisational structures • One needs to consider carefully this context when designing European funding policies

15. Type of PF instruments

16. PF as % of BIP

17. From academic to thematic and back… % Thematic instruments % Academic (bottom-up) instruments

18. From steering to markets • Today’s highly differentiated project funding system resembles increasing to a market • Where loosely coordinated agencies buy research services from performers • With strong strategic behavior of applicants • Depending on their strenghts, needs, etc. • Building stable market structures • Needed to support long-term research, but also predictable allocation of funding • Crucially based on shared representations on others actors behaviour • Of funding agencies • Of potential competitors • Some evidence that core markets are relatively small-scale • Strong segmentation by domain/type of research/topic • Few key players with high success rates, many marginal players

19. CEEC project • To analyse the changes in the organization of public funding in CEEC • Three countries as a test (Czech Republic, Estonia, Poland) • Look also for availability of data (including time series) • Approach • Identify the main funding streams and to characterize them • Draw structural diagrams • Understand the role of funding agencies and allocation criteria • See Karel’s presentation for more details on specific results

20. MSHE Institutional funding 24,19% 70,66% 30,81% 17,37% 22,48% Higher education AS Government units 23,81% 7% HEI Institutes Institutes Institutes 13,20% 3,35% 5,2% 0,55% 0,49% 0,22% 0,17% 0,26% 0,68% 2,45% project funding Private 1,46% EU FP 1,57% 0.23% EU SF 1,52% MSHE fund 0,60% FPS Poland

21. Structural models • A step towards identifying the main structural features of national funding models • Taking into account the interactions between modes and layers (institutional complementarities) • Different models related to different national contexts • Which can be stable in an evolutionary perspective • Characterize the models in terms of some key features • Ability of the State of steering the system • Levels of delegation • Performance vs. innovation • Three basic models seems to emerge: • Project-funding based • Centralised • Mixed models

22. The project-funding based model • Most of research funding coming from highly differentiated set of non coordnated agencies • High share of project funding and low of core funding • State « buys » research services from groups • With limited attempt of central coordination • Mission agencies funding also basic research (DoD) • Reliance on competition rather than on coordination to get the required results • stratified university system with concentration in a few players • International excellence as the main criterion • Limitations of this model when the system is too small • Cumulative effects suppress diversity of the system (the Estonian case)

23. The centralised model • A single large research organisation funded by the state (outside the university sector) • The old CNRS model before joint laboratories • The old system of Academy of Sciences in communist countries • The today’s Polish system • The PRO has more functions than just funding • Decision on priority areas • Creation and restructuring of laboratories • Mostly coopting scientists in the PRO board • A strongly centralised system • Define priorities and concentrate effort • Central planning, but with all its rigidities in face of new scientific development • Less adapted to new science dynamics more based on complementarities than on heavvy investments (new « search regimes »; Bonaccorsi)

24. The mixed model • Some balance between core and project funding • Say 60% to 40% • A differentiated performers sector • Large university sector alongside a number of non-university research institutes • Trying to achieve some balance between • Targeting the best groups and spreading out resources (also because of the link with education and regional development) • Differentiating the performers sectors to answer to different missions vs. creating competition between performers • Typical of most Continental European countries including Switzerland • Best performing in small rich countries rather that on large ones • Is the efficiency loss and the lack of central coordination acceptable? • The Czech republic as a typical case

25. Conclusions (1) • It is basically feasible to characterize funding systems using existing data at national level • Mostly detailed budgetary data • Complemented with data from agencies and R&D statistics • A very good knowledge of national systems is required • To identify and classify funding streams • To identify the really relevant features • National experts are required for these tasks

26. Conclusions (2) • This works provides very interesting insights on the structure of funding systems • Quantitative indicators are essential to characterize them and to distinguish variants • The next avenue is to go towards data on micro-structures of funding systems

27. References(see www.enid-europe.org) • Lepori B., van den Besselaar P., Dinges M., van der Meulen B., Potì B., Reale E., Slipersaeter S., Theves J., (2007), Comparing the Evolution of National Research Policies: what Patterns of Change?, Science and Public Policy 34 (6), 372-388. • Lepori B., Dinges M., Potì B., Reale E., Slipersaeter S., Theves J., van d. (2007). Indicators for Comparative Analysis of Public Project Funding. Research Evaluation, 16 (4), 243-255. • Lepori B., Masso J., Jablecka J., Sima K., Ukrainski K. (2008). Research funding system in Central and Eastern European countries: a comparative analysis. Paper presented at the ENID-PRIME Indicators Conference, Olso, May 2008.