Giorgio Sirilli Research Director

1. Statistical indicators: their relevance, limitations and use in innovation studies and policy making Giorgio SirilliResearch Director

2. Statistical indicators Statistic: A numerical fact or datum, i.e. one computed from a sample Statistical data: Data from a survey or administrative source used to produce statistics Statistical indicator: A statistic, or combination of statistics, providing information on some aspect of the state of a system or of its change over time. (For example, gross domestic product (GDP) provides information on the level of value added in the economy, and its change overtime is an indicator of the economic state of the nation)

3. Theory and praxis “Thereisnothing more practicalthan a goodtheory.” Albert Einstein

4. The value of science William Gladstone Michael Faraday William Gladstone, then British Chancellor of the Exchequer (minister of finance), asked Michael Faraday of the practical value of electricity. Gladstone’s only commentary was ‘but, after all, what use is it?” “Why, sir, there is every probability that you will soon be able to tax it.”

5. Measuring in natural sciences In searchof the fundamentallaws, the “truth”

6. CERN (European Organization for Nuclear Research)

7. Measuring in natural sciences In searchof the fundamentallaws, the “truth”

8. Measuring in engineering The highestaccuracypossible Ohm's law I = V/R

9. Measuring in the social sciences Statistics

10. Statistical indicators

11. Statistics and chicken LA STATISTICA Sai ched'è la statistica? È na' cosa che serve pefà un conto in generale de la gente che nasce, che sta male, che more, che va in carcere e che spósa. Ma pè me la statistica curiosa è dove c'entra la percentuale, pè via che, lì, la media è sempre eguale puro co' la persona bisognosa. Me spiego: da li conti che se fanno seconno le statistiche d'adesso risurta che te tocca un pollo all'anno: e, se nun entra nelle spese tue, t'entra ne la statistica lo stesso perch'è c'è un antro che ne magna due. Trilussa (1871-1950)

12. Measuring in the social sciences The assmptionofcoeterisparibus i. e. other things being equal or held constant Whenyoumeasure a system, youchange the system

13. Data sources Statistical offices/agencies Private data producers

14. Science and technology indicators After more that 50 years, what can besaidaboutindicators?

15. Science and technology indicators “Oste, il vino è buono?” “Innkeeper, is the wine good?”

16. Science and technology indicators S&Tindicators are defined as “a series of data which measures and reflects the science and technology endeavor of a country, demonstrates its strengths and weaknesses and follows its changing character notably with the aim of providing early warning of events and trends which might impair its capability to meet the country’s needs”. Indicators can help “to shape lines of argument and policy reasoning. They can serve as checks, they are only part of what is needed”. (OECD, 1976)

17. Science and technology indicators Indicators are a technology, a product, which - governs behaviour - is modified by users (outside of the producer community) - develops in response to user needs Data sources • Surveys, administrative data, private files, case studies • Data collection is informed by manuals Data populate statistics which can be indicators Decisions are taken on the basis of indicators

18. The linear model of innovation Research Production Development Design Engineering Based on research Sequential Technocratic

19. Linear model « L'intendance suivra »  (Logistics, supplies will come later)

20. The chain-linked model of innovation Research Ricerca Knowledge Invention/ analytical project Detailed project & test Re-design & production Distribution & market Potential market Invenzione/ progettazione analitica Progettazione dettagliata e test Distribuzione e mercato Mercato potenziale Based on design Interactive Researchisnot a pre-requisiteforinnovation

21. The triple helix model of innovation

22. The open innovation model

23. Measuring in the context of a model Models or, at least, a scheme Corrado Gini

24. The OECD manuals

25. A breaf history of STI policy From patronage to public and private investment Patronage fromrulers Industrial Revolution Between the First and the Second World War rockets, nuclearenergy, operationsresearch, DDT After the Second World War science and technology policy fromgovernments

26. A brief history of the measurement of STI Second world war USA vs USSR The roleof the OECD Deeper and deeper……

27. OECD publications in the area of STI

28. STI indicators - R&D - bibliometrics - innovation - patents - TBP - human resources for S&T - peer review - students - graduates - spin-offs - contracts and other funding - other

29. Science and technology policy Report “Science the Endless Frontier”, 1945 • Vannevar Bush

30. “Science the Endless Frontier” Concerns: Military security and Health Solution: Science policy “The Government is particularly fitted to perform certain functions, such as the coordination and support of broad programs on problems of great national importance” “Scientific progress on a broad front results from the play of free intellects, working on subjects of their own choice, in the manner dictated by their curiosity for exploration of the unknown. Freedom of inquiry must be preserved under any plan for Government support of science”

31. The triangle in S&T policy making Policy makers design the future Analysts interpret today Data producers measure the past

32. Luigi EinaudiItalian President Policy makers “Conoscere per deliberare” Know first and then sanction

33. Policy makers • Vannevar Bush • Report “Science the Endless Frontier” 1945 President Roosevelt

34. Policy makers “Why shoud we pay the researchers if we make the best shoes in the world?” Silvio Berlusconi “Culture doesnotprovidefood” (Con la cultura non si mangia) Giulio Tremonti

35. The dangerous business of statisticians What matters is not how one fashions things, but what one does with them; not the weapon, but the battle (…). The making and the using of the tool are different things. O. Spenger, Man and Technics: A Contribution to a Philosophy of Life, 1932

36. S&T statisticians: arms producers? “I have no regret. Others are responsible for the bloodshed caused by the AK-47 machine gun. It is politicians’ fault non to be able to find appropriate solutions but rather to resort to violence.” (M. Kalashnikov)

37. Data producers Statistical agencies Private sources

38. Bibliometrics • Number of publications • Number of citations • Impact factor • h-index

39. Use of publications for decision making • The case of China (SCI) • The case of Russia

40. The h-index (Jorge Eduardo Hirsch) In 2005, the physicist Jorge Hirsch suggested a new index to measure the broad impact of an individual scientist’s work, the h-index . A scientist has index h if h of his or her Np papers have at least h citations each and the other (Np − h) papers have ≤ h citations each. In plain terms, a researcher has an h-index of 20 if he or she has published 20 articles receiving at least 20 citations each.

41. Impact factor (Eugene Fardfield) The impact factor (IF) of an academic journal is a measure reflecting the average number of citations to recent articles published in that journal. It is frequently used as a proxy for the relative importance of a journal within its field. In any given year, the impact factor of a journal is the average number of citations received per paper published in that journal during the two preceding years. For example, if a journal has an impact factor of 3 in 2008, then its papers published in 2006 and 2007 received 3 citations each on average in 2008. ("Citable items" for this calculation are usually articles, reviews, proceedings, or notes; not editorials or letters to the editor).

42. Nobel laureates and bibliometrics (Boson in 2013) Peter Ware Higgs 13 works, mostly in “minor” journal, h-index = 6 Francois Englert 89 works, both in prestigious and minor journals, h-index = 10 W. S. Boyle h-index = 7 G. E. Smith h-index = 5 C. K. Kao h-index = 1 T. Maskawa h-index = 1 Y. Namby h-index = 17

43. How difficult is to evaluate! Bruno Maksimovič Pontekorvo “Physicsis a single discipline butunfortunatelynowadaysphisicistsbelongtotwodifferentsgroups: the theoreticians and the experimentalists. If a thoreticiandoesnotpossesanextraordinaryabilityhis work doesnotmakesense….Forexperimentalistsalsoordinarypeole can do a useful work …” (Enrico Fermi, 1931)

44. Science and ideology: the impact on citations Fall of the Berlin wall Berlin Nov. 1989

45. R&D data from Frascati and Oslo do not match BERD (Frascati), Annual, all NACE industries, no threshold CIS (Oslo), Selected NACE industries, 10 employees or more, sample

46. The use of indicators

47. A rhetoric device: a plethora of figures and graphs “Secure a quantitative statement of the critical elements in an official’s problem, draw it up in concise form, illuminate the tables with a chart or two, bind the memorandum in an attractive cover tied with a neat bow-knot (…). The data must be simple enough to be sent by telegraph and compiled overnight” (Mitchell, 1919)

48. A rhetoric device: a plethora of figures and graphs “In the various studies on productivity and the New Economy the OECD constantly reminded the reader that the links between science, technology and productivity have not been demonstrated”. “A large series of graphs and figures could persuade the reader of the seriousness of the study. Although no statistics could be used to prove the emergence of the New Economy, graphs and figures nevertheless served the purpose of empiricism”. (Godin, 2004)

49. A rhetoric device: a plethora of figures and graphs

50. EU Innovation Index: four groups