A Methodology for Evaluating Ex Ante the Regional Benefits of R&D Projects

1. A Methodology for Evaluating Ex Ante the Regional Benefits of R&D Projects Stephen Roper, Nola Hewitt-Dundas and James H LoveAston Business School, Birminghams.roper@aston.ac.uk

2. Why is ex ante evaluation of R&D projects particularly difficult? • Uncertain outcomes or success, i.e. risk • Uncertain timing of outcomes or benefit stream • Difficulty of assessing chance of technology developed being exploited locally by the R&D performer • And, crucially, difficulty of assessing the extent of local spillovers through: • technology or knowledge transfer • supply chains • development of research trained labour

3. The Task • Key questions: • What is best practice in ex ante evaluation of publicly funded R&D projects? • Can we create a framework/tool to enable us to better evaluate the returns to R&D projects? • Elements of study • Discussions with agencies in Scotland, Wales and DTI • International consultation – Finland, Israel, US • Extensive review of evaluation/academic literature • Development of tools for Invest NI • Calibration and revision of tools in seven case studies of existing R&D Centres

4. Best Practice? • Best practice/ international/ lit survey - no existing systems for predicting returns (or ex ante evaluation) – No ‘silver bullet’ • Instead we found different strategies: • Avoidance of issue (Israel) • Peer review of projects (US, Finland, Scotland) • Strategic rather than economic evaluation (DTI, Wales) • But: evaluation studies do provide rich knowledge base of ex post effects of different types of R&D projects.

5. Key Questions • So .. can we use existing knowledge base to draw inferences ex ante about the likely regional benefits of different types of R&D project. • RQ1: What (global) benefits stem from the establishment of a new R&D centre or facility? • RQ2: What determines the share of these benefits which accrue to the host region? • How do we codify the answers into a ‘tool’ usable by a regional development agency?

6. RQ1: What global benefits stem from the establishment of a new R&D centre or facility?- An Inventory Approach

7. RQ1: What global benefits stem from the establishment of a new R&D centre or facility?- An Inventory Approach

8. RQ2: What determines the share of these benefits which accrue to the host region? • The Profile of the R&D Centre or Project • Type of R&D • Institutional or Organisational Setting • Applicability • Business Ownership • Embeddedness of the R&D centre • Parallels with high-tech plants? • Effects of region size • Absorptive capacity of the RIS • Industrial composition • Absorptive capability of local firms • Public and private knowledge mediating institutions • Networks

9. Key Conclusions from Research Phase • Ex ante impact assessment needs • benefits inventory • knowledge base from prior evaluations/studies • profile/landscape/synergy consideration • The bottom line? • Private benefits depend crucially on type of business • Rent based spillovers depend on embeddedness (labour may be stronger than product market) • Pure knowledge spillovers depend on synergy/RIS

10. Tool Building and Validation • Spreadsheet based scoring tool for agency staff • Developed using inventory x regional share approach • ‘Validated’ using 7 existing Centres and comparison of outcomes to ex ante predictions • Tool developed can – at best- give a broad indication of expected effects • So, use alongside a narrative template developed to reflect same structure

11. End Notes • As yet no way of making reliable ex ante predictions of social rate of return from individual R&D projects • But we can use our framework to identify a comprehensive list of potential benefits and a general idea of their likely importance to host region. • Individual case-specific factors will still be important, however.

12. Spreadsheet ModelStep 1: Global benefits of R&D Project

13. Spreadsheet ModelStep 2: Regional Share of Benefits

14. Narrative Template – Private Benefits

15. Narrative Template – Wider Benefits