Ex post evaluation 2007-2014 of ERDF and CF: first discussion with Member States

1. Ex post evaluation 2007-2014 of ERDF and CF: first discussion with Member States DG Regional and Urban Policy Brussels, 30 May 2013 Cohesion Policy

2. Introduction • Legal obligation – until 31 December 2013 • Thematic breakdown plus presentation of data and findings by Member State. Synthesis report. • Methods depending on questions • Combination of "simple" review with deepening studies on selected questions • A dialogue oriented evaluation: hearings, discussions with Member States • Coordination with ESF

3. SME, research, Innovation • Whythis evaluation? 25% ofexpenditure, somecategories do not tellwherethemoneygoes (€15 billion "Other investment in firms", €9 billion "othermeasurestostimulateresearch, innovation, entrepreneurship in SMEs"). • Questions: Whatistheoverallpolicy mix in regions? Whatisthe rationale? • Whatistheeffectivenessof different schemesfor SMEs? Whatisthejustificationforfinancialsupport?

4. SME, research, Innovation • Methods: • Literaturereview and factfindings: 15 (?) casestudies on OPs investingthemost in RDI and SMEs • Theorybasedimpactevaluations on behaviouralchanges on 3 or 4 selectedcases. • Expert meetings, twoseminarswith MS Contact in REGIO: Adam Abdulwahab and Marielle Riché

5. SME, research, Innovation –questions for Member States • Anyevaluationsavailableorplanned in 2013-2015? • Can yousuggestacademicexperts and literature? • Anyotherevaluationquestions? • Anysupportschemethatwouldbeinterestingtoassess?

6. Venture capital • Whythisevaluation? Thereissomeevidencethatfinancialinstrumentsaremorecost-effectivethangrants (andthattheinnovationsuccessofthe US is in partbased on VC) Questions: • Whatarethemainschemes? How do theywork? Whatcosttoset-upandrun? • Whataretheireffects? Whatisthemechanism? Are theresituations/contextswherethisworks/doesn'twork?

7. Venture capital 2 Methods: casestudies, beneficiarysurveys Questions: • Do youhavesignificant VC schemes? Howcostlytoset-up/run? Do theyfocus on specificsectors/firm size? Are theyconsideredsuccessful? • Do youhavegoodevaluations/casestudiesalready? • Contact in REGIO: Daniel Mouqué

8. Large enterprise support • Whythisevaluation? Counterfactuals so far do not favourmoneyto large firms. This evaluation will use non-CF techniquestoassesstowhatextentthisapplieselsewhere (egacrossCohesion countries, tofdi) andhow non-financialsupportcompares? Questions: • Whatarethemainformsofsupport? • Whataretheireffects? Whatisthemechanism? • Differbysupport type, country, new/existing, etc? • Are impactssustainable? Wider benefits?

9. Large enterprise support 2 Methods: literaturereview, beneficiarysurveys, casestudies Questions: • Do youhavesignificantsupportto large enterprises? Are theyusuallyexisting/new/FDI investments, financial/non-financialsupport? • Do youhavegoodevaluations/casestudiesalready? • Contact in REGIO: Daniel Mouqué

10. Transport • Why this evaluation? Transport remains a major investment area for cohesion policy (planned € 75 billion). Need to assess the impact. It will include mapping of major projects to verify if they amount to a network. • Questions • What has the ERDF contribution helped to achieve? • To what extent have ERDF contributed to creation of a coherent transport network? • Are the supported (major) projects financially sustainable?

11. Transport • Methods • Desk research and fact finding (incl. analysis of information provided in the monitoring system, AIRs, evaluation reports, etc.) • Modeling of transport effects (time saving, CO2 emissions, GDP?) with TransTools • 10 (?) regional case studies to assess – among others – the financial sustainability of transport projects • 3 meetings with the MS to discuss findings

12. Transport • Questions to Member States • Do youhaveanycommentson questionsandmethods? • Are thereanyvolunteersforthecasestudies? • Are there any evaluations in this field you would recommend to us? • Do you plan to launch any evaluations in this field between 2013 and 2015? • Can you suggest any academic experts and literature? • Contact: Jan Marek Ziółkowski, DG REGIO

13. Environment - Financial Sustainability • Primary areas: water supply, wastewater treatment, municipal solid waste treatment • Polluter/user pays principle and affordability • Reviewing provisions in MS to ensure financial sustainability • Reviewing financial sustainability of projects that were subject to ex post CBA (we already have a baseline), plus examining a limited number of additional projects • Limited number of case studies

14. Environment - Green growth • Why this evaluation? "Green growth" has become a popular concept but it lacks clear and shared understanding • Questions: • Is there a widely accepted definition of "green growth"? • If not, what could be a definition for development policy purposes? • Methods • Literature review • Expert seminars • Possibly a very limited number of case studies

15. Energy Efficiency • Whythisevaluation? During the period 2007-2013 €6 billion has been dedicated to support energy (EE). This is a new area for many MA and it is important to evaluate achievements so far and identify conditions for success/failure • Questions: Are EE projects selected within an integrated approach? What are the achievements so far delivered by EE in buildings? In the case of residential buildings, to what extent have EE investments contributed to the alleviation of energy poverty? Have absorption patterns changed since 2009?

16. Energy Efficiency • Methods: • Literaturereview and in-depthanalysis: 10 casestudies on OPs investingthemost in EE • Financial instruments: Analysis in 3 or 4 selectedcases. • Expert meetings, seminarwith MS • Questions: • Suggestionsforacademicexperts/ existingliterature? • Evaluations availableorplanned in 2013-2015? Contact in REGIO: Patty Simou 16

17. Tourism and Culture • Volumes: Nearly € 12 billionallocated 2007-13 • Whythis evaluation? Wide variationoffundingacross MS. Lack ofevidence on theaddedvalueofinvestment in thesepolicyareas. • Questions: What is the evidence in terms of growth and jobs delivered by ERDF support in these policy fields? (could include the issue of social inclusion); To what extent are the projects financially sustainable? • Methods: Useofcategorisationdatato ID theOpswithlargestallocations; mapping, asurvey of MAs to explore the focus and enquire the logic of intervention, case studies;

18. Tourism and Culture • Comments on questionsandapproach? • Anyvolunteersforcasestudies? • Whichofyourevaluationsyourecommend? Do you plan moreevaluations in 2013-2015? • Can yousuggestacademicexpertsandliterature? • Contact in REGIO: Samuele Dossi

19. Urban Development (including Social Infrastructures) • Whythisevaluation? : €11 billion ERDF co-financing projects within the territorial dimension theme, including more than €10 billion for integrated projects for urban and rural regeneration. • Questions: • What are the main themes addressed by integrated urban development strategies in OPs devoting a substantial financial allocation to this theme? • What are the objectives … local urban development "per se" or strategic territorial development objectives?

20. What is the evaluation evidence available on ERDF co-financed urban development strategies and interventions currently available? • How is integrated urban development organized within OPs? Degree of freedom allowed to localities/ municipalities? Degree of coordination / • What are the main rationales to justify the specific delivery mode? • Methods review of evaluation evidence in MS and studies carried out by EC, themetic seminar with MS, expert meetings, conceptual analysis;

21. Urban Development (including Social Infrastrutures) • Comments on questionsandapproach? • Whichofyourevaluationsyourecommend? Do you plan moreevaluations in 2013-2015? • Can yousuggestacademicexpertsandliterature? • Contact in REGIO: Samuele Dossi

22. European Territorial Co-operation • Why this evaluation? : - Over €7 billion allocated to ETC. • 70 ETC programmes; • Outputs and results of programmes not very clear. • Evaluation questions: For Cross-Border programmes: how has co-operation been enhanced, and what has been delivered via cooperation - in the fields of economic activity, environment and transport?

23. - For Interreg, URBACT, INTERACT: are there structures in place for knowledge capitalisation? What evidence is there that regions/partners/Member States are using this knowledge? - For TNC programmes: What evidence is there of the influence of TNC programmes on 'mainstream' programmes, for example via alignment of resources? • Methods - Review of quantitative and qualitative evidence contained in AIRs. Review of other evaluation evidence in MS and held by technical secretariats eg INTERACT. - Case studies of up to 4 regions, survey, interviews. - Thematic seminars with groups of MS, expert meetings.

24. European Territorial Co-operation Questionsfor Member States: • Anyevaluationsavailableorplanned in 2013-2015? • Can yousuggestacademicexpertsandliterature? • Anynominationsforprogrammestobeevaluated? • Contact in DG REGIO: Juliet Martinez

25. Geography of expenditure Whythisevaluation? to determine the regional breakdown of ERDF and CF at NUTS3 for each general category of expenditures. Questions: 1. What is the regional breakdown at NUTS3 level for each general category of expenditures financed from ERDF /CF? 2. Are there significant differences between payments and commitments data? Methods: Desk research, quantitative (especially statistical) analysis and reliability checks, samples, surveys, other sources of information outside of the monitoring system Contact in REGIO: Alida STAICU Cohesion Policy

26. Geography of expenditure- questions for the MS - • - Comments on questions and approach? • - Do you have NUTS 3 breakdown in your monitoring systems?

27. Macro-modelling and econometrics • Whythisevaluation? Need toestimateimpactsatthe regional and national levels. • Questions: WhatarethegrowthandjobsimpactsofCohesionPolicy? • Methods: 2 Macro-models (RHOMOLO and QUEST) aswellascounterfactualeconometricwork • Contact in REGIO: Daniel Mouqué

28. Delivery system • Whythis evaluation? Wide variationoffundingandperformanceacross MS. Delays arewidespread. • Questions: Reasonsforsuccessorfailure. Doescapacitybuildingmake sense? Would a concentrationof REGIO on problematic programmes make sense? Is a "lighter" approachtosmallerand well-run programmes appropriate? • Methods: Literaturereview, around 15 casestudies. Expert meetings, twoseminarswith MS • Contact in REGIO: Kai Stryczynski

29. Delivery system – questions for Member States • Comments on questionsandapproach? • Anyvolunteersforcasestudies? • Whichofyourevaluationsyourecommend? Do you plan moreevaluations in 2013-2015? • Can yousuggestacademicexpertsandliterature?

30. Please send yourcommentsbythe end ofJune • Regio-evaluation@ec.europa.eu