1 / 39

Monitoring of implementation progress of the recommendations plan resulting from evaluation

Monitoring of implementation progress of the recommendations plan resulting from evaluation. Dace Grūberte Deputy Director of EU Funds Department Head of Public Infrastructure Unit Monitoring Committee meeting 14 June 2007. Contents.

stacia
Télécharger la présentation

Monitoring of implementation progress of the recommendations plan resulting from evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Monitoring of implementation progress of the recommendations plan resulting from evaluation Dace Grūberte Deputy Director of EU Funds Department Head of Public Infrastructure Unit Monitoring Committee meeting14 June 2007

  2. Contents • Progress of technical assistance project Evaluation and research • Recommendation monitoring process • Recommendation implementation statistics • Recommendation implementation means • Evaluation reports in detail Managing Authority of structural funds – Ministry of Finance

  3. Progress of technical assistance project Evaluation and research (1) • 2 evaluations and 1 research has been completed since MC meeting in 2006: • Evaluation of EU structural funds coherence to national strategic documents for employment • Evaluation of horizontal priorities implementation • Research of current and future status of Analysis of training in agriculture, forestry and entrepreneurship for rural areas Managing Authority of structural funds – Ministry of Finance

  4. Progress of technical assistance project Evaluation and research (2) • 2007 activities: • Ex-ante evaluation of EU structural funds and Cohesion fund planning documents • In 2007 the research Labour market competitiveness of remuneration in institutions managing EU structural funds was launched(following the proposal from MC in June 2006) • Currently bids submitted for Evaluation of macroeconomic impact of EU funds are evaluated • 2 procurement announcements published in June: • Research Potential project applications in entrepreneurship and innovation activities of 2007-2013 programming period • Evaluation of expenditure eligibility check from efficiency and compliance perspective Managing Authority of structural funds – Ministry of Finance

  5. Progress of technical assistance project Evaluation and research (3) • Total project costs: 723 209 LVL • Contract between MoF and CFCA for 457 653 LVL • 31.5% of all project funding has been used by the end of 2006 • It is planned to spend 53.54% of all funding (including final ex-ante payment) by the middle of 2007 • Considering funding committed for on-going (procurement procedures under way) and future evaluations and researches, the remainder for further evaluation and research activities amounts to ~ 140 000 LVL • Costs for single evaluation or research are 7 000 LVL per activity, and up to 150 000 LVL (VAT included) for ex-ante evaluation  there is still possibility to implement ~ 5 evaluations or researches Managing Authority of structural funds – Ministry of Finance

  6. Recommendation monitoring process • MoF has compiled recommendations from evaluation reports and prepared or requested the respective line ministries to report on implementation of said recommendations • It is intended to update recommendation implementation status at least once in six months with TEAWG responsible for information coordination • Recommendation implementation plan has been updated before the MC meeting after review by TEAWG Managing Authority of structural funds – Ministry of Finance

  7. Evaluation recommendation statistics (1) • In total evaluators have made 378 recommendations Managing Authority of structural funds – Ministry of Finance

  8. Evaluation recommendation statistics (2) • Recommendations (10) of evaluation report Evaluation of the aid scheme guideline and project selection criteria compliance to SPD and Programme Complement within SPD sub-activity 2.2.1.2. Support to investment in business development in specially supported territories • implemented 100% • Recommendations of research of current situation analysis and future perspectives in the theme Analysis of training in agriculture, forestry and entrepreneurship for rural areas • implemented 100% • Recommendations (13) of evaluation of EU structural funds monitoring indicators system • implemented 61,54% • partially implemented/ relate to the next programming period 38,46% Managing Authority of structural funds – Ministry of Finance

  9. Evaluation recommendation statistics (3) • Recommendations (74) of SPD activity implementation evaluation • implemented 60,81% • partially implemented/ relate to the next programming period 18,92% • constantly 13,51% • not relevant 6,76% • Recommendations (56) of evaluation of EU structural funds programming documents and activities for 2004-2006 • implemented 50,88% • partially implemented/ relate to the next programming period 42,11% • not relevant 7,02% Managing Authority of structural funds – Ministry of Finance

  10. Evaluation recommendation statistics (4) • Recommendations (22) of SPD implementation system efficiency evaluation • implemented 40,91% • partially implemented/ relate to the next programming period 31,82% • constantly 22,73% • not relevant 4,55% • Recommendations (30) of EU structural funds selection criteria evaluation • implemented 40% • partially implemented/ relate to the next programming period 60% Managing Authority of structural funds – Ministry of Finance

  11. Evaluation recommendation statistics (5) • Recommendations (55) of evaluation of EU structural funds coherence to national strategic documents for employment policy • implemented 27,27% • partially implemented/ relate to the next programming period67,27% • constantly 3,64% • rot relevant 1,82% • Recommendations (13) of research Role of culture in Latvian economy and absorption of EU structural funds • implemented 23,08%, • partially implemented/ relate to the next programming period30,77% • constantly 46,15% Managing Authority of structural funds – Ministry of Finance

  12. Evaluation recommendation statistics (6) • Recommendations (37) of evaluation of EU structural funds impact on regional development in Latvia • implemented 16,22% • partially implemented/ relate to the next programming period 59,46% • constantly 13,51% • not relevant 10,81% • Recommendations (29) of horizontal priorities implementation • implemented 10,71% • partially implemented/ relate to the next programming period 89,29% Managing Authority of structural funds – Ministry of Finance

  13. Recommendation implementation means • Clarify conditions (e.g. criteria) in activity documents (regulations of CoM) • Develop programming documents (NSRF, OP, OPC) • Develop methodology guidelines (Methodology for development of project selection criteria) • Develop requirements for absorption of funds (law, regulations of CoM) • Organise conferences, seminars and public awareness campaigns, as well as involve target groups Managing Authority of structural funds – Ministry of Finance

  14. Evaluation report details • During its 9 November 2006 meeting MC proposed to review one or two evaluation reports in its next meetings • Reports included in this MC meeting are: • SPD implementation system efficiency evaluation • Evaluation of EU structural funds selection criteria • Evaluation reports and researches available at:http://www.esfondi.lv/page.php?id=302 Managing Authority of structural funds – Ministry of Finance

  15. Evaluation of the efficiency of the SPD implementation system Contractors: SIA „Ernst & Young Baltic” Contract: 17.02.2006.–17.07.2006. Amount (excl.VAT): 21 080,00 LVL

  16. Evaluation of the efficiency of the SPD implementation system – Objective of evaluation • To evaluate the overall efficiency of the SPD implementation system, involved institutions, as well as draw conclusions and practical recommendations to improve the implementation system Managing Authority of structural funds – Ministry of Finance

  17. Evaluation of the efficiency of the SPD implementation system– Evaluation methodology(1) System processes Efficiency indicators Speed Indicators Quality Resources Managing Authority of structural funds – Ministry of Finance

  18. Evaluation of the efficiency of the SPD implementation system–Evaluation methodology (2) • Speed (Ātrums) – characterises the amount of time necessary to perform various tasks of the system, measuring absolute values, as well as relative against plans. A critical SPD system implementation speed indicator is related to the financial progress (financial disbursements) ‘1’- deadlines met and progress observed in just 20% of cases ‘5’- deadlines met and progress observed in 100% of cases • Quality (Kvalitāte) – characterises the quality of processes and relevance to the anticipated result. Quality indicators include, e.g., number of justified refusals, number of complaints, number of modifications to documents etc. ‘1’- Unsatisfactory implementation of functions, a threat to SF implementation ‘5’- High quality performance of functions • Resources (Resursi) – characterises the utilisation efficiency of resources (human resources and financial resources) for the process. Measured as an amount of resources necessary to achieve a specific result etc. ‘1’- more than one half of institutions record resource utilisation efficiency differences of 80% ‘5’- homogenous efficiency rates Managing Authority of structural funds – Ministry of Finance

  19. Evaluation of the efficiency of the SPD implementation system–Scope of evaluation • Institutions evaluated • 1st level IBs , 2nd level IBs and GSM • Objects of data analysis • Defined processes • Defined measurable indicators • System performance evaluation period • System performance in 2004 and 2005 • Data collection and analysis – Feb 17 – May 26, 2006 Managing Authority of structural funds – Ministry of Finance

  20. Evaluation of the efficiency of the SPD implementation system– Main conclusions and recommendations (1) • Drafting implementation documents: • Conclusion: initial high-quality versions of implementation documents considerably delayed in both their drafting stage (by 2 months on average) and approval stage (by 2 months on average) • Recommendation: setting up a single monitoring system for planning and monitoring activity implementation, as well as a risk management system to identify and prevent critical delays Managing Authority of structural funds – Ministry of Finance

  21. Evaluation of the efficiency of the SPD implementation system–Main conclusions and reccomendations(2) • Selection of applications: • Conclusion: divided responsibility in drafting guidelines for selection and procurement procedures and a two-tiered system for grants • Recommendation: responsibility for guidelines should be vested with one IB, a coordination mechanism to be set up • Conclusion: insufficient customer service systems of intermediate bodies/agencies (previous experience of the institution) • Recommendation: improve customer service systems: categorising questions raised and systematic analysis to make improvements; customer-friendly documents and informational material (design, layout, structure); training customers Managing Authority of structural funds – Ministry of Finance

  22. Evaluation of the efficiency of the SPD implementation system–Main conclusions and reccomendations(3) • Appraisal of applications: • Conclusion: distributed responsibilities to appraise applications • Recommendation: concentration of the appraisal task within one institution • Conclusion: excessive appraisal period (12 months). Guarantee funds of applications extend the budget cycle, validity of guarantees and certifications of third parties, reliability of project cost forecasts. • Recommendation: shortening appraisal periods (6 months). In addition – reducing the amount of documents to be submitted, tapping in alternative sources of verification, computerised appraisal. Substitution verifications of pre-project information with checks during project implementation. Managing Authority of structural funds – Ministry of Finance

  23. Evaluation of the efficiency of the SPD implementation system–Main conclusions and recommendations (4) • Contracting/amendments: • Conclusion: excessive contracting procedure (on average - 50 days for national programmes and 260 days for open calls) and the long period of time required for contractual amendments (60-70 days on average) • Recommendation: to limit the contracting period to 2 months and extend powers of the 2nd level IBs and GSMs to approve contractual amendments without prior consent of the ST or 1st level IBs, but rather informing the MA and 1st level IB on planned amendments, obtaining their approval on substantial amendments in writing, requesting approval when in doubt Managing Authority of structural funds – Ministry of Finance

  24. Evaluation of the efficiency of the SPD implementation system–Main conclusions and reccomendations(5) • Processing SF payment claims and declaration of expenditure: • Conclusion: complicated requirements and format for SF payment claims, as well as excessive reviewing phase (on average, 40 days and 2-3 times) • Recommendation: improved SF payment claim documents – simplifying tables, minimising information. Risk-based checking of payment claims – checked the total amount in 100% of cases, checking the remaining data on a sample basis Managing Authority of structural funds – Ministry of Finance

  25. Evaluation of the efficiency of the SPD implementation system–Main conclusions and reccomendations(6) • Monitoring and control measures: • Conclusion: partial storage of information on checks – no information accumulated on serious deficiencies identified in desk-top checks of reports by final beneficiaries and on-the-spot checks • Recommendation: a single storage of information on the results of all checks, detailing deficiencies and irregularities identified Managing Authority of structural funds – Ministry of Finance

  26. Evaluation of the efficiency of the SPD implementation system–Main conclusions and recommendations (7) • Overall evaluation of the system • Speed • Delays in drafting implementation documents • Delays in appraising applications • Delays in amending contracts • Quality • Need to improve the customer service system • Resources • Control and improvement of contributions made by human resources Managing Authority of structural funds – Ministry of Finance

  27. Evaluation of selection criteria for EU structural funds Contractor:  SIA „PricewaterhouseCoopers” Contract:  10.02.2006.–11.07.2006. Amount (excl.VAT): 18 880,00 LVL

  28. Evaluation of selection criteria for EU structural funds–Evaluation objective • Qualitative thematic evaluation of selection criteria used for EU SF project in order to determine their efficiency and draw conclusions and make recommendations for improving efficiency • To evaluate the following aspects of administrative eligibility, quality and specific appraisal criteria: • efficiency; • compliance with the SPD objectives; • justification and applicability; • Draw conclusions and make practical recommendations. Managing Authority of structural funds – Ministry of Finance

  29. Evaluation of selection criteria for EU structural funds–Evaluation methodology • Data sources: • Analysis of documents, • Personal and group interviews with staff members of involved institutions, • Discussions with foreign experts. Managing Authority of structural funds – Ministry of Finance

  30. Evaluation of selection criteria for EU structural funds–Scope of evaluation The project sample included: • Four 1st level IBs (MoE, MoES, MoW, MoA) • Two 2nd level IBs and one GSM (RSS, CFCA, LIDA) • Representatives of the MA, MC and social partners • The project also involved an analysis of a sub-set of specific selection criteria of 31 activities Managing Authority of structural funds – Ministry of Finance

  31. Evaluation of selection criteria for EU structural funds–Main findings and recommendations (1) • Developing selection criteria: • Finding: Criteria aimed at increased funding absorption speed rather than achieving objectives. • Finding: In some cases criteria failed to identify projects with the highest potential of solving a specific problem (e.g.employment issues). • Recommendation: collect experience of the current programming period on the impact of criteria on achieving SPD goals. • Recommendation: measures to exchange information between project applicants and bodies setting criteria Managing Authority of structural funds – Ministry of Finance

  32. Evaluation of selection criteria for EU structural funds–Main findings and reccomendations(2) • Developing selection criteria: • Finding: Insufficient time allocated for developing selection criteria (30 d), irrelevant development methods. • Finding: Unclearly defined principles for setting criteria (appraisal scale, types of criteria, how broad range of criteria). • Recommendation: single methodology for the next programming period for developing selection criteria (types of criteria, development mechanisms, scale) Managing Authority of structural funds – Ministry of Finance

  33. Evaluation of selection criteria for EU structural funds–Main findings and reccomendations(3) • Developing selection criteria: • Finding: eligibility and quality criteria not efficient for selection in all cases under SPD activities. • Finding: selection criteria not discussed with sectoral professionals, social partners (result – criteria fail to identify the best performing projects to achieve objectives). • Recommendation: When developing criteria, needs and possibilities of the target groups should be considered – better involvement of social partners required Managing Authority of structural funds – Ministry of Finance

  34. Evaluation of selection criteria for EU structural funds–Main findings and reccomendations(4) • Content of selection criteria: • Finding: administrative eligibility criteria are the same for all target groups, thus they are difficult to appraise under specific activities. • Finding: The PC provides that scores may be used for specific criteria, however, fails to detail scales for eligibility and quality criteria. • Finding: Unclear borderline between quality and specific criteria (sometimes even under one evaluation system, e.g., MoE activities). • Recommendation: Assuming the next programming period will have only two implementation schemes (open calls and NP projects), a minimum set of criteria recommended for both implementation schemes, e.g., minimum requirements for a beneficiary and for a project. A set of specific criteria should be designed for projects under each specific activity. Managing Authority of structural funds – Ministry of Finance

  35. Evaluation of selection criteria for EU structural funds–Main findings and reccomendations(5) • Content of selection criteria: • Finding: In several cases selection criteria fail to address needs of final beneficiaries. • Finding: In several cases criteria prevent selecting projects that comply with NP requirements (incompliance with SPD and NP objectives). • Recommendation: Possibilities of beneficiaries and result indicators should be considered more carefully when developing criteria • Recommendation: Selection criteria should be developed for NP projects, applicable to project selection Managing Authority of structural funds – Ministry of Finance

  36. Evaluation of selection criteria for EU structural funds–Main findings and reccomendations(6) • Content of selection criteria : • Finding: several sets of criteria difficult to appraise against and require large resources • Recommendation: different sets of criteria for large and small projects Managing Authority of structural funds – Ministry of Finance

  37. Evaluation of selection criteria for EU structural funds–Main findings and reccomendations(7) • Content of selection criteria : • Finding: Formal evaluation of compliance with horizontal priorities (equal opportunities, sustainable development, regional development, information society), inter alia, due to lack of understanding of contents of the priority and its appraisal mechanisms. • Recommendation: Compliance with horizontal objectives should be one of selection criteria • Recommendation: minimum values to achieve under horizontal priorities should be set in the SPD, later to be transformed into selection criteria Managing Authority of structural funds – Ministry of Finance

  38. Evaluation of selection criteria for EU structural funds–Main findings and recommendations (8) • Content of selection criteria : • Finding: Unclearly formulated and justified criteria. Beneficiaries require consultants to draft applications satisfying such criteria. • Finding: 2nd level IBs revealed in interviews that selection criteria are not always easy to use in the appraisal process resulting in additional recourses used for making explanations. • Recommendation: Explanatory notes needed for selection criteria increasing understanding among applicants • Recommendation: Developing self-explanatory selection criteria thus saving resources on explanations and selection burden. Managing Authority of structural funds – Ministry of Finance

  39. Thank you!

More Related