Rome, july 5, 2006

1. Observing project implementation and conducting project analysis (UVER) Presentation by Luigi Guerci Rome, july 5, 2006

2. Public Investments Verification Unit (UVER) • The Unit performs controls during implementation and after completion of projects with: • On site control • Desk monitoring

3. Public Investments Verification Unit (UVER) During the implementation On site control The activity consists of inspections on the construction sites and periodic contacts with project managers to find out potential problems and suggest solutions.

4. Public Investments Verification Unit (UVER) During the implementation On site control Provides support and technical assistance to selectedimplementing administrations to accelerate the execution of projects.

5. Public Investments Verification Unit (UVER) During the implementation • Single project solution • facilitation on project achievement • raising up potential problems • speed up procedures • General solution • feed back on general procedures • suggestion of changes in laws

6. Public Investments Verification Unit (UVER) During the implementation • Projects are usually grouped with regard • to the financial sources e.g. : • National additional resourses • EU funds • Legge Obiettivo funds

7. Public Investments Verification Unit (UVER) During the implementation • Framework programme agreements consist of: • more than 60 bil € • more than 10.000 projects

8. Public Investments Verification Unit (UVER) During the implementation Framework programme agreements • All projects are verified with a monitoring procedure (performed by SPTSI) • 450 projects (selected among the most critical and important) were verified on site

9. Public Investments Verification Unit (UVER) • Checking 450 projects we found out some common operational impediments as: • financial problems (flows and coverage) • design deficiency • failure of the permission procedure • inability of the implementing • administration

10. Public Investments Verification Unit (UVER) During the implementation - Effect of the operational impediments Some results

11. Public Investments Verification Unit (UVER) During the implementation In the period after the preliminary design (which usualy represents the inclusion of the project in the Framework Programme Agreement) and before the start of the construction work, there are nearly no expenditures

12. Public Investments Verification Unit (UVER) After the preliminary design and before the construction

13. Public Investments Verification Unit (UVER) Risk Financial programmes can be ineffective with the assigned resourses unemployed for 2 years (average).

14. Public Investments Verification Unit (UVER) Feed back The ex ante evaluation must consider the real capacity of a project of being implemented in a specific period.

15. Public Investments Verification Unit (UVER) After the completion of a project Effectiveness Evaluation (in progress activity) Why ? • To answer to the questions • “ Was it worth it?” • “Is the service offered congrous with the preset target ?” • “Are users satisfied?”

16. Public Investments Verification Unit (UVER) • GENERAL GOALS • Disseminating effectiveness evaluation culture (self monitoring) • Improve the governance of projects • Building a participatory process of evaluation (involvement of key stakeholders at all stages) • Create a public interest disseminating evaluation results • Providing guidelines for decision making by selecting best practices and quantitative indicators with their targets

17. Public Investments Verification Unit (UVER) After the completion of a project Effectiveness Evaluation (in progress acitivity) Verify and measure actual product and impact of concluded projects in terms of improved or added services to beneficiary communities.

18. Public Investments Verification Unit (UVER) After the completion of a project Effectiveness Evaluation (in progress acitivity) Check of physical realization, direct results and indirect results with quantitative indicators.

19. Public Investments Verification Unit (UVER) HOW DO WE DO Apply a standardized but flexible conceptual framework through the following stages ü Desk reviews of project related documents ü Brainstorming with key stakeholders (objectives and needs identification) ü Selection of performance indicators (process, outputs, outcomes, impacts) ü Data collection (administrative and statistical sources, structured interviews, ad hoc questionnaires) ü Fill up standardized projects forms containing summary information ü Presentation and dissemination of evaluation results in specific and general reports

20. Public Investments Verification Unit (UVER) After the completion of a project Effectiveness Evaluation (in progress acitivity) Project selection from a group of 310 called “Completamenti” (1,8 bil €). It is a group of non completed projects, that in 1999, we decided to carry on after an ex ante evaluation made by UVAL.

21. Public Investments Verification Unit (UVER) After the completion of a project Effectiveness Evaluation (in progress acitivity)

22. Public Investments Verification Unit (UVER) After the completion of a project Effectiveness Evaluation (in progress acitivity)

23. Public Investments Verification Unit (UVER) After the completion of a project Effectiveness Evaluation (in progress acitivity) For the uneffective projects the department suggested Cipe to deduct an equal amount of funds from the next assignment to the administrations involved.

24. Public Investments Verification Unit (UVER) Quality results • General achievement of results related to the construction • Lack of targets related to operational results and impact • Presence of gaps between construction and operation

25. Public Investments Verification Unit (UVER) Basic Ideas • Evaluate the real capacity of a project of being implemented in a specific period • Keep control on the main step of a project, especially during the period after the preliminary design and before the start of construction (with nearly no expenditures) • Fill the gap between the physical implementation and the working phase thinking from the beginning the services the project will provide and finding indicators to qualify the targets to be achieved