In collaboration with

1. Development of integrated project and financial management tools in conformity with the 2003 FPA Design an “evaluable project” In collaboration with

2. ECHO and the evaluability

3. The Commission shall carry out evaluations of the humanitarian operations financed (art. 22.1. Provisions; art. 8 CG) Carrying out of the mission in co-ordinated way by ECHO and the NGO The Commission shall make the draft of the report available to the NGO for eventual comments If agreed on, the comments of the NGO shall be inserted in the Final Report The NGOs must make ex ante/ex post evaluations where possible (art. 22.2 Provisions) Include evaluation cost in budget Keep a constant monitoring Insert generic ToR in Single Form Present specific ToR to ECHO before the mission with Curriculum Vitae of evaluators Present Evaluation Report with Final Report ECHO and the evaluation

4. Introduces the principle of responsability towards beneficiary, donors and partners Control of results based on the promotion of best practice Obligatory insertion PCM, LF and indicators (project forms) Application of HOLIS program – Standard Operative Procedures Introduces the principle of measurability and monitorability Allocation of aid in a balanced and verifiable way for beneficiary Possibility of “de-selection” of under-performing partners (monitoring & evaluation) ACCOUNTABILITY (From ECHO Aid Strategy 2003)

5. Control of results using “best practice” approach Application of HOLIS program – Standard Operative Procedures Obligatory insertion LF and indicators Possibility of Partner “de-selection” Conceptual work on a methodology to allocate aid in a more balanced way per beneficiary has progressed in 2002, highlighting, however, problems with data quality. Data availability and quality will therefore be addressed as a priority in the 2003 HOLIS work programme. Allocation of aid in balanced way for beneficiary Adoption of measures focusing on control of results using a “best practice” approach. Capacity building through the dissemination of best practices. The establishment of final Standard Operating Procedures for crisis response in humanitarian emergencies will also contribute to ECHO´s quality-based approach and its operational management capacity. In order to facilitate the monitoring of results, the objectives of an operation and the performance indicators applicable to measure achievements will become constituent parts of the new project proposal standard forms. The concept of quality partnership also involves a regular monitoring of performance, with the concrete possibility of “de-selection” of under-performing partners. ACCOUNTABILITY (From ECHO Aid Strategy 2003)

6. Obligation to draw up a Log frame Project presentation: mandatory LF of maximum 3 pages The partner shall guarantee an internal monitoring of the project, informing ECHO through the Interim Report Accountability of ECHO on money spent Final Report: the LF is adapted in light of the completed operation The base of the monitoring system is given by the Work Plan Lessons learned Evaluation Monitoring Interim Report: the LF is adapted in light of what has been done ACCOUNTABILITY (From ECHO Manual Project Cycle Management 2003)

7. The “evaluable” quality

8. CLEARNESS Precise identification of the beneficiaries and the objective Identify needs, problems, priorities and risks Detailed analysis – elements defined (better if supported by Report) TRUSTWORTHINESS Expected results in relation to defined indicators (SMART) Realistic planning – efficient expense organization RESPONSABILITY Continuous monitoring, ability to adapt, noover budgeting Transparency of procedure, correctness of work Attention to priority themes (Gender, Human Rights) Quality control CROSSOVER Visibility and General Protocols and safety details Key points expressed by ECHO in “Qualitative Approach” The Project (the Operation)

9. … Evaluation Approval Identification Preparation • PCM phases with emphasis on qualitative aspects of the construction of the proposal: • Problem analysis, single out objectives/strategies • Work plan, monitoring • Evaluation, best practices … Liquidation • Phases relative to project iter foreseen by GC and documents: • Responsability and maturity towards ECHO • Procedure, statements and mandatory documents … Closure • Real definition phases and project management: • Predisposition of statements • Reference system (sinergies) • Supervision e monitoring Project phases Passages project Logic Feasability Passages ECHO procedures Comunication Passages Internal Projects Start

10. Focal points The project cycle shall keep in account the various needs according to points of view which must be considered: • Respect the orthodoxy of the project • Respect the ECHO procedures • Respect internal HO procedures

11. The project implicates Focal points in practice Qualitative approach From the proposal to the end focus on method Evaluation and Monitoring Demonstrate know how and knowledge of procedures Preventive approach Plan the work from the initial phase

12. The basic idea is… Qualitative approach Evaluation and monitoring The “perfect” Project perhaps Preventive approach

13. Quality of Reports

14. The “Five points” Qualitative approachbasic elements • Relevance • Effectiveness • Efficiency • Impact • Sustainability

15. Stakeholder Analysis Qualitative approachbasic elements • Potential of beneficiaries (number, category, types), institutions, local groups, etc… • Limits (obstacles) • Interests • Possible role of stakeholders in the project

16. StakeholdersAnalysis tool

17. Problem and needs analysis Qualitative approachbasic elements • Identify general problems • Identify particular problems and needs • Identify needs of beneficiaries (by categories) • Correlation and articulation of main elements (logic of analysis)

18. Macro contest shown by ECHO/CE Intervention focus Problem Tree

19. Analysis of objectives Qualitative approachbasic elements • Define (strategic) general objectives (connecting strategy EU and HO) • Identify quantitative and qualitative beneficiaries • Identify objective of the opertion (services given to target group) • Define observtion criteria (type of need) and selection (inclusion-exclusion)

20. Objective Tree

21. Analysis of results and activities Qualitative approachbasic elements • Define general framework of services (expected results in line with budget) • Identify activities (description, management, human resources, times, detailed description of costs) • General framework of main points relative to implementation (purchase/tender, safety, visibility, monitoring plan)

22. Analysis of risks and assumptions Qualitative approachbasic elements • Identify (evaluate) risks inherent to the action • Identify (evaluate) assumptions • Identify mitigation meausures

23. Imp! Imp! Imp! Imp! Refer to the EC/ECHO strategic framework Log Frame Cannot be modified, refer to ECHO decision Consider costs and organization of a monitoring plan Refer to the budget lines

24. Workflow

25. Quality factors Qualitative approachbasic elements • Ownership by beneficiaries • Policy support (positive/negative) • Appropriate technology • Socio-cultural issues • Equality of all (especially the most vulnerable group) • Institutional and management capacity • Co-ordination between the different actors

26. Cross cutting issues Qualitative approachbasic elements • LRRD (towards self sufficiency) • Gender (roles, impact of project, women’s role as development factor) • Human Rights (impact on HR) • Safety • Visibility

27. Indicators and sources of verification Qualitative approachbasic elements • Clear definition (quantity, relevance, thresholds, minimum standards) • Feasability of control • Relevance of project variable observed

28. Define and plan steps; • General indication of operative formalities and more critical points (personnel, tender, means, safety); • Experience, ideas, strategies. • Indication on UE/ECHO strategies on regional, national and local levels. • PCM synthesis (with attention to preliminary analysis and log frame construction); • Terms and synthesis of criteria for defining, monitoring and evaluating the projects. • Defines procedures, times, and focal elements of work; • Elegible expences, purchases, visibility, responsability. • Indicates main variables for the defining, monitoring and evaluation of the activity (performance e benchmark); • Formation of log frame. Reference Documents Data Base and HO praxis Country Strategy Paper ECHO/UE PCM ECHO and Solint manual FPA documents Sphere Manual

29. Collects preliminary data; • Meetings with key actors; • Meetings (Focus Group) with privileged actors and stakeholdersanalysis. • Clear identification of beneficiaries and project components; • Clear identification of critical points; • Coherent base of project logic (attention to objectives, not activity) • Lists minimum elements for defining the proposal. • Defines project logic and analysis (deviation from plan, hiring, indicators); • List key elements (objectives, results activity and indicators). • Realistic analysis of times; • Initial Assessment and verification of responsability and focal moments of intervention. Instruments and methods Rapid survey Problem Tree Chek List Log frame Work-flow (activity plan)

30. The end