PM E

1. PME This presentation looks specifically at evaluation. It is based on the positioning document validated by the Programme Board of Directors on 4th June 2004 Double-click on the icon to open this positioning document : Version 1– July 2004

2. Contents • Our external evaluation practice. • Why evaluate? • Who decides to evaluate? • When to evaluate? • Who evaluates ? • What to evaluate? • How to evaluate?

3. Our external evaluation practice • Little mobilisation of external expertise in relation to the number of projects : 7 external evaluations in 2001, 10 in 2002, 8 in 2003. • More external evaluations in the former ANS fields, in particular in rural development (9 external evaluations out of the25 identified). • A concentration on certain programmes and projects.

4. Why evaluate? • To analyse an operation, • To give account, • To learn from experience, • To contribute towards social change.

5. Who decides to evaluate? • Handicap International • The funding body(ies) • The partner(s)

6. When to evaluate? • At the implementation stage of the operation, at the end of the operation, at a later date. • The resources are identified during the co-development stage of the project. • The system is defined in detail at thestart of the project. • Any project of 3 years’ duration or more should be evaluated.

7. Who evaluates? • One person or a team of people, internal or external. • If a project of three years’duration or more is only to be evaluated once, this evaluation will be carried out by an external consultant. • To avoid being both judge and judged, an internal evaluator must not have been involved in the development or implementation of the project.

8. What to evaluate? - One or several projects or one or several programmes, - A partnership, - A dynamic, a process, - The effects, the impact of an operation, - An approach, a concept, - A structure.

9. How to evaluate? 1 - Decision to plan an evaluation 2 – Drawing up of the terms of reference 3 – Selection of the evaluator(s) 4 – Definition of the method 5 – Preparation of the mission 6 – Completion of the mission in the field 7 –Writing of the report 8 –Presentationof the report 9 – Diffusion of the report 10 –Follow up of the recommendations 11 – Valorisation of the report 12 – Impact of the evaluation

10. A few quality criteria • Impartiality & independence of the evaluators. • Credibility & transparency of the evaluators. • Utility of the evaluation, and the way it is run, in decision-making. • Participation of the parties involved in the project so as to strengthen their capacity in the field of evaluation, but mainly to make it easier for them to appropriate the lessons drawn from the exercise.

11. Bibliography COTA, 2002, Organiser l’ évaluation d’une action de développement dans le sud,[Organising the evaluation of an operation in the south] 82p. Feuerstein Marie-Thérèse, 1986, Partners in evaluation – Evaluating development and community programmes with participants, Macmaillan publishers, TALC, 194p. Heeren Nicolas, 2002, Evaluation, a management tool, Handicap International, 10 p. Heeren Nicolas & Villeval Philippe, 2002, Une pratique et une culture de l’évaluation qui permettent une conduite effective du changement [A practice and culture of evaluation for the effective management of change], Contribution to the seminar on 16th and 17th December 2002 « Les pratiques d’évaluation des acteurs français du développement et du l’humanitaire [Evaluation practices of French development and humanitarian stakeholders] », MAE/HI, Lyons, France, 3p. Graugnard Gilbert & Heeren Nicolas, 1999, Prise en compte de l’impact et construction d’indicateurs d’impact [Consideration of the impact and construction of impact indicators], F3E/ CIEDEL, 50p. – Document available on F3E’s website. Guéneau Marie-Christine & Beaudoux Etienne, 1996, L’évaluation un outil au service de l’action [Evaluation, a tool to assist the activity, F3E/ IRAM, 71p. –. Document available on F3E’s website Save the children, 2003, Toolkits – a practical guide to planning, monitoring evaluation and impact assessment, second edition, 341 p.

12. END