WELCOME to the webinar “The Professionalization of Evaluation”

1. WELCOME to the webinar“The Professionalization of Evaluation” This Live Webinar will start at 9:30 AM, New York time. All microphones & webcams are disabled and we will only enable microphones during the Q&A portion. Therefore, you will not hear any sound/noise till the beginning of the webinar.

2. The Professionalization of Evaluation Live Webinar 30th November 2010 DevInfo

3. This series of webinars are based on the book published by UNICEF in partnership with key international institutions • Authors: 40 global evaluation leaders • Partnership: UNICEF, WB, UNDP, WFP, UNIFEM, IDEAS, IOCE, DevInfo

4. Available for free download at www.mymande.org

5. Webinars on National evaluation capacity development

6. Webinars on National evaluation capacity development

7. Webinars on Emerging Practices on Development Evaluation:

8. The recording of the previous webinars are available at www.mymande.org 1 2

9. The recording of the previous webinars are available at www.mymande.org

10. The recording of the previous webinars are available at www.mymande.org

11. Professionalization of evaluation • Why to professionalize evaluation? • What do we mean with “professionalization” of evaluation? • How to “professionalize” evaluation, taking into consideration different contexts, risks and opportunities?

12. Keynote Speakers Linda G. MORRA IMAS Advisor for evaluation capacity development, The World Bank, and Co-Director, International Program for Development Evaluation Training (IPDET). Robert PICCIOTTO, Visiting Professor, King’s College, London and former Director General, Evaluation, the World Bank. Jean Serge QUESNEL Senior Facilitator, UNSSC, Adjunct Professor, ENAP Quebec and Carleton University, and former Director of Evaluation at UNICEF, IADB and CIDA.

13. Agenda • 9h30 – 9h35 Welcome and introduction • Marco Segone, Systemic management, UNICEF Evaluation Office • 9h35 – 9h50 Linda G. Morra Imas, Advisor for evaluation capacity development, The World Bank, and Co-Director, International Program for Development Evaluation Training (IPDET) • 9h50 – 10h05 Robert Picciotto, Visiting Professor, King’s College, London and former Director General, Evaluation, the World Bank • 10h05 – 10h20 Jean Serge Quesnel, Senior Facilitator, UNSSC, Adjunct Professor, ENAP Québec and Carleton University, and former Director of Evaluation at UNICEF, IADB and CIDA • 10h20 – 10h50 Questions and Answers • 10h50 – 11h00 Wrap-up • Marco Segone

14. Questions and Answers 1 3 2

15. Linda Morra Imas World Bank Advisor & Consultant Co-Director, IPDET IDEAS Board of Directors 30 November 2010 “Professionalizing Development Evaluators: Travelling the Road to Results"

16. L. Morra Imas 30 NOV 2010 WHY DO IT?PROFESSIONALISM

17. L. Morra Imas 30 NOV 2010 Who Is Diving In? • National & Regional Evaluation Associations • Universities • Development Organizations • Standard-Setting Organizations • IDEAS Global Competencies

18. L. Morra Imas 30 NOV 2010 Bottom Line: Time has come • Evaluation competencies are here to stay • Issues can be addressed

19. L. Morra Imas 30 NOV 2010 But…how many evaluators work only in a single context?

20. L. Morra Imas 30 NOV 2010 A Call for a Global Standard • Premise • Core competencies exist for evaluation professionals-- across organizations, nations, and regions; having strong evaluation skills is paramount-- add local requirements

21. L. Morra Imas 30 NOV 2010 IDEAS Initiative • Origin: IDEAS Board Meeting July 2008 • Volunteer IDEAS committee formed with great interest- 60 volunteers • Membership survey conducted and results presented at 2009 Global Assembly • Initiative endorsed and expanded at GA • Draft competencies presented Sept. 2010 • Membership vote upcoming 2011 Global Assembly

22. INVITATION TO REVIEW AND COMMENT • Linda Morra Imas <lindag1000@aol.com> International Development Evaluation Association www.ideas-int.org L. Morra Imas 30 NOV 2010

23. L. Morra Imas 30 NOV 2010 Finally, consider… • Is recognition of evaluation as a specific discipline and its stature more likely to increase with global competencies?

24. Robert Picciotto Evaluation: a profession?

25. Evaluation internationalization demands professionalization • Canada has taken steps towards evaluators’ accreditation but no similar action is currently on evaluation associations’ agendas within Europe or the developing world • Yet evaluation is now called upon to carry out increasingly complex regional, multi-country and development evaluations across borders • This calls for global upgrading of evaluation skills and harmonization of competency standards – hallmarks of professionalization

26. Expert opinion about evaluation professionalization is divided. • For Michael Patton evaluation is already “a demanding and challenging profession” • Ernest House concurs: he views evaluation as a “new profession” • On the other hand, Lincoln calls evaluation “a fledging profession” • Rossi, Lipsey and Freeman disagree: “evaluation is not a profession at all”

27. Yet, evaluation has matured as a knowledge occupation… • Evaluation is now a discipline in its own right since it has acquired: • a agreed well defined body of expert knowledge • a recognized set of specialized skills • broadly ethical guidelines • Evaluation is also a trans-discipline: its concepts and tools support the social sciences • The demand for high quality, independent evaluation is growing rapidly around the world

28. … and evaluators display a growing sense of self-identity and collegiality. • The “paradigm wars” are over: advocates of qualitative methods agree that quantitative methods complement qualitative methods and vice versa • National and regional evaluation guidelines share: • a commitment to systematic inquiry • a respect for diverse peoples and cultures • an eagerness to achieve results • an unwavering public interest orientation

29. Unfortunately evaluation independence is under chronic threat • Absent the self management regime of a profession, most evaluators do not enjoy adequate autonomy or independence: • Decision makers in the public, private and voluntary sectors tend to view evaluation as a management tool • Evaluators within organizations are rarely protected from external or internal pressure • Evaluation consultants are fee dependent and subject to the whims of commissioners

30. What is professionalism? • In late Middle English usage a profession is: • “a declaration of belief in and obedience to a religion” or “a vow made by a person entering a religious order” • In contemporary usage a profession is: • “any occupation as a means of earning a living”. • “a body of qualified persons in a specific occupation or field” • “ a vocation, a calling, esp. one requiring advanced knowledge or training in some branch of learning or science”

31. But there is more to it than that: the professions enjoy special privileges… • a franchise to operate with autonomy in the public interest • significant influence over clients and society • specialized knowledge and expert practice honed by experience • esoteric or mystifying language that excludes the uninitiated • access to specialized education and training programs • membership in associations that promulgate and enforce standards • plentiful job prospects • good remuneration

32. … and professionalism would put order in a chaotic market place • Much progress has been made, at least at the national level, towards agreed evaluation guidelines and competency standards • Yet, quacks and amateurs feel free to use the evaluator title and to market their services • This is incompatible with the increasingly demanding quality requirements evinced by decision makers and evaluation commissioners all over the world

33. This underlies the rationale for controlled designation of evaluators • Evaluators certified by a legitimate body have demonstrated the competency to perform to a specified standard of competency • Two approaches to competencies: • Input based models (Germany, UKES and EES) • Outcome based model (CES) • A first step consists of accreditation of education courses and training programs

34. Reasonable controls over entry would pave the way for professionalization • The history of professions demonstrates that prestige and influence would result: • A recognized place in the upper regions of the occupational ladder • High and rising demand for services • Substantial monetary rewards • Public respectability

35. But this is not automatic: professionalization must be earned • It calls for specific actions by the occupational group so that public trust is gained through: • A credible body of advanced knowledge and proven practices • Ethical safeguards • Verifiable competencies • An effective self management regime • Reliable professional credentials

36. The rewards to society would be large… • The professions are critical to the effective working of modern economies • They act as counterweight to the coercive power of the bureaucratic state and the tyrannies of private interests • A franchise to practice in the public interest implies safeguards that distinguish professionals from amateurs and charlatans

37. …but the risks are substantial and must be managed • Monopolistic practices: • over-dependency of vulnerable consumers • supply of unnecessary services • unwarranted privileges • prohibitive fees • De-professionalization due to: • Misguided professional governance • Coercive state intervention • Subversion or capture by private interests (e.g. funding with strings)

38. Are we a profession? Not yet… • We exercise limited influence • We are still few in numbers and receive modest monetary rewards • Mixed quality of services • Scarcity of education and training • Slow convergence of standards • Limited autonomy • Meager progress towards accreditation

39. Has the time come? • Both evaluation and professionalism seek to shelter their adherents from vested interests (independence) • Both value work quality ahead of narrow efficiency considerations (excellence) • Both put the general welfare ahead of economic gain (public interest) • Equally, professionalism needs evaluation for improved learning and increased accountability

40. What specifically is to be done? • Increased supply of high quality education and training • Accelerated harmonization of ethical, quality and competency standards • Increased professional autonomy • Accreditation of training programs • Designation of evaluators

41. Do you agree with this diagnostic? • If you don’t please explain where I went wrong • If you do, would you endorse your evaluation association adopting a comprehensive professionalization agenda? • Would the impact of professionalization on your own activities as an evaluator be positive or not? • Would you be prepared to volunteer to make professionalization happen?

42. Jean Serge Quesnel 30 November 2010 Role of Evaluation Associations and Networks in the Professionalization of Evaluation

43. Evaluation Associations/Networks • Some 111 Evaluation Associations/Networks presently exists in the world. • Most were created since year 2000 • This exponential movement is in itself a strong indication of the need for systematizing and professionalizing the discipline of evaluation

44. International Evaluation Groups

45. Regional & National Associations & Networks 2010 International RFÉ OICE IDEAS IPEN EES NESE ReLAC AfrEA AES Regional Sub-regional ACE CoEsa National 14 countries 8 countries 41 countries 17 countries 14 countries AEA , CES Sub-national SWEP SQEP, SEA NWEA, WREN

46. Evolving roles of Evaluation Networks & Associations • The following slides present four generations of Evaluation Networks and Associations • The roles that Evaluation Networks and Associations play in support of the professionalization of evaluation vary according to the level of their respective institutional maturity.

47. 1st Generation • Aims at creating an awareness • Rallies providers and users of evaluation • Creates a community of evaluophiles • Facilitates professional exchanges • Offers informal apprenticeship

48. 2nd Generation • Demystifies evaluation • Fosters harmonization of concepts et tools • Allows an ownership of evaluation approaches and techniques • Offers basic training in evaluation • Rallies a community of practitioners and users of evaluation

49. 3rd Generation • Promotes the good use of the evaluation function via the sharing of real life experiences • Fosters the quality of evaluation products • Facilitates working groups on new themes offering a challenge to evaluation practice (R&D) • Provides methodological guides • Sets norms and standards • Offers specialized training in evaluation

50. 4th Generation • Assumes a normative role • Sets a code of conduct for evaluators • Identifies professional competencies • Offers a competency-based training program • Rules and bestows professional designations