Planning an Effective Peer Review DISCUSSION POINTS

1. Planning an Effective Peer ReviewDISCUSSION POINTS Addis Ababa, 20-21 February 2007

2. Introduction & Welcome

3. Kojo Busia, UNECA

4. Peer Learning Objectives • Forum for NFPs to learn from pioneer countries, interact, exchange ideas, peer learning • Offer time for practitioners to share best practices and lessons • Help develop plans for technically sound research, public involvement and comprehensive POAs • Reflect on both theory and practical aspects of APRM processes • Opportunity to exchange views, share country experiences on technical and administrative challenges of the CSAR process SESSION 1

5. Ahmed Mohiddin

6. Workshop Overview:Key Messages • Why do we need APRM? • What are the potential benefits? • APRM is complex, so understand how it works, anticipate challenges, manage process carefully • It takes a lot of work, time, staff, money, political will – but it can be done well by all countries! • Undertake the best APRM process yet • Learn from the pioneers • Innovate • Ensure process is rigorous, transparent, consultative and POA really makes a difference SESSION 1

7. Ross Herbert, SAIIA

8. Understanding goals • Anticipating problems • Putting in place strategies before problems strike Good Planning

9. So this means • Anticipating problems • Designing effective strategies • Learning from pioneers’ best practices • Avoiding known obstacles • Innovating to set better benchmarks

10. Practical Tasks • Broad civil society and business consultation required – at all levels • Form governing body • Form research, consultation & writing plan • Find funding for research and hosting • Execute the plan: • Write national self-assessment report • Write national programme of action • Host country review mission

11. DAY 1: Tuesday 20 February • Seminar Overview, Background, Objectives • Understanding the positive benefits of APRM – Interactive discussion • APRM Basics – Structures and Stages • Understanding the Questionnaire and Research Methods • Making a Consultation Plan: Taking a Best Practices and Lessons Learned Approach • Planning for an APRM National Survey • Discussion • Cocktail reception – Delegate’s Lounge SESSION 1

12. DAY 2: Wednesday 21 February • Recap of Day 1 • Key Roles of the National Focal Points • Developing the Programme of Action • Communication, Public Information & Media Strategies • Country Review Missions – Lessons from Ghana, Rwanda, Kenya and South Africa • Planning and Budgeting For APRM at the National Level • Discussion/Final Advice from Pioneer Countries • Closing SESSION 1

13. Session 2: Understanding the Benefits of APRM • Francis Appiah, Ghana • Steven Gruzd, SAIIA

14. Understanding APRM’s Positive Benefits: Exercise • Why are we doing APRM? • How do we plan in order to realise positive benefits? • Why are governments apprehensive about peer review, and why can civil society be sceptical? SESSION 2

15. Understanding APRM’s Positive Benefits • APRM is solid, taken seriously, being watched – by Forum, Panel, Secretariat, review teams, citizens, media, the world – so countries cannot avoid the key governance issues • Be proactive: design systems and institutions to pre-empt problems and complaints • Embrace constructive comment in spirit of peer learning • Can only enhance image if process is genuinely open and free of controversy • How Ghana dealt with these issues – NGC design, independent secretariat, managing funds, validation, POA implementation and linking reform to APR SESSION 2

16. Nana Boateng, APRM Secretariat

17. APRM Basics: Structures & Stages • APRM institutions • 5 stages of the process, focus on • Preparing the self-assessment & POA • Hosting the country review mission • Experiences from early countries • Progress and lessons from early reviews SESSION 3

18. Understanding the Questionnaire • Ross Herbert , SAIIA • Baffour Agyeman-Duah, UNDP Tanzania and former head of CDD research body in Ghana

19. Understanding the Questionnaire & Research Methods (pg 5) • 88 Pages, 25 Objectives, 58 Questions, 183 Indicators, 4 themes • Political and Democratic Governance • Economic Governance and Management • Corporate Governance • Socio-Economic Development • Requires 2 kinds of input • Technical • Popular/broad based SESSION 4

20. Localising the Questionnaire • Not organised as survey questionnaire • Complex language – needs some translation • Adding local issues, i.e. crime, land, local sources of conflict, media freedoms • Organising for Researchability • Mixed objectives • Theme vs institutional focus • Overlaps between themes • Decentralisation • Gender • Oversight and Corruption • Economic strategy, investment climate & sustainable development

21. Organisational Issues • Elections and need for independence • National Governing Councils – Eminent vs representative • Perceptions of COPPER • To pay NGC members or not? • Research institutions • Transparent selection process • Availability of staff • Availability of expertise & subcontracting (pg 9)

22. 6 Tasks to Desk Research (pg 9) • Identify the relevant existing reports, • Prepare a bibliography and experts list • Prepare key issues list from existing reports, • Under each issue, provide supporting evidence, using footnotes to make it easy to find the relevant portions in future. • Extract from the existing reports a list of their recommendations and propose others that are suggested by the evidence. • Translate each issue paper into local languages.

23. Desk Research ARPM Workflow List of Issues & Actions Flesh out list with citizen survey & workshops Expert Survey & Workshops Validation Processes POA Workshops Final POA & Self Assessment

24. Research Methods

25. Understanding the Questionnaire and Research MethodsFocal Point APRM Training Workshop, Addis Ababa 20-21 February 2007 Presentation by Baffour Agyeman-Duah, Senior Governance Advisor, UNDP-Tanzania

26. Introductory Comments • Although a peer review of African Heads of States, the philosophy and guiding principles of the APRM assessment are “people-centered.” • It is designed to be mass-based, it’s all-inclusive and non-exclusionary and the assessment should be TRULY NATIONAL - full participation of all citizens! • The Questionnaire requires both technical research (hardware/scientific) and citizens’ input (software/popular) • Key question to answer: How do you ensure the voices of the people are truly reflected in the assessment?

27. Approaches to ensure popular participation in Assessment: The 4 approaches to Governance Research: • Desk research – literature review: all-inclusive, don’t be selective! • Expert surveys – ranging between 100-200 experts: Who are experts? • National citizen surveys – mass/popular surveys: Sample size? • Focus group discussions – on key issues of each thematic areas

28. Using research agencies/ selection criteria – the case of Ghana • Ghana boasts of a number of reputable research institutions: ISSER, IEA, CDD-Ghana, IDEG, PEF, Universities, etc • But selection requires: • Demonstrable capacity of the institution including • Quality of staff • Track record in research • Capacity for networking

29. Use of both in-house and out-house expertise • Research institutions should have the latitude to commission specialized experts to help on certain questions. In Ghana, CDD-Ghana commissioned • Educated and article chiefs (Nana Asante Bediatuo) to address chieftaincy • On Women and Gender: Nana Lithur Oye, Mrs. Quaye, etc. • On Conflicts: Dr. Anin, Security personnel who kept the peace in Yendi

30. Documentation, Validation and Quality Controls 1. The importance of documenting contentions, footnoting, • Document issues of particular contention, e.g., the Yendi conflict in Ghana, land tenure system, to give insights to the reviewers (APRM Panel) • Divergent views should equally be documents, i.e., where there is no consensus • Documentation include commissioned reports, footnoting in the text, etc.

31. 2. The importance of Validation steps to debate and discuss issues • In Ghana some drafts (commissioned papers) were given to specialists/experts for validation • Draft reports given in advance to stakeholders before validation meeting • Stakeholders should include both sides of the argument: including opposition groups acceptability

32. 3. Issues of Quality Control, Timeframe, and Cost • Senior staff at research institutions should be fully engaged • Supervision should be strong to ensure quality control • Technical Teams and APRM Secretariat should agree on timeframe and manpower requirements • Costing should be adequately discussed and agreed upon, but additional costs possible once work is underway!

33. Workshop Overview:Key Messages • Why do we need APRM? • What are the potential benefits? • APRM is complex, so understand how it works, anticipate challenges, manage process carefully • It takes a lot of work, time, staff, money, political will – but it can be done well by all countries! • Undertake the best APRM process yet • Learn from the pioneers • Innovate • Ensure process is rigorous, transparent, consultative and POA really makes a difference SESSION 1

34. Practical Tasks • Broad civil society and business consultation required – at all levels • Form governing body • Form research, consultation & writing plan • Find funding for research and hosting • Execute the plan: • Write national self-assessment report • Write national programme of action • Host country review mission

35. Key Elements of Consultation Plan • Independent Governing council • Adequate Budget (overall & line items) • Technical inputs plan (desk research, experts papers, focus groups, surveys) • Public input plan (surveys, invitations, workshops) • Government involvement • Plan to form & debate a robust POA • Media strategy • Validation SESSION 5

36. Planning an APRM Survey • Amadee Darga • Prof. Ayenew Ejigou, UNECA Survey Consultatnt • Prof. Wafula Masai, ACEG Kenya

37. Planning for an APRM National Survey • Achieves rigorous & objective broad consultation: gathers input from across the country, highlights location of problems for POA • BUT needs much planning, time, staff, money • Creating a survey instrument • Translation • Training administrators • Choosing sample set • What happens in practice – Kenya’s case SESSION 6

38. END OF DAY 1

39. Key Roles of Focal Points • Scant written guidelines emphasise person’s level of access & liaison role • Roles include • Building relationship with NGC • Linking government & civil society • Facilitating research agency access to government • Mustering political support for POA • Arranging and supporting country missions • Gathering info on compliance to standards & treaties • Ensuring effective budget management & fundraising • FP’s words and actions signal intent, determine perceptions about fairness, candour and rigour SESSION 7

40. Developing the POA • POA is critical, but difficult to do and often neglected afterthought • Focal point must bring government and civil society together, build consensus • Build the generation of solutions into the research and consultation process • Plan for dedicated POA research and workshops • Make the POA central, not peripheral – integrate existing plans, reform processes and future opportunities SESSION 8

41. Communication, Public Information & Media Strategies • APR involves interaction and information, so strategies matter • Tone and style of communication sends signals about openness, candour, sincerity • African populations know very little about APRM • Sensitisation should ideally precede consultation • What media and methods best inform and involve citizens? • Tv, radio, newspapers, website, meetings • Language and message considerations • What do different target audiences need? • Government -- Political parties • Civil Society -- Business • Researchers -- APR Sec, Forum, Panel • Development partners • How to get the media on board SESSION 9

42. Country Review Missions – Lessons from Ghana, Rwanda, Kenya, South Africa • Assess integrity and inclusiveness of national process • Rigorously probe key governance issues, whether in CSAR or not • Focal point handles logistics and agenda • Allow civil society freedom to interact • The same criteria of openness, candour and rigour apply to these missions SESSION 10

43. National Level APRM Planning and Budgeting • Build a budget to achieve goals, don’t modify goals to fit budget • How many staff members are needed? • What logistical support is required? • When will funds come from, and how will they be managed? • APR is not much cheaper or less complex in small countries • Building a budget – the Lesotho example SESSION 11

44. Discussion & Final Advice • Advice from focal point veterans … if I knew then what I know now • What makes a good APR? • open & transparent process • candid & rigorous report • POA that will make a real difference SESSION 12

45. END OF DAY 2