Evaluation of Policy Initiatives: Transparency and Accountability

1. Evaluation of Policy Initiatives: Transparency and Accountability Presented to: Anti-Corruption Policy: Can International Actors play a Constructive Role? Bellagio, June 13-17, 2011 Presented by: Francesca Recanatini Senior Economist PREM Public Sector Governance World Bank

2. Outline • Setting the stage – a few illustrations • Framing the problem • An alternative approach • Lessons learnt and remaining challenges (for local and international actors)

3. In a low-income, democratic country in Latin America, top political leaders paid and accepted bribes from an intricate network, spanning all aspects of state and society. Through the network, the leadership controlled judiciary, legislature, and media. Oligarchs in a country in northern Africa have divided the economy into lucrative monopolies, through government supported restrictions on competition. Civil society is weak. The President of a resource-rich country in Central Asia accepted millions of dollars in bribes, placing them abroad in a major financial center. There are few checks on executive power. Setting the stage…

4. An African country has signed a peace agreement after a decade of conflict. Formal institutions (especially local ones) are largely absent. Capacity is limited. Power is highly concentrated – a few leaders control allocation of land and awarding of public contracts for reconstruction. Nepotism and patronage are widespread. In a low-income, democratic country in Africa, businesses finance political parties in exchange for preferential treatment. Petty corruption is widespread. Efforts to combat corruption stop prior to elections, out of fear that party financing will be cut off. A chronically under-funded military in a country in East Asia resorted to self-financing, establishing commercial enterprises and foundations. Generals enrich themselves by sitting on boards. Setting the stage, cont.

5. Emerging “features” of the “corruption” challenge • In practice, an heterogeneous challenge • Different country’s reality (political, institutional and cultural) • Different forms of corruption • Different level of skills and resources • It involves diverse actors and stakeholders (local, national and international) • It requires a significant re-allocation of powers and rents

6. How to address this challenge? The emerging policy should… • Be country specific • Ensure and sustain coordination and collaboration among different actors • Have both short term and a medium term vision • Promote local capacity building • Monitor progress and adjust to new circumstances

7. Selecting an approach

8. A possible approach: Governance and A-C Diagnostic Surveys • A participatory process to identify governance challenges and build local capacity • Key features: • Medium/long term partnership between multiple actors for design and implementation • Mechanism to facilitate feedback from different stakeholders (focus groups) • Multiple sources of data (from households, firms and public officials) focused on experience • Rigorous technical implementation • Local institutionimplements

9. What are the likely outcomes? Unbundled evaluation of corruption – administrative, state capture, bidding, theft of public resources, purchase of licenses, nepotism Identification of weak and strong institutions Assessment of the impact and costs of corruption on different stakeholders To what extent has this approach helped shape public sector reforms? Only when paired with political will, donor coordination and (true) medium term vision Lessons learnt

10. Lessons learnt, cont. • What are the advantages and disadvantages of this approach? Pros: Greater local capacity, consensus and ownership that can ensure sustainability of reform process; “south-south” knowledge sharing (Costa Rica, Zambia, Mozambique, Haiti) Cons: Time consuming and costly; challenging to coordinate many different actors, especially international ones; unforeseen political changes • How do we “reconcile” aggregate indicators with national assessments? Two sides of the same coin with different objectives. Key => the two approaches complement each other

11. Lessons learnt, cont. • Who should be involved in measuring? It depends on the country reality. Our experience: the country as a whole. But donors, INGOs can play a very important role (Sierra Leone, Mozambique, Haiti). NSOs should also be involved to promote sustainability (Peru, Paraguay) • How can we help various stakeholders make appropriate use of these tools? Local focus, local capacity building, participation, and learning by doing (Paraguay, Peru, Sierra Leone)

12. Thank you!Q & A

13. Addendum

14. The role of governance assessments Will and Political Leadership for Institutional Reforms On governance Strategy and Action Plan • Governance assessment: • Identification of severe obstacles • Vulnerability of each institution • Identification of priorities Empirical Tools and Data Capacity building and coalition building

15. Country Diagnostic Results Extent of corruption, (Selected Countries ‘03-’05)

16. Corruption imposes barriers to households to access basic services,Sierra Leone 2003 Sierra Leone Roads Transport Authority

17. Mechanisms to participate to the policy process % of households reporting to use the following channel to participate in the policy process (Sierra Leone, 2003)

18. Bribes to win contracts with Government, (as reported by public officials, 1999-2005)

19. Public funds are mismanaged by agency(as reported by Public Officials, Sierra Leone, 2003) % of Public Officials that said irregularities/(misappropriations) are frequent

20. Agency-level Indicators • Using responses from public officials • Public officials are employees of each agency • Public official’s responses are re-scale (from 0 to 100) and then aggregated by agency using factor analysis technique • 0 always meaning the lowest level of quality of governance, corruption, access or service performance

21. Governance and corruption indicators by province

22. Sector Level Diagnostic Surveys: Key questions (e.g. transport) • Institutional structure of the sectors under study • How does transport work in Mauritania? What needs to happen at the implementation level? • Possible vulnerabilities • Internal to the sector: transporters, officials and regulations. External to the sector: linked to banking sector? International issues? • Mechanisms of poor governance • Is it difficult to get a trucking license? Are bribes required to cross borders with freight? Do civil servants have necessary capacity? Are rules clear? • Costs of poor governance • What price do transporters and customers pay to ‘facilitate’ antiquated or inadequate processes? What is the mark up on contracts due to fraud? How many roads are narrower than they should be because of corruption? • Who are the major players and what are their policy needs? • Potential entry points for reform? Who plays a role in the reform process? Who can be a potential deal breaker? How can we understand the political landscape to ensure policy outcomes are politically viable?

23. Methodological Approach Tools & Info Process Needs Iterative process: 8-12 months Desk Study: history, sector structure, current context Outcomes: 1. Governance Baseline 2. Agency Specific Indicators 3. Public Dissemination & Participatory Policy Process Transparent Process Sector Level Governance Assessment Local Partnership Gov’t + Civil Society In-Depth Interviews Donor Partnership w/Active Donors Cross-Sector Team (PREM, WBI, SDV,PDS, Procurement) Country Ownership Political Economy Assessment 3 Surveys (Experience Based) Partnership w/ Bank Country Team Audit & Project Data Broad Peer Review Team Capacity Needs

24. Innovative features • Sector-specific focus: • Apply methodology and solutions to country and sector realities • Mixed methods: • Focus groups & In-depth Interviews • Surveys (households, businesses & civil servants) • Desk study • Project cost data • Active participation of civil society and government to contribute to policy making process. • Close collaboration with donors’ colleagues. • Active links to on-going sector projects • WB transport and port projects • EU transport ministry aid project