World Bank “Detailed Methodology for Procurement Country Systems Piloting Program” (2007)

1. World Bank “Detailed Methodology for Procurement Country Systems Piloting Program” (2007) Evaluation and Comments 10 December 2007 Development Finance International, Inc.

2. Public procurement is the governmentactivity most vulnerable to corruption. • OECD (Integrity in Public Procurement, 2007) The multiplicity of rules may have a negative impact on transparency and lead to legal uncertainties and high transaction costs • OECD (Bribery in Public Procurement, 2007) Procurement agencies may also purposely use and abuse the regulatory diversity. For instance, they may privilege firms by opting for tendering procedures which require no controls. They may also formulate requirements which favour specific firms and constrain market access to specific suppliers. • OECD (Bribery in Public Procurement, 2007)

3. Industry – WB Shared Objective: Strengthen the national procurement systems of Bank borrower countries. Concerns: The proposed pilot methodology and process does not meet the standards of the World Bank’s current procurement system and 50+ years of improving practices in (i) measuring “equivalence” in selection of pilots and (ii) building local capacity. The many loopholes and omissions in the current proposal run counter to, and conflict with, the Bank’s strong position against corruption and for increased transparency.

4. Recommendation: Proposal and tools should be redrawn to preserve and maintain international best practice in procurement The WB should convene a technical working group of world-class experts to develop effective tools to assess a country’s adoption and implementation of international best practice in procurement. The working group should include experienced US and European firms who have been actively involved in analysis of the World Bank’s country system methodologies and the OECD/DAC’s assessment tools and methodologies.

5. Basis for Detailed Comments: World Bank procurement guidelines and Standard Bidding Documents side-by-side with: - Detailed Methodology for Procurement Country Systems Piloting Program, Draft for Consultations and related PowerPoint presentations - OECD/DAC Version 4 Methodology for Assessment of National Procurement Systems (laws and regs only) - CPAR data available (WB website) - Loan agreements where available (limited via WB website data; NCB side letters not available to public) - 2005 Country Systems proposal indicators * Capacity Assessment, Risk Assessment tools/details are not available for review

6. The 2007 Proposal for Country Systems Pilot Program Fails to articulate, maintain, and encourage international best practices Reduces transparency and accountability, increasing potential for corruption Undermines reliability and accessibility of procurement information Decreases protections for biddersin bidding and tendering processes

7. Start country-level Qualitative Assessment Process Flowchart 1: Country Level Qualitative Assessment Process (partial only – see Annex 1) Mandatory score On Critical sub-indicators? CPAR action plan implemented? OECD/DAC Completed & Validated Review NCB side letter YES NO YES Complete OECD/DAC Acceptable score on other sub-indicators YES Country is cleared for equivalency

8. Fails to articulate and encourage international best practices Under Country Systems Proposal Currently “…the Bank has developed world class standards, bidding documents and selection procedures that maximize the bidding opportunities and provide for a fair and open playing field for all eligible participants” - WB Detailed Methodology, p. 5 World Bank ICB Guidelines International Procurement Best Practices ICB Guidelines largely omitted “suggested minimum content of bidding documents” (OECD) – undefined Standard Bidding Documents (SBDs) Standard Bidding Documents waived and replaced by “suggested minimum content” UNCITRAL Model Law UNCITRAL Model Law not reflected WTO Agreement on Government Procurement New York Convention on Foreign Arbitration WTO work on transparency and multilateral guidelines is not considered

9. Reduces transparency and accountability, increasing probability of corruption • Under the proposal, a pilot country would not be required to: • Apply Bank’s Standard Bidding Documents to international tenders (2E) • Adhere to Bank norms on fraud and corruption and demonstrate a strong anti-corruption enforcement record (12B) • Ensure competitively selection of country procurement personnel based upon procurement knowledge and skills (6A) • Ensure that compliance auditors are sufficiently informed about procurement to conduct quality audits that contribute to compliance with regular reporting of issues to management (6A) • Conduct effective procurement data collection, analysis, reporting (5B) • Make Codes of Conduct / Ethics obligatory or enforceable (12G) • Disclose procurement system deficiencies until after loan execution (and then only in loan agreements) (n/a)

10. Undermines reliability, transparency, and accessibility of procurement information • Under the proposal, a pilot country would not be required to: • Make all procurement information publicly available free of charge and both comprehensive and easy to understand (5A) • Specify time frames for disseminating procurement information (invitations, procurement notices or awards) (5A) • Use or regulate model tender documents – where documents exist, only a minimum set of clauses is required (2B) • Regularly update and make transparent implementing regulations for the country’s procurement systems (2A) • Possess a procurement manual for contracting entities (2E) • Maintain procurement documents in accordance with international standards (6)

11. Decreases protections for biddersin bidding and tendering process • Under the proposal, a pilot country would not be required to: • Expressly state all evaluation criteria and stipulate lowest evaluated cost as the sole award criteria (1F) • Provide a clear method for evaluating non-price criteria in goods and works tenders (1F) • Specify the amount of time for bid preparation (1E) • Supply prospective bidders with all information pertaining to a bid • Ensure that all pre-qualified applicants are allowed to bid (2C) • Supply all bidders with the record of bid opening and explicitly deny bids received after the close date or not read publicly (1G) • Guarantee formal recourse for complaints to an independent body such as the World Bank (1H) • Specify in advance in the bid documents how currency and payment will be handled(1E)

12. OECD/DAC Tool vis-à-vis 2005 WB Proposal Requires 2 + Capacity Development Plan* Requires 3 Requires 2 * Set forth in Loan Agreement only; no methodology for monitoring; consequences for non-compliance unclear Public Procurement Legislative and Regulatory Framework 1a 1c 1d 1e 1f 1g 1h 1b Existence of Implementing Regulations / Documentation 2f 2c 2d 2a 2b 2e Public Procurement System Integrated with Governance System 3b 3c 3d 3a Country has a Functioning Regulatory Body 4b 4a 4c 4d Existence of Institutional Development Capacity 5a 5b 5c 5d Country Procurement Operations and Practices are Efficient 6c 6a 6b 6d Functionality of the Public Procurement Market 7c 7a 7b 7d/9e Existence of Contract Admin. and Dispute Resolution 8a 8b 8c Country has Effective Control and Audit Systems 9a 9b 9c 9d Efficiency of Appeals Mechanism 10a 10b 10c 10d 10e Degree of Access to Information 11a Country has in place Ethics and Anticorruption Measures 12a 12b 12d 12f 12c 12e 12g

13. Check OECD/DAC Tool Against WB Rated Indicators Requires 2 + Capacity Development Plan 2005 proposal:Withdrawn amidst concerns that procurement standards were “set too low” by making only 35% sub-indicators “mandatory” 2007 proposal: While 40% of sub-indicators now require “full compliance” for country systems pilot program eligibility, some have been downgraded, others watered down. Many are less rigorous than WB’s ICB. Requires 3 Requires 2 1a 1c 1d 1e 1f 1g 1h 1b 2f 2c 2d 2a 2b 2e 3b 3c 3d 3a 4b 4a 4c 4d 5a 5b 5c 5d 40% 6c 6a 6b 6d 7c 7a 7b 7d/ 9e 8a 8b 8c 9a 9b 9c 9d 60% OR 10a 10b 10c 10d 10e 11a 12a 12b 12d 12f 12c 12e 12g

14. Equivalence Methodology Comments

15. “Detailed” Methodology is vague on key procedures OECD/DAC baseline indicators are substantially more ambiguous than current ICB Guidelines; even “critical” indicators do not require the same degree of rigor as the current ICB Guidelines OECD/DAC v.4 lacks meaningful operational assessment tools or methodology Unclear how OECD/DAC v. 4 Benchmarking Tool will be applied or how it will be validated (NB: Will it remain a country self-assessment and self-validation as it is today?) Unclear why only a select few deficiencies require improvement through a Capacity Development Plan. Further the deficiencies will be included in the loan agreement only post-effectiveness. Today, loan agreements are withheld by the Bank pending effectiveness and are not easily available to the public, if at all, on Bank’s website. Unclear why certain sub-indicators are “critical” meaning required to qualify for pilot, others are “critical” but not required to qualify for pilot, and others “non-critical”

16. “Detailed” Methodology is vague on key procedures • Unclear how the Bank’s CPAR, NCB Side Letter, and OECD/DAC assessments are factored into a measurement of “equivalence” • Lacks comprehensive details of the “step-by-step analysis” to determine “equivalence” (who is involved, how it proceeds, how it is assessed and validated, and to whom testing results will be made available) • Unspecified time frame for implementation of agreed upon Capacity Development Plan or guidance regarding assessment and validation • No evident procedures for assessing progress required by the Loan Agreement, or specifics should a country fail to take effective actions. • No detail provided for new project risk assessment or capacity assessment tools and methodology

18. Summary: ICB Guidelines Acceptably Covered by Required Sub-Indicators GL 1.10: Joint Ventures GL 2.7-2.8: Notification and Advertising* GL 1.12: Misprocurement GL 2.43: Dispute Settlement* GL 1.15: Fraud Provisions / Bid Documents of Large Contracts** GL 2.19-2.20: Standards and Brand Names

19. ICB Guidelines NOT Acceptably Covered GL 2.59: Award of Contract GL 2.38: Conditions of Contract GL 1.6-1.8: Eligibility GL 2.42: Force Majeure GL 2.47: Confidentiality GL 2.49-2.54: Evaluation and Comparison of Bids GL 2.58: Postqualification GL 1.3-1.4: Appropriate International Procurement GL 2.6: Two-Stage Bidding GL 2.39-2.40: Performance Security GL 2.37: Alternative Bids GL 1.16: Procurement Plan GL 2.45: Bid Opening Procedures GL 2.15: Language Summary: ICB Guidelines Acceptably Covered by Required Sub-Indicators GL 2.13-2.14: Bid Validity and Security GL 2.55-2.56: Domestic Preferences GL 1.10: Joint Ventures GL 2.46: Clarifications or Alterations of Bids GL 2.7-2.8: Notification and Advertising* GL 2.57: Extension and Validity of Bids GL 1.12: Misprocurement GL 2.2-2.5: Type and Size of Contracts GL 2.43: Dispute Settlement* GL 1.15: Fraud Provisions / Bid Documents of Large Contracts** GL 1.9: Advance Contracting GL 2.19-2.20: Standards and Brand Names GL 2.34-2.36: Terms of Payment GL 2.16-2.18: Clarity of Bidding Documents GL 2.21-2.23: Pricing GL 2.65: Debriefing GL 2.28-2.33: Currency Provisions GL 2.26-2.27: Transportation and Insurance GL 2.44: Time for Preparation of Bids GL 1.14: Fraud and Corruption GL 2.41: Liquidated Damages and Bonus Clauses GL Appendix 3: Complaints GL 2.24-2.25: Price Adjustment GL 2.11-2.12: General Principles of Bidding Documents GL 2.61-2.64: Rejection of All Bids GL 2.9-2.10: Prequalification of Bidders

21. ICB guidelinesacceptably covered 12% ICB guidelinesNOT acceptably covered88% • Conclusions • The 2007 methodology for assessing laws / regulations does not improve upon the withdrawn 2005 proposal. • The proposal offers no methodology to assess day-to-day operational procurement practices. • The proposal does not measure equivalence with current Bank procurement rules and practice. • Essential elements of the proposal are omitted (compliance and risk assessment tools)

22. The 2007 Proposal for Country Systems Pilot Program Fails to articulate, maintain, and encourage international best practice Reduces transparency and accountability, increasing potential for corruption Undermines reliability and accessibility of procurement information Decreases protections for biddersin bidding and tendering processes A step backwards in effective and accountable international procurement Counter to good governance and sustainable economic growth =