Managing for Results in the Road Sector

1. Managing for Results in the Road Sector 16 May 2011 Singapore Adrien Véron, EARD, ADB

2. Session Organization Four Modules Managing Road Sector Performance • Country Presentation Pakistan Delivering to and with Road Users • Country Presentation Nepal Performance Contracting • Group Discussion Road Asset Management • Country Presentation Mongolia

3. Module 1: Managing Road Sector Performance • Key Learning Objectives • Understanding what results-based management can mean in the road sector • Being aware of the internal and external factors that affect performance • Learning about which performance indicators and when to use them • Getting to know about good practices in road sector performance management

4. Why Results-Based Management? What is Results-Based Management? “A government that works better and costs less” “A government that does the right things right” Birth Growth Upgrading Mature Focus Doing Managing Contracts Accounting for Performance Delivering Services Funds Government Government / IFIs Government / Private Government / Private / Users Private Sector Force-account Supervision, Contracting Design, Maintenance PPPs, Asset Management, Source: Adapted from OECD, 2003, Internal Performance Improvement of Road Administrations

5. Trends in Public Sector Management • Separation political / execution levels • Decentralization • Independence of agencies • Contracting-out & privatization • Transparency & user participation Government Role • From fixing problems to creating outcomes • From one-year to multi-year • High-level program /output / sector budget allocations Planning / Budgeting • Defining performance and targets • Allocating budgets to performance targets and reporting • Performance agreements within agencies’ business plans • Delegating authority to managers • Performance-based managers’ contracts Management Models Systems • Integrated financial systems • Information & management systems • From control of inputs to outputs • From pre-audit to post-audit • Performance audits & evaluation • From external control to internal audit Control

6. Factors Influencing Road Sector Performance Drilling Down Performance in a Road Maintenance Program No capacity to measure indicators Teething conflicts Excessive externalization Nascent planning Ad hoc approach to projects Overlapping responsibilities & multiple hierarchical lines • Clarity and Measurability of Objectives • Incomplete • Unmeasured • Un-integrated • Unused • Efficiency in Planning and Funding • Rigid budget structure • No budgeting for DD • & supervision • Non-technical planning • Extra-budgetary practices • Unskilled civil servants • Poor program perception Separate systems Ad hoc approach to information gathering • Cost-Efficiency in Execution Mechanisms • Cumbersome bidding processes • Slow payments / old contracts • Systematic 25% cost overrun • High monopoly bitumen price • Weak control of supervision • Corruption incentives Weak demand for results • M&E Systems • High rates of error • No management view • No dynamic / warning • approach Many veto players Political investments Public sector HR Unsolved issues Budget rigidity Paralyzing control Leadership changes Political staff Erratic demands for information Source: Based on case study from Brazil

7. Factors Influencing Road Sector Performance Drilling Down Performance in a Road Maintenance Program Creation of institute Long-term planning New staff hiring Program clean-up Performance-based contracts Unification of program structure • Clarity and Measurability of Objectives • Incomplete • Unmeasured • Un-integrated • Unused • Efficiency in Planning and Funding • Rigid budget structure • No budgeting for DD • & supervision • Non-technical planning • Extra-budgetary practices • Unskilled civil servants • Poor program perception Single-point entry system User feedback and transparency • Cost-Efficiency in Execution Mechanisms • Cumbersome bidding processes • Slow payments / old contracts • Systematic 25% cost increases • High monopoly bitumen price • Weak control of supervision • Corruption incentives Transport Council • M&E Systems • High rates of error • No management view • No dynamic / warning • approach Strong demand for results Priority program definition PPPs Performance audits Regular performance reviews Source: Based on case study from Brazil

8. A Performance Framework for the Road Sector Transport Sector Outputs Desired - Road system - Maintained assets - User services - Information Undesired - Road works safety - Land occupation - Environmental degradation Outcomes Users - Accessibility - Road quality - User satisfaction - Congestion - Accidents Community - Urban quality - Air pollution - Landscapes Road Agency Impacts Employment Market Shifts Equity Environment Use of Resources Activity Patterns Land Use Price Changes INPUTS CapitalMaterialsLabor Assets Systems HR Processes - Manage organization - Gather information - Plan activities - Allocate resources - Procure & deliver products & services - Evaluate efficiency GOVERNMENT GUIDANCE Objectives & goals Priorities Policies & regulations Resources Road User and Community Feedback Source: Adapted from OECD, 2003, Performance Indicators for the road sector

9. Performance Indicators: When? Indicators as a key management tool Objectives And Goals Road Program Development Impacts Reflected Evaluated Performance Indicators Observed & analyzed Monitored Road Program Delivery Road Program Performance Source: Adapted from OECD, 2003, Performance Indicators for the road sector

10. SMART indicators: Specific Measurable Attributable Realistic Targeted Government Road Agency Road Users Accessibility • HCM levels of Service • Average road user costs • Transport cost / GDP • Accessibility index • O&M costs by veh.km • Travel time and reliability • Quality of traffic information to users • Level of satisfaction on traffic time and info. • Hours of congestion delay • User charges / veh.km Safety • Fatalities per population • Accidents by veh.km • % of accidents involving • drunken driver • % of traffic speeding • Safety program assessments • % of roads not meeting design standards • Unprotected road user risk • Time from alert to treatment Environment • Air quality standards • Cumulative land taken % • Environmental program • Emissions per capita for CO2, NOx, VOC, PM • % of pop exposed to noise levels > 65db • % of pop exposed to air quality below standards Equity • Regional distribution of roads • Laws for access by reduced mobility people • Balance of user charges / expenses by region • Market research & customer feedback • Travel costs /user group • Accident risks /user group • Satisfaction with feedback capacity Source: Adapted from OECD, 2003, Performance Indicators for the road sector

11. SMART indicators: Specific Measurable Attributable Realistic Targeted Government Road Agency Road Users Program Performance • Value of assets • Ex-post value of CBA • Trends per program • Road expenditures / GDP • Pavement roughness • Bearing capacity • % of defected bridges • Congested roads – km • Truck overloading incidence • Surface condition • Rest areas / 100 km • % of main roads lighted • Frequency of emergency and winter road closures • User information system Program Preparation and Delivery • Clarity of objectives • Stability of leadership • Degree of completion of long-term programs • Sufficiency of maintenance funding • Forecast costs vs. actual • Cost of O&M per lane.km • Overhead percent • Staff / lane.km • Delay from preparation to implementation of projects • % of contracts with overruns and delays • User delay associated with maintenance and works • Road works safety and nuisances • Transparency & competition Source: Adapted from OECD, 2003, Performance Indicators for the road sector

12. Good Practice (1) New Zealand Multiyear fiscal planning Annual business plans and strategic plans Statement of Intent and Performance Reports approved by Parliament Baseline expenditure reviews to assess output prices Annual performance and value-for-money auditing by auditor general 5-year performance based contracts for chief executives See: http://www.nzta.govt.nz/planning/index.html

13. Good Practice (2) Japan Performance plans and achievement reports Data disaggregated at prefecture level Customer surveys Outcome-based budgets Analysis of effectiveness Internal motivation on outcomes through publicity and peer comparisons Source: Adapted from T. Hishio, Y.Tsukada, T.Oba and M.Ohno, 2005, Outcome-oriented Performance Management of Road Administration in Japan. MLIT See also: http://www.mlit.go.jp/road/road_e/index_e.html

14. Module 2: Delivering for and with Road Users • Key Learning Objectives • Understanding the different forms of user participation • Learning about specific participation mechanisms: Perception Surveys, Transformation Forums, Road Boards • Understanding the concepts of levels-of-service

15. Acknowledging Stakeholders Who? Motorists’ associations Public transport users associations Taxi Associations Truck operators associations Road users Local governments NGOs for special interests, inc: - Pedestrians - Environment, etc. Communities affected Industrial / mining associations Chamber of Commerce Tourism industry Agricultural groups Downstream businesses Professional Associations Industry Association Scientific and academic groups Road industry

16. Involving Stakeholders Increasing Level of Public Impact INFORM CONSULT INVOLVE COLLABORATE EMPOWER GOAL To inform users and assist them in understanding problems To obtain public feedback on analysis and alternatives To work with the public to ensure public issues are considered To partner with the public in all analysis and decision-making steps To place decision-making power in the hands of the public COMMITMENT We’ll keep you informed We’ll keep you informed and listen to your feedback We’ll ensure your concerns are reflected and tell you how We’ll always seek your advice and recommendations We’ll do what you decide TOOLS • Reports • Websites • Open houses • Comment • Call centers • Surveys • Public meetings • Workshops • Deliberative meetings • User advisory groups • Participatory decision-making • Road Boards • Delegated decisions Source: Adapted from International Association for Public Participation www.iap2.org

17. Good Practice (1) Perception Survey Governance in Kazakhstan / Armenia Four dimensions: Road Agency Governance Index • Transparency, disclosure and accountability • Transparent procurement systems • Financial management systems • Administrative procedures and anticorruption Small-scale / low cost Internal & external stakeholders Kazakhstan Armenia Source: C.Queiroz, A.Martinez, S.Ishibara, K.Homman, 2011, Road Asset Governance Filter: Case Study of Kazakhstan and Armenia, World Bank, DC.

18. Good Practice (2) Participatory Decision-Making Mongolia, National Road Sector Capacity Development Roadmap • Diagnostic • Capacity • assessment • Collaborative • Preparation • Shared • diagnostic • Propose • objectives • Evaluate • options • Concrete • propositions • Evaluation Consultation • Preparation of • roadmap • Consultations • on draft • Finalization • Formal • drafting • Presentation • to Cabinet • Implementation • Set-up task • forces • Conduct legal • changes • Trainings • Monitoring • & Evaluation W H O ? Consultant, ADB Stakeholder groups Facilitator Road Agency + Stakeholders Ministry Road Agency Stakeholders Consultants 6 months 4 months 3 months 2-4 months 5 Years? How long? Source: ADB, 2010, Technical Assistance for the Preparation of a National Road Sector Capacity Development Roadmap, Manila

19. Good Practice (3) Road Boards Second Generation Road Fund Criteria Road Fund only relies on road user fees to pay for roads O&M Road Fund to act as purchaser, not provider User fees to be directly deposited in the Road Fund Road Fund to be concentrating on road maintenance Road Fund to be managed by an independent Board Road Board to recommend level and adjustments to road fees Road Board to be composed in majority of user and business representatives Road Board to have a secretariat and be managed under sound commercial practices Source: ADB, 2003, Roads Funds and Road Maintenance, an Asian Perspective, Manila, and World Bank, 1995, African Road Funds, What Works and Why?, Washington, DC

20. Levels of Service (LoS) Managing the Roads for the Users “Levels of Service describe the quality of services provided by the asset for the benefit of the customers.” “Levels of service are the manner by which the highway authority engages with the customer.” “Levels of service are about reflecting the customer’s interests in terms that can be measured and evaluated.” Road users satisfaction Levels of Service Budgets Plans Works Works Plans Budgets Levels of Service Road users satisfaction Source: CSS (2004) Framework for Highway Asset Management, County Surveyors Society

21. Levels of Service (LoS) Use of Levels of Service Document and measure services provided Rationally evaluate service versus cost trade-offs Determine if adequate focus is given to what matters to the customer Plan operational activities to support strategic goal to meet LOS Type of Levels of Service Objective / structural Condition of asset Perceived by the users Safety Performance (congestion, availability) Service aspirations Environmental impacts User services Others. Source: CSS (2004) Framework for Highway Asset Management, County Surveyors Society

22. Levels of Service (LoS) Source: Hertfordshire’s Highway Asset Management Plan

23. Module 3: Contracting Maintenance for Performance • Key Learning Objectives • Being aware of the various ways to contract road maintenance • Understanding what is performance-based contracting and its benefits • Getting familiarity with performance specification issues • Getting familiarity with compliance monitoring issues

24. The Maintenance Contracting Menu ROUTINE PERIODIC EMERGENCY MANAGEMENT • Input based In-House Maintenance • Hors-d’oeuvre • Low private sector • involvement Salesman model • Unit prices • 1 year / seasonal Partial competition • Unit prices • 1 year / seasonal • Unit prices • Indicative BOQs • 3-5 years Framework model • Entree • Performance • contracts • Lump-sum / PBC • 5-10 years Bundling model • Lump-sum / PBC • 1-3 years Routine maintenance • Lump-sum / PBC • Unit / Global prices • 5-10 years Rehabilitation and Maintenance • Desert • Full devolution • Lump-sum / PBC • 5-10 years Integrated / area model • Tolls / penalties • 20-35 years Brownfield concessions • Tolls • Lump-sum / PBC • 20-35 years PPPs

25. Performance-Based Maintenance Contracts (PBCs) What is a Performance-Based Maintenance Contract? In PBCs, the contractor gets paid for delivering a service, not physical works Client specifies only performance targets, not methods and materials Service is considered delivered when expected performance indicators are reached: - Roads free of potholes - Grass cut, etc. To meet performance targets, contractor is free to decide: - What to do - When to do - How to do - Where to do - To do himself or subcontract (with limits) Lump sum payments are made periodically - some items based on unit prices - adjustment for change in factors (e.g. traffic, materials prices, inflation) If some performance indicators are not met, contractor does not get full payment for the related service (or gets penalties) Contractor is responsible for Quality Assurance and data gathering Procurement combines price and technical quality Contracts last several years Source: Adapted from http://www-esd.worldbank.org/pbc_resource_guide/TrainingMaterial.htm

26. Performance-Based Maintenance Contracts (PBCs) What are the Benefits of Performance-Based Contracts? Cost savings • Incentives for innovation and good management • Reduction in administrative expenses • Possibility to reduce client staffing levels • Greater flexibility to reward performance Risk sharing Cost Saving Potential Predictability of costs and financing • Reduction in variation orders • Support for ensuring stable • financing Life-cycle cost decisions Focus on users / Levels of Service Source: Finish Road Enterprise, 2002, Innovative Project Delivery Methods for Infrastructure, Helsinki Source: Adapted from World Bank, 2005, Performance-based Contracting for Preservation and Improvement of Road Assets, Washington, DC

27. Compliance monitoring Internal Q&A It is Contractor’s responsibility to: Ensure performance standards are constantly met Implement a Quality Control Plan for constant monitoring Act upon deficiencies in a pro-active manner Make available information to the Client: - summary sheets - daily logs of deficiencies and actions taken - inspection logs in accordance with QA plan Inspections Formal monthly inspections on selected number of roads - to verify summaries and inspection logs Informal inspections, at the client’s initiative, at any time and anywhere - Notification to contractor within ideally 24 hours of finding - Contractor to take remedial action within a certain time frame - In specified cases, result in penalties Road user complaints Source: Adapted from http://www-esd.worldbank.org/pbc_resource_guide/TrainingMaterial.htm

28. Setting-up Performance Standards: Group Exercise Contract Type National roads, 200km bituminous, 150km gravel Traffic at 3,000 vehicles per day on the first stretch and 500 vehicles per day on the second Contract for the rehabilitation and maintenance of the roads, 10 years Hilly terrain, with occasional snow-blocks and heavy rainfalls Road Agency Objectives Minimize life-cycle and administrative costs Satisfy users’ aspirations - accessibility - comfort - speed - safety - information Source: Adapted from http://www-esd.worldbank.org/pbc_resource_guide/TrainingMaterial.htm

29. Setting-up Performance Standards: Exercise Sheet Performance Standard Measurement Mechanism Response time and corrective action Routine Maintenance Periodic Maintenance Source: Adapted from http://www-esd.worldbank.org/pbc_resource_guide/TrainingMaterial.htm

30. Module 4: Asset Management • Key Learning Objectives • Understanding Road Asset Management Planning and Life-Cycle Cost concepts • Understand the different components of a Road Asset Management System • Be aware of the various outputs produced by a standard HDM-IV based system • Be aware of data gathering tools, strategies and sustainability issues

31. Asset Management How much are asset worth? UK Highways = IBM Japan Road Public Corporation = General Motors Central and Eastern Europe = $550 billion Typical developing country: up to 50% of GDP What is Road Asset management? “The combination of management, financial, economic, engineering and other practices applied to road assets with the objective of providing the required level of service in the most cost- effective manner” Basic Asset Management “…relies primarily on the use of an asset register, maintenance management systems, job/res- source management, inventory control, condition assessment and defined levels of service in order to establish alternative options and long-term cash flow predictions” Advanced Asset Management “…employs predictive modeling, risk management and optimized renewal decision making techniques to esstablish asset licycle options and related long-term cash-flow predictions” Source: Adapted from http://www-esd.worldbank.org/pbc_resource_guide/TrainingMaterial.htm and New Zealand Infrastructure Asset Management Manual

32. Asset Management Planning • Customer expectations • Levels of Service • Cost of transport Organizational Vision Strategic Planning Process • Government guidance • Financial • Environmental • Non-Asset Solutions • Capacity Building • Demand management Asset Management Process Existing Assets New Assets Surplus Assets Maintain Renew Upgrade Assets Create Assets Dispose Assets Improved Performance Source: Adapted from World Bank, University of Birmingham, Senior Road Executives Program, http://www.worldbank.org/transport/training/birmingham-un/start.html

33. Asset Management Benefits What are the Benefits of Performance-Based Contracts? Better understanding of service level options and requirements Minimal (optimized) long-term costs for given LoS at the network level Better understanding and forecasting of asset-related options and costs Managed risk of failure Informed and objective (“scientific”) decision-making based on costs and benefits Clear justification for budget requests Improved organization image Asset valuation capacity Source: Adapted from World Bank, University of Birmingham, Senior Road Executives Program, http://www.worldbank.org/transport/training/birmingham-un/start.html

34. Life-Cycle Costs Source: World Bank, University of Birmingham, Senior Road Executives Program, http://www.worldbank.org/transport/training/birmingham-un/start.html

35. Maintenance Strategies Deterioration EXCELLENT Preventive treatments Level of Service POOR TIME Asset Condition Source: Adapted from World Bank, University of Birmingham, Senior Road Executives Program, http://www.worldbank.org/transport/training/birmingham-un/start.html

36. Asset Management Planning Management Cascade • Strategic • Planning • Mid-term budget • needs to meet LoS • Forecast pavement condition • User cost forecasts • Compared NPV of alternative strategies, standards Data Collection • Programming • Annual budgets • List of road sections with recommended treatment / cost • Rolling program • Preparation • Project design or formulation • Procurement • Works • Performance standards • Invoices • Progress reports • Cost analyses Road Asset Management System Source: Adapted from World Bank, University of Birmingham, Senior Road Executives Program, http://www.worldbank.org/transport/training/birmingham-un/start.html

37. Road Asset Management Systems Components Data Collection Database Decision support tools Management Information • Inventory • GIS coordinates • Riding quality • Surface distress • Pavement strength • Traffic flow/load • Brides • Furniture • Forecasting models • Optimization models • -> HDM-IV • Standard reports • Custom reports Source: Adapted from World Bank, University of Birmingham, Senior Road Executives Program, http://www.worldbank.org/transport/training/birmingham-un/start.html

38. Road Management Systems Sample Outputs Long-term Needs Intervention Strategies Optimal Budget Allocations Rolling Works Plan Source: Adapted from World Bank, University of Birmingham, Senior Road Executives Program, http://www.worldbank.org/transport/training/birmingham-un/start.html

39. Data Collection What to collect? Factoring in data costs, skill needs and use Collect only what you need Collect at the lowest level of details you need Collect when you need Information Quality Level (IQL) IQL-5: High-level aggregate data - performance indicators IQL-5 IQL-4: Data used for planning and management reports IQL-4 IQL-3: Network-level survey data, simple data gathering methods IQL-3 IQL-2: Data used for detailed designs and project-level decisions IQL-2 IQL-1: Research level data, many attributes IQL-1 Source: Adapted from World Bank, 2006, Data Collection Technologies for Road Management, Washington

40. Data Collection: Cost/Performance trade-offs Source: World Bank, 2006, Data Collection Technologies for Road Management, Washington

41. What have we learnt? Four Modules Managing Road Sector Performance • Country Presentation Pakistan Delivering to and with Road Users • Country Presentation Nepal Performance Contracting • Group Discussion Road Asset Management • Country Presentation Mongolia

42. Thank you averon@adb.org