Measuring Progress and Efficiency in Public Administration (Government at a Glance)

1. Measuring Progress and Efficiency in Public Administration (Government at a Glance) Oscar Huerta-Melchor OECD - Public Governance and Territorial Development Directorate (GOV) Tukums, Latvia, 28 August 2008

2. Presentation outline 1. Describing Government at a Glance (GaaG) 2. Measuring Efficiency in the Public Sector 3. Building an HRM Indicator – delegation of authority

3. 1. Describing Government at a Glance (GaaG) The OECD GaaG project aims to provide comparative data in order to: • enable countries to better understand their own practices; b) benchmark their own achievements through international comparisons; c) learn from the experiences of other countries facing similar challenges.

4. …. and in the longer term • Contribute to OECD-wide learning about public sector efficiency and institutional effectiveness. • Explore possible causes of performance differences among governments. • Begin to identify the impact of public sector reforms.

5. GaaG will contain data on: Antecedents Revenue Inputs Process Outputs Outcomes What are the goods and services which the public sector produces? What is the context in which government operates? How much money does the public sector collect? How much and what kind of resources does the public sector use? What does the public sector do and how does it do it? What is the resulting impact on the country? Framework for understanding and measuring government’s activities

6. a) Antecedents The political institutions and administrative structures provide the context that dictates public sector efficiency and effectiveness. b) Revenue data It reflects the redistributive power of government and the degree of ‘excludability’. c) Input data Expenditure structure - Expenditures: Labour as input for the production of public goods and services. - Employment data: The costs of public employment in order to gain proper understanding of production costs. - Compensation costs: - Production costs: Composition of employees, direct production cost, indirect production cost.

7. d) Data on processes / institutional arrangements An annual budget law conditions the implementation of political decisions. - Budget practices: Governments are under pressure to improve efficiency and effectiveness while controlling public expenditure. - Performance measurement: Efficiency in the management of human resources largely conditions the effectiveness of governmental policies. - HRM practices: It underpins government’s ability to ensure that regulations are efficient, effective and of good quality. - Regulatory quality management:

8. Integrity is critical for achieving good governance. However, existing indicators are mainly based on perceptions and do little to track changes over time. - Integrity framework: Public sector transformation and e-government are increasingly seen as closely linked policy areas. - e-Government: e) Output data Outputs and outcomes of public administration are concerned with the machinery of government. Defining outputs for measurement is a complex matter as what is an output for one sector is an input for other one. Therefore it is necessary to ask the question: outputs for whom? f) Outcome data Executive governance indicators: trust in government, citizens satisfaction, equity and fiscal/economic stability.

9. The outcome at one level is an input at a higher level

10. Outcome as the starting point

11. 2. Measuring Efficiency in the Public Sector a) Why measure government? • The size and economic significance of the public sector make it a major contributor to growth and social welfare. • The government is a major economic actor in modern society. It provides goods and services in kind and it redistributes income through taxation and transfers. • The government also provides non-economic and regulatory services (less tangible outcomes). • The way of measuring the output of government strongly affects the growth performance of the national economy.

12. b) What are productivity and efficiency? • Productivity is the ratio of a volume measure of output to a volume measure of input (OECD). • Efficiency has two dimensions: • Technical (or operational) efficiency. • Allocative efficiency. Technical efficiency Allocative efficiency Economic (or cost) efficiency • Two key objectives of efficiency measurement in the public sector: • To trace technical inefficiencies • To identify inefficiencies in the mix of production factors.

13. c) How to measure them? • Measuring productivity and efficiency requires quantitative information on inputs (costs) and outputs (volume) of public service provision. BUT • Public services are complex activities and therefore defining the relevant input and output variables for efficiency or productivity analysis is not straightforward. Output measurement – some challenges • “Activities where non-market producers dominate pose specific problems of productivity measurement, due to the difficulty or impossibility of observing and/or defining market prices or output” (OECD). • The input method is based on the assumption that output equals inputs for non-market services, then productivity is considered constant over time and across countries.

14. It is necessary to carry out quality adjustments. • Micro level • Intermediate level • Macro level • Outputs can be measured at various levels: BUT • Only final outputs should be counted since intermediate outputs may lead to double counting . • Aggregation requires weight . • The output mix of many public sector organisations includes intangibles. • The challenge of measuring outputs for collective services since it is difficult to define the output. • The creation of incentives to ‘game’ or cheat.

15. Input measurement • Efficiency analysis requires cost accounting. • For costs to be imputed to outputs it is necessary to separate costs and outputs. • Input information is not always available in monetary terms. Difficulties in measuring efficiency • Weak data. • Poor-quality measures of output with perception-based quality indicators • More importantly, the definition of the output and the methodology to measure it.

16. The impact of institutional arrangements on efficiency • Understanding institutional arrangements is essential to understand efficiency of public sector production processes. • Evidence on the impact of institutional drivers on efficiency in the public sector is rather limited. However: • Efficiency gains may be obtained by increasing the scale of operations. • Efficiency may increase via functional or political decentralisation to sub-national governments. • Human Resource Management practices largely influence efficiency. • Findings are inconclusive on the impact of ownership, competition and agencification over efficiency.

17. d) Exemplifying institutional drivers of efficiency – workforce issues Workforce size Are larger organisations more efficient than smaller ones? Workforce composition What is the impact of workforce diversity and representation on efficiency? Extent and nature of unionisation composition What is the impact of unions on public sector efficiency? Attractiveness of the public sector How to recruit and retain highly skilled staff?

18. 3. Building an HRM Indicator – delegation of authority a) Why delegate? • Good management is essential for improving performance and efficiency and improving services to citizens. • Hence, OECD governments have enable and empowered public managers to adapt their HR systems to the business needs of their organisations. • Delegation of authority to managers makes public organisations attractive employers. • The speed and the extent of the reforms vary across countries due to differences in political, cultural and historical context.

19. b) What can be delegated? Delegation of manpower planning. Delegation of recruitment functions. Delegation of staff training and development. Careers systems and planning. Organisations have to be able to motivate their staff. Working arrangements. Retirement benefits. A good indicator of freedom to manage is the degree to which departments control the personnel budget.

20. c) OECD criteria for building composite indicators a) A clear theoretical framework. b) Indicators must be selected on the basis of their quality and relevance. c) Explain and justify the selection of weights and aggregation methods. d) Select a method for handling missing values. e) Normalise indicators to render them comparable. f) Explicit assessment on the robustness of the composite indicator. g) Presentation should clarify not mislead. h) The composite indicator should be easily replicated.

21. d) Delegation composite – the process • Scoring and selection of each variables done within an expert group • Variables aggregated and weighted according to an underlying theoretical framework (Alesina & all.) • Implementation of statistical checkings: • Questions not statistically relevant were removed for building the composite (i.e. Factor analysis) • Good correlation among the set of variables (i.e. Cronbach alpha) • Index robustness (i.e. Sensitivity analysis) • The Delegation Index sticks to the steps identified in the OECD Handbook

22. e) The variables used for the HRM composite on delegation • Existence of a central HRM body (question 20) • Delegation of establishment (question 21) • Delegation of compensation levels (question 24) • Delegation of position classification, recruitment and dismissals (question 27) • Delegation of conditions of employment (question 30) • The arrangements for delegation (question 35) • Impact of delegation for pay/terms and conditions of employment across government organisations.

23. f) Example of scoring • Q. 20 Is there a central agency/department in charge of human resources at central/national/federal government? • Yes: 0.000 • No. 1.000 • Not responsible, but a central agency/department aims to coordinate the HR policies across departments: 0.5

24. Q. 21 Delegation of establishment

25. Delegation in the public service in OECD countries

26. g) Pros and cons of composite indicators Pros Cons • Can summarise complex, multi-dimensional realities to support decision-making. • May send misleading policy messages if poorly constructed or misinterpreted. • Are easier to interpret than many separate indicators. • May invite simplistic policy conclusions. • Can assess progress of countries over time. • May be misused. • The selection of indicators and weights could be subject of political dispute. • Condense information and thus make possible to include more information. • May disguise serious failings in some dimensions. • Place issues of country performance and progress at the centre of the policy arena. • May lead to inappropriate policies if dimensions of performance that are difficult to measure are ignored. • Help to construct narratives. • Enable users to compare complex dimensions effectively.

27. Further reading: • OECD (2007), “Towards Better Measurement of Government”, OECD Working Papers on Public Governance, 2007/1, OECD Publishing. • doi: 10.1787/301575636734 • Lonti, Z. and M. Woods (2008), “Towards Government at a Glance: Identification of Core Data and Issues related to Public Sector Efficiency”, OECD Working Papers on Public Governance, No.7 OECD Publishing. • doi: 10.1787/245570167540 • OECD (2008), “Handbook on constructing Composite Indicators: methodology and user guide”, OECD JRC European Commission. OECD Statistics Working Paper. • European Commission (2005), “Input to handbook of good practices for composite indicator’s development”, Knowledge Economic Indicators; Workpackage 5 .

28. OECD GOV Technical Papers: • Technical Paper 1: ‘How and why should government activity be measured in ‘Government at a Glance’. • Technical Paper 2: ‘Issues in output measurement for ‘Government at a Glance’. • Technical Paper 3: ‘Issues in Outcome Measurement for ‘Government at a Glance’. • Technical Paper 4: Institutional Drivers of Efficiency in the Public Sector’. Information available on line: www.oecd.org

29. Oscar Huerta-Melchor Policy Analyst OECD GOV Budgeting and Public Expenditures Division oscar.huertamelchor@oecd.org For further information on the OECD Government at a Glance project contact: Zsuzsanna Lonti Project Manager zsuzsanna.lonti@oecd.org Laurent Nahmias Statistician laurent.nahmias@oecd.org