The worldwide governance indicators

1. The worldwidegovernanceindicators Methodological and Analytical Issues Gaia Dallera 6 June, 2012

2. introduction • Introduction • DefiningGovernance • Governance Data Sourcesfor the WGI • Constructing the Aggregate WGI Measures • Using and Interpreting the WGI Data • AnalyticalIssues • Conclusions

3. Introduction • The WGI are a research project to develop cross-country indicators of governance. • Wewill focus on the methodology and key analyticalissuesrelevantto the overall WGI project.

4. 2. DEFINIng GOVERNANCE • “the traditions and institutionsbywhich authority in a countryisexercised…. ..Thisincludesthreeareas: • The processbywhichgovernments are selected, monitored and replaced. • The capacityof the governmenttoeffectively formulate and implement sound policies. • The respectofcitizens and the state for the institutionsthatgoverneconomic and social interactionamongthem”.

5. Operationalization: indicators

6. 3. Governance data sources These indicators are based on several hundred variables optained from 31 different data sources capturing governance perception Availability of the underlying data from (with few exceptions) the individual data sources (transparency and replicability) The WGI data sources reflect the perceptions of a very diverse group of respondent: There are four categories: • “Surveys” ofdomesticfirms and individualswith first handknowledgeof the governance situation • “Public sector data providers” • Commercial business information providers • Non-governamentalorganizations

7. 3. GOVERNANCE DATA SOURCES (CONT’D) • An important qualification is that the sources of data provide different coverage across countries (some cover a majority of countries and some cover small groups of countries). • Eachindividualvariable are notcomparableacrosscountries

9. 4. Constructing the aggregate wgimeasures • Combines the six aggregate governance indicators using a statistical tool known as the “Unobserved Component Model” (UCM) • The underlying premise of this statistical approach is straightforward – each of the individual data sources provides an imperfect signal of some deeper underlying notion of governance that is difficult to observe directly • This means that we face a signal extraction problem: • How can we isolate an informative signal about the unobserved governance component common to each individual data source? • How can we combine the many data sources to get the best possible signal of governance in a country? • Therefore we construct a linear regression model in order to: • Standardize the data from these diverse sources into comparable units; • Construct an aggregate indicator of governance as a weighted average; • Construct a margin of error that reflects the unavoidable imprecision in measuring governance.

10. 4. CONSTRUCTING THE AGGREGATE WGI measures (CONT’D) • Regression model ykj=ak+bk(gj+ejk) • Different thing to note about this regression model is that the error term is considered as an independent variable. • “a” and “b” are parameters that reflect the fact that different sources use different units to measure governance. • The errorterm and preciselyitsvariancecapturestwosourcesofuncertainty.

11. 4. CONSTRUCTING THE AGGREGATE WGI MEASURES (CONT’D) • We consider the estimate of governance as a weighted average of the re-scaled scores for each country. E[gj I yj1,…., yjk ]=∑ wk • The re-scaling puts the observed data from each source into the common units we have chosen for unobserved governance. • The larger the weights, the smaller the variance of the error term. • A crucial observation is that there is an unavoidable uncertainty around this estimate of governance. K yjk - ak k=1 bk

12. 5. USING AND INTERPRETING THE wgi data • We report the aggregate WGI measures in twoways: • Standard normal units of the governance indicator ranging from -2.5 to +2.5. • Percentile rank terms ranging from 0 (lowest) to 100 (highest).

15. 5. USING AND INTERPRETING THE wgi data (CONT’D) • Size of the confidence intervals varies across countries. • Resulting confidence intervals are substantial relative to the units in which governance is measured.

16. 5. USING AND INTERPRETING THE wgi data (CONT’D) • Many of the small differences in estimates of governance across countries are not likely to be statistically significant at reasonable confidence intervals, since the confidence intervals are likely to overlap. • Example of overalapping confidence intervals:

17. 5. USING AND INTERPRETING THE wgi data (CONT’D) • This is not to say that aggregate indicators cannot be used to make cross country comparisons. • There are in fact many pair-wise country comparisons that are statisticallysignificant. • 63% of the pairwaisecomparisons the confidenceintervals do notoverlap

18. 5. USING AND INTERPRETING THE wgi data (CONT’D) • For example: • 2009 Control of Corruption indicator covers 211 countries; • Total of 21,155 pair-wise comparisons.

19. 6. Analytical issues aggregation methodology (cont’d) • UCM has three main advantages: • It maintains some of the cardinal information in the underlying data. • It provides a natural framework for weighting the re-scaled indicators by their relative precision. • It naturally emphasizes the uncertainty associated with aggregate indicators. In fact it formalizes the signal extraction problem and by doing that it provides a rationale for a more inclusive approach to combining data from different types of sources.

20. 6. Analytical issues aggregation methodology (cont’d) • Unbalanced vs balanced samples: • Refers to the fact that WGI use all available data sources for all countries as opposed to using only those data sources that cover all countries and in all time periods (permitting balanced comparison).

21. 6. Analyticalissues:useofperception data • This choice is based on the view that perception data has a particular value in the measurement of governance because: • Agents (citizens and enterprise) base their action on their perceptions; • In many areas of governance there are few alternatives to relying on perception data e.g. corruption ; • The “jure” notion of laws often differs from the de-facto reality.

22. 6. Analyticalissues:useofperception data (cont’d) • Potential problems: • Interpretation of subjective data • Perception data is imprecise; • But all measures of governance are imprecise proxies for the broader concept; • Therefore it underscores the importance of using empirical methods and taking seriously the extent of imprecision. • Systematic biases in perception data on governance introduced by: • Different types of respondents may differ systematically in their perceptions of the same underlying reality; • The ideological orientation of the organization providing the subjective assessments of governance. • Possibility that different providers of governance perceptions data rely on each other’s assessments (correlated perception errors). • BUT THERE IS LITTLE EVIDENCE OF SUCH BIASES!!!!

23. 7. conclusionS • WGI reports six dimensions of governance covering 200 countries since 1996. • Updated annually. • Based on hundreds of variables from many different data sources.

24. 7. conclusionS (CONT’D) • Due to the inherently unobservable nature of the true level of governance in a country, any observed measure is only a proxy. • Consequence is that our estimates are subject to non-trivial margins of error.

25. 7. conclusionS (cont’d) • Don’t over interpret small differences in performance (across countries or over time). • Presence of errors does not mean that the WGI cannot be used to make meaningful comparisons. • Estimation of and emphasis on such margins of error is intended to enable users to make more sophisticated use of imperfect information.

26. 7. conclusionS (cont’d) • In fact it is possible to make meaningful comparisons cross-country and over time. • Almost 67% of all cross-country comparisons in 2009 result in highly significant differences at 90% confidence intervals. • More than one quarter of countries show a significant change in at least one of the six WGI measures during the 2000-2009 decade.

27. THANK YOU