How data quality affects poverty and inequality measurement

1. How data quality affects poverty and inequality measurement PovcalNet team DECPI The World Bank

2. Outline • Household survey data • Sampling, questionnaire design, interview methods, income/consumption aggregates • Non-response bias • Data processing and analysis • Overtime comparison • Other data • National account (NA) data • Price data • PPPs and ICP data

3. Household survey data • Household survey design • Sampling • daily dairy vs. recall • different recall periods • different income/consumption modules • Non-response bias • Common problems on data processing • Common mistakes when calculating poverty measures

4. Household survey data: sampling Malawi 1997 and 2004 household survey

5. Household survey data: sampling Malawi 1997 and 2004 household survey more than 4000 households drop from 1997 sample

6. Household survey data: sampling • Different sample size/frame will cause comparison problems. • Vietnam 2010 survey vs. previous rounds • India NSS: thick and thin rounds • Indonesia and other countries • China 2013 national household survey • census frame vs. legal resident registration • how to compare with previous rural/urban surveys?

7. Household survey data: daily dairy vs. recall Example of China SW poverty monitoring survey 1995-1996 1995 survey: one time recall method 1996 survey: daily dairy method 1995 mean income per capita: 854.56 Yuan 1996 mean income per capita: 992.74 Yuan Is there 16% increase in per capita income in one year?

8. Household survey data: daily dairy vs. recall Example of China SW poverty monitoring survey 1995-1996 1995 survey: one time recall method 1996 survey: daily dairy method 1995 mean income per capita: 854.56 Yuan 1996 mean income per capita: 992.74 Yuan Is there 16% increase in per capita income in one year? 10-15% increase is due to the switch from recall to dairy.

9. Example of China SW poverty monitoring survey 1995-1996

10. Household survey data: different recall periods Example of India NSS 55th round

11. Household survey data: different recall periods Example of India NSS 55th round Result: poverty estimates from NSS 55 are incomparable with previous years

12. Household survey data: different income/consumption modules Example of Honduras 1997 and 1999 surveys

13. Household survey data: different income/consumption modules Example of Honduras:

14. Different income modules?

15. Household survey data: different income/consumption modules Example of Ethiopia 2000 surveys:

16. Household survey data: different income/consumption modules Example of Ethiopia 2000 surveys: Reason: different consumption modules

17. Nonresponse bias in measuring poverty and inequality • High nonresponse rates of 10-30% are now common • LSMS: 0-26% nonresponse (Scott and Steele, 2002) • UK surveys: 15-30% • US: 10-20% • Concerns that the problem might be increasing

18. Nonresponse bias in measuring poverty and inequality Compliance is unlikely to be random: • Rich people have: • higher opportunity cost of time • more to hide (tax reasons) • more likely to be away from home? • multiple earners • Poorest might also not comply: • alienated from society? • homeless

19. Probability of being in UHS in 2004/05 plotted against income (n=235,000)

20. Common problems on data processing • Income/consumption aggregates • Valuing income in kind • Missing value • Outliers

21. Income/consumption aggregates

22. Income/consumption aggregates

23. Missing, zero and outliers • Never mix missing value and zero; • Examples from LAC labor force surveys • Outliers: check carefully and always keep original records • Income by sources • Sub components of consumptions

24. Argentina (urban) 2001-2010

25. Annual income growth of bottom 40% (circa 2005-10)

26. Annual per capita GDP growth is less than 1% during same period

27. Missing and outliers Examples from Colombia 2000 survey – 7% are zero income

31. Welfare indicator: income vs. consumption –LAC

32. More than 14% of people with zero income!

33. Welfare indicator: income vs. consumption –East Asia

34. Common mistakes on calculating poverty measures • Ranking variable • Weights • Household weight • Sampling weight • Outliers and missing • Adult equivalent

35. National Account data • GDP • Private consumption • Population • CPI – sample, weights change overtime • Spatial price • Currency change over time

36. PPPs and ICP data • ICP rounds: 1985, 1993, 1996 and 2005 difference in coverage • PPPs • PWT PPP and the World Bank’s PPP • GDP PPP and consumption PPP • PPPP: PPP for the poor

37. Biases in 2005 ICP • “Urban bias” in price surveys • China: 11 cities; reasonably representative of urban areas but not rural • Similar problems for Argentina, Brazil, Bolivia, Cambodia, Chile, Colombia, Pakistan, Peru, Thailand and Uruguay. • Correction using urban/rural poverty line differentials. • India: ICP surveys under-represent rural areas (only 28%) • Implicit PPPs for urban and rural India (Rs 17 and Rs 11) • PPP’s for the poor: Deaton and Dupriez have re-weighted the PPPs for sub-sample of countries with the necessary data and find similar results

38. 1600 1400 1200 1000 800 600 400 200 0 r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r u u u u u u u u u u u u China • First time China has participated in the ICP • Urban bias: prices collected from 11 cities • Correction using urban/rural poverty line differential

39. 200 250 200 150 150 National poverty line ($/month at 2005 PPP) 100 National poverty line ($/month at 2005 food PPP) 100 50 50 0 0 3 4 5 6 7 3 4 5 6 7 Log consumption per person at 2005 PPP Log consumption per person at 2005 PPP Note: See Figure 1 Note: See Figure 1 Fisher PPPP (Deaton-Dupriez) Food PPP Lower poverty line: $22.72 $1 a day line! $31.72