What information to collect?

1. What information to collect? Jan Smit Statistics Division, ESCAP With thanks to Wei Liu What information to collect?

2. Outline • Overview of ESCAP/WHO project • Disability question set testing • What questions were asked? • Sampling & sample characteristics • What info should be collected? • Comparison between Washington Group and WHO questions What information to collect?

3. ESCAP/WHO project • ESCAP/WHO Project on Improving Disability Data • 2004-2006 • Goal: improve national disability statistics by applying the ICF framework and using international measurement standards What information to collect?

4. ESCAP/WHO project (cont.) • Target group: statisticians and experts in disability from 20 countries • Expected outcome: capacity to produce better disability data that meet policy needs • Partners: Australian Bureau of Statistics, Washington Group What information to collect?

5. Project countries • Afghanistan • Brunei Darussalam • Cambodia • China • DPR Korea • Islamic Republic of Iran • Japan • Hong Kong, China • Laos • Macao, China • Republic of Korea • Singapore • Sri Lanka • Thailand • Vietnam Taking part in pilot test • Fiji • Indonesia • India • Mongolia • Philippines What information to collect?

6. Motivation for project • Biwako Millennium Framework for Action (BMF) Principle 6 to “Strengthen national capacity in data collection and analysis concerning disability statistics” • However, many data issues within ESCAP region What information to collect?

7. Regional disability data issues • Data not available to inform policy • Disability prevalence is underestimated • Traditional view of disability prevails at every layer of societies, incl. government • No comparability of data; need for use of international standards and guidelines What information to collect?

8. Lack of data comparability What information to collect?

9. Project components • Phase 1: • Training in the use of ICF for disability collection, in standard methodologies and in design of collection tools • National action plans for improving disability information systems • Preparation of disability statistics training manual & material What information to collect?

10. Project components (cont.) • Phase 2: • Field testing of standard sets of disability questions through identical protocols • Evaluation of results for application to real data collections • Building a network of experts What information to collect?

11. Main project activities • 4 regional workshops for advocating and training on the ICF approach and its implementation • Pilot testing of disability questionnaires for use in census and surveys and results analysis and dissemination • The ESCAP/WHO Disability Statistics Training Manual What information to collect?

12. Disability question set testing • Study 1: Sensitivity and specificity testing (and more) • Study 2: Test-retest reliability • Study 3: Cognitive testing What information to collect?

13. Pilot test: Study 1 • Through NSOs of five countries • Fiji (FJI) • Indonesia (IDN) • India (IND) • Mongolia (MNG) • Philippines (PHL) • Data collection period: May-July 2005 • In local languages What information to collect?

14. Questions • Demographic characteristics • 6 Washington Group questions (W) • 53 WHO questions (B and D) • 17 body structure/function questions from World Health Survey • 36 activity & participation questions from Disability Assessment Scale (DAS II) • 4 Australian Bureau of Statistics questions on need for assistance (E) What information to collect?

15. Activity & participation quest. • Understanding & communicating (D1) • Getting around (D2) • Self care (D3) • Getting along with people (D4) • Life activities (D5) • Participation in society (D6) • Incl. 2 questions (D6.2-3) that might be considered as addressing environmental factors What information to collect?

16. Sampling • Each NSO was requested to administer the questionnaire to a sample of at least 1,000 respondents of 18 years and older representative of the national population in terms of age, sex and educational attainment • Interpreted by the collaborating NSOs in different ways • Sample not representative for whole population What information to collect?

17. Sample characteristics: Sex Country Sex IND IDN PHL FJI MNG Total Female 245 449 606 527 596 2423 Male 345 351 433 468 466 2063 Total 590 800 1039 995 1062 4486 Col % Col % Col % Col % Col % Col % Female 41.5 56.1 58.3 53 56.1 54 Male 58.5 43.9 41.7 47 43.9 46 Total 100 100 100 100 100 100 What information to collect?

18. Age What information to collect?

19. Education What information to collect?

20. Maritalstatus What information to collect?

21. Employmentstatus What information to collect?

22. What info should be collected? • Depends on question(s) to be answered (already discussed) • What domains are most important? • No question about respondents overall judgement of their own health (or perhaps happiness, quality of life) • What are the main underlying ‘factors’? • (Exploratory) factor analysis What information to collect?

23. Factors • Day-to-day life (D1.3, D5.1, D5.2, D5.3, D5.4, D5.5, D5.6, D5.7, D5.8 and D6.8) • Self-care (W5, D3.1, D3.2, E1 and E2) • Resources and emotion (D6.4, D6.5, D6.6, D6.7 and D6.8) • Getting along with people (D4.1, D4.2, D4.3 and D4.4) • Mobility (W3, B1.13, B2.1, B2.2 and B2.5) What information to collect?

24. Factors (cont.) • Hearing (W2, B1.8 and B1.9) • Learning and understanding (D1.4 and D1.5) • Seeing (W1, B1.6 and B1.7) • Pain and discomfort (B1.1 and B1.2) • Sadness and anxiety (B1.16 and B1.17) What information to collect?

25. Prevalence What information to collect?

26. Sensitivity & specificity Sensitivity: __a___ a + c Specificity: __d__ b + d What information to collect?

27. Sensitivity, specificity, gamma WHO question Sensitivity Specificity Gamma Seeing B1.6 0.60 0.91 0.83 B1.7 0.63 0.90 0.84 B1.6 and B1.7 0.58 0.94 0.85 Hearing B1.8 0.39 0.98 0.91 B1.9 0.38 0.97 0.88 B1.8 and B1.9 0.34 0.98 0.90 Mobility B1.13 0.47 0.93 0.80 D2.5 0.51 0.91 0.79 B1.13 and D2.5 0.43 0.96 0.82 What information to collect?

28. Sensitivity, specificity, gamma WHO question Sensitivity Specificity Gamma Cognition D1.1 0.46 0.88 0.70 D1.2 0.48 0.92 0.79 D1.1 and D1.2 0.43 0.94 0.79 Self care D3.1 0.51 0.98 0.96 D3.2 0.58 0.98 0.96 D3.1 and D3.2 0.49 0.99 0.96 Communication D1.5 0.31 0.96 0.80 D1.6 0.31 0.96 0.80 D1.5 and D1.6 0.28 0.97 0.80 What information to collect?

29. Ongoing research… • Can info on some ‘important’ domains be recovered from a limited number of questions/variables? • E.g. can D5.5 be explained by the 6 WG questions (or WHO counterpart questions) + demographic info? • Initial results in terms of model fit promising • Difficulty is accuracy of prediction What information to collect?

30. Not only collect data… • But document and preserve data collection exercise • Instrument, scope, coverage, etc. • Response options, labels, etc. • Microdata Management Toolkit • www.internationalsurveynetwork.org • And preferably disseminate micro data (unit records) • Possibly anonymized What information to collect?