The Definition and Measurement of Disability: The Work of the Washington Group

1. The Definition and Measurement of Disability: The Work of the Washington Group Jennifer Madans National Center for Health Statistics, USA for the Washington Group SPECA Workshop on Disability Statistics, Dec 13-15, 2006

2. The Washington Group on Disability Statistics • In June of 2001, the UN International Seminar on the Measurement of Disability recommended the development of principles and standard forms for global indicators of disability to be used in censuses • There was a broad consensus on the need for population based measures of disability for country use and for international comparisons

3. The Washington Group was set up to: • Foster international cooperation in the area of health and disability statistics • Develop disability measures suitable for censuses and surveys that will provide basic information on disability • Untangle the web of confusing and conflicting disability estimates

4. Current Problem • Currently national censuses in developing countries use one of three types of questions that provide widely differing estimates of national prevalence of disability: • The three types of questions include • A. Generic question about the presence of a condition • B. Generic questions about the presence in the household of a person with a disability followed by a list of impairments • C. Checklist of impairments

5. Variety in National Estimates • Developing Countries: • Cyprus – 4.0% • Uganda – 1.2% • Nigeria – 0.5% • Developed Countries: • United Kingdom – 12.5%

6. Washington Group: Objectives • Develop a small set/s of general disability measures • Recommend extended set/s of items to measure disability as components of population surveys / supplements • Address methodological issues associated with disability measurement

7. Preliminary work to meet objectives • Clarify the purpose of data collection in order to identify appropriate measures • Understand choices being made when time, expenses and respondent burden limit number of questions

8. Moving from Concept to Definition to Measurement: The Conceptual Model ICF selected as the conceptual model: • Common point of reference • Common vocabulary • Does not provide an operational definition or a way to measure the concepts

9. Moving from Concept to Definition to Measurement: The Definitional Paradox • There is no single operational definition of disability • Different operational definitions lead to different estimates • The question you are trying to answer (the purpose) will determine which definition to use • Need to understand the choices that are being made when a purpose and a definition are chosen • Need to understand the choices that are being made when time, expenses and respondent burden limit number of questions

10. Purpose of Data Collection 3 major classes of purposes at aggregate level • Service Provision • Monitoring functioning in the population • Assess equalization of opportunities 2 criteria for selection of a purpose • Relevance—particularly for policy makers and program officials • Feasibility

11. Purpose: Service provision • Seeks to identify those with specific needs, usually the most serious problems • Requires detailed information about the person and the environment • Influenced by the organization and structure of service organizations within a particular culture

12. Purpose: Monitoring functioning in the population • Seeks to identify all those with activity or participation limitation • Response comparability problematic since participation is culturally and environmentally determined Population reporting work limitation

13. Purpose: Equalization of opportunities • Seeks to identify all those at greater risk than the general population for limitations in activity or participation • Disability as a demographic % Employed

14. Applying criteria to select purpose • Service provision: • Level of detail necessary not feasible in a census format • Nature of service provision varies across cultures • Monitoring functioning: • Response comparability problematic since participation is culturally and environmentally determined • Assessing equalization of opportunities: • If we conceive of disability toward the most basic elements of activity, without tying it to participation, we limit the number / types of questions thus enhancing feasibility

15. Locating Risk in the ICF Model Health Condition ACTIVITY ? Body Functions & Structure Participation Environmental Factors Personal Factors Source: ICF, WHO, 2001

16. Moving from Concept to Definition to Measurement: Measurement of equalization of opportunities • Locate the definition of disability at the most basic level of activity/participation in core domains • This level is associated with the ability or inability to carry out basic bodily operations at the level of the whole person (i.e. walking, climbing stairs, lifting packages, seeing a friend across the room)

17. Benefits of this approach • Development of a demographic means of understanding disability (can compare persons with and without disability) • Connection between disability and participation can be made during data analysis • Effectiveness of programs / policies to promote full participation can be monitored

18. Possible types of questions • Questions that measure functioning in various domains such as mobility, cognition, sensory, etc. • A qualifier would need to ascertain that the action was accomplished without human or mechanical assistance

19. Mobility Walking Climbing stairs Bending or stooping Reaching or lifting Using hands Sensory Seeing Hearing Communicating Understanding Speaking Cognitive functions Learning Remembering Making decisions Concentrating Emotional functioning Interpersonal interactions Psychological well-being Possible question options

20. Criteria for inclusion of domains • Cross cultural comparability • Suitability for self-report • Parsimony • Validity across various methodological modes

21. WG draft questions for censuses (short disability measure) Core Questions: • Do you have difficulty seeing even if wearing glasses? • Do you have difficulty hearing even if using a hearing aid? • Do you have difficulty walking or climbing stairs? • Do you have difficulty remembering or concentrating? Additional Questions: • Do you have difficulty with (self-care such as) washing all over or dressing? • Because of a physical, mental, or emotional health condition, do you have difficulty communicating (for example understanding others or others understanding you)? Response categories:No - no difficulty; Yes – some difficulty; Yes – a lot of difficulty; Cannot do at all

22. Objectives • Identify persons with similar types and degree of limitations in basic activities regardless of nationality or culture • Represent the majority (but not all) persons with limitations in basic activities in any one nation • Represent the most commonly occurring limitations in basic activities within any country • Capture persons with similar problems across countries

23. Intended use of data • Development of a demographic means of understanding disability: Can compare persons with and without disability on levels of participation in employment, education, or family life to see if persons with disability have achieved social inclusion • Monitor effectiveness of programs / policies to promote full participation • Monitor prevalence trends for persons with limitations in specific basic activity domains

24. Limitations • One set of measures will not satisfy multiple needs for disability data • Excluded populations: • Very young children • Institutionalized population • Homeless • Floating populations • It is not our purpose to: • identify every person with a disability within every community • replicate a population evaluated across a wider range of domains that would be possible with other forms of data collection

25. Constraints of the census format • Limited space • Indicated domains reflect types of questions that would identify largest population for this purpose and those most culturally comparable • Allows some flexibility for additional culturally relevant questions for specific countries

26. Pre-test implementation • Implementation protocols developed: • Objectives and evaluation plan for field / cognitive testing • Cognitive test plan • Translation • Enumerator training • Sample design issues • Plan for tabulation, analysis and report writing • Two regional workshops successfully implemented • June 20-22, 2005 / Nairobi, Kenya • September 19-20, 2005 / Rio de Janeiro, Brazil

27. Standardized testing • 15 countries participating • 13 funded via World Bank grant, 2 self-funded • Cognitive tests in 12 countries • Congo, Egypt, Gambia, India, Kenya, Lesotho, Mauritius, Philippines, Uganda, Mexico, Tanzania, Vietnam • Field tests in 2 countries • Gambia, Vietnam • Combined cognitive/field test in 3 countries • Argentina, Brazil, Paraguay

28. Pre-test objectives To determine if: • the questions are being interpreted as intended by the developers in that they are capturing the important aspects of the functional domains selected and • the questions are interpreted consistently across countries.

29. Pre-test evaluation Evaluation • Validity: • Content validity: How well WG question set compares with expanded disability measures • Criterion related validity: How well individual WG questions compare to relevant similar concept in a comparison measure • Face validity: Does the measure ‘look to be valid’? • Reliability: test/re-test

30. Cognitive test Cognitive test • Objective: to determine if questions are being interpreted as intended and if interpretation is consistent across countries • WG cognitive test more structured than usual • Ensures a greater level of standardization across test sites • Understand how the response mechanisms operate in the different countries

31. Cognitive test Components of cognitive test • Purposive samples • Asked WG questions and series of detailed questions in the same domains • Traditional cognitive probes • Collected information on response process • Analyzed problematic response patterns • Interviewer report on problems respondent had with questions

32. Detailed questions on functioning (example, cognition) WG question: • Do you have difficulty remembering or concentrating? Detailed questions: • Do you have difficulty remembering the names of people or places? • Do you have difficulty remembering appointments? • Do you have difficulty remembering how to get to familiar places? • Do you have difficulty remembering important tasks, like taking medications or paying bills? • Do you have difficulty concentrating on doing something for ten minutes? • Do you have difficulty learning a new task, for example, learning how to get to a new place? • Do you have difficulty finding solutions to problems in day to day life?

33. Information on response process • Questions to interviewers • Need to repeat any part of the question? • Any difficulty in using response options? • Did respondents ask for clarification or qualify their answers? • Follow-up probes to understand response process

34. Correlates of cognitive difficulty

35. Correlates of cognitive difficulty

36. Socio- demographic correlates of cognitive difficulty

37. Overall, questions worked well • Well understood • Responses consistent • Problem areas • Glasses clause – WG is revisiting but solution has a lot to do with translation protocol • Communication Question – WG will revisit to see if question can be simplified • Introduction to the question set

38. Field test • Conditions closely approximate how final study will be done • See how WG core questions function in different countries • Useful to compare WG set to a larger set of more detailed questions to determine whether the same population is identified by each set

39. Field test • Undertaken in Gambia, Vietnam, and South Africa • Checking for internal consistency • Results comparable to changes seen in other countries with move towards functional approach to measuring disability

40. Field test: Analysis • Constructed three thresholds for disability based on WG census question: • D1=1 if any difficulty • D2=1 if a lot • D3=1 if unable to do • Constructed three thresholds for disability based on extended questions: • ED1 • ED2 • ED3 • What was the D measure picking up compared to the ED measures?

41. Example: Mobility question • WG question: • Do you have difficulty walking or climbing steps? • D1=1 if “some difficulty”, “a lot of difficulty”, or “cannot do at all” • D2=1 if “a lot of difficulty” or “cannot do at all” • D3=1 if “cannot do at all” • Extended Questions: • Do you have difficulty going outside of your home?(0) • Do you have difficulty walking a long distance such as a kilometer (or a mile)? • ED1=1 if “some”, “a lot” or “cannot do” for either question • ED2=1 if “a lot” or “cannot do at all” for either question • ED3=1 if “cannot do at all” for either question

43. Disability data collection recommendations • Do not use the word disability • Functional approach • Separate functional domains • Scaled responses • Definition of disability can be made ex post, and hinge on domains and thresholds used

44. Meeting products and more information • Executive summaries of meetings, presentations and papers posted on the Washington Group website: • http://www.cdc.gov/nchs/citygroup.htm • Publication of key papers in a special issue of Research in Social Science and Disability