Measuring Disability in a Survey or Census Context: Parallel Work Advancing the Field

1. Measuring Disability in a Survey or Census Context: Parallel Work Advancing the Field Barbara M. Altman, Ph.D. Disability Statistics Consultant

2. Active Measurement Efforts • Washington Group – Measurement on an international level • ISDS – Federal efforts to prepare for improved data collection • ACS – Development of improved questions for Census Survey

3. Measurement Issues • Issue 1 - Addressing the problems of the wide ranging variety of questions and wording. • Issue 2 – Understanding the nature of the conceptual model serving as a basis for definition and measurement. For example, a theoretical or legislative model. • Issue 3 – How does the purpose of data collection dictate the way the conceptual element from the model is operationally defined? • Issue 4 - Is the measurement of a ‘true’ population feasible or desirable? Is there “one” true number?

4. The Washington Group on Disability Statistics • In June of 2001, the UN International Seminar on the Measurement of Disability recommended the development of principles and standard forms for global indicators of disability to be used in censuses • There was a broad consensus on the need for population based measures of disability for individual country use and for international comparisons

5. The Washington Group was set up to: • Foster international cooperation in the area of health and disability statistics • Develop disability measures suitable for censuses and surveys that will provide basic information on disability • Untangle the web of confusing and conflicting disability estimates

6. Washington Group: Objectives • Develop a small set/s of general disability measures for use in Censuses • Recommend extended set/s of items to measure disability as components of population surveys / supplements • Address methodological issues associated with disability measurement

7. Current International and National Situation • Currently national censuses in developing countries use one of three types of questions that provide widely differing estimates of national prevalence of disability: • The three types of questions include: • Generic question about the presence of a condition • Generic questions about the presence in the household of a person with a disability followed by a list of impairments • Checklist of impairments • Current problems nationally • Question domains and wording varied across surveys and census –ability to crosswalk surveys limited • Many questions in use predate improved theoretical approaches

8. Moving from Concept to Definition to Measurement: The Conceptual Model • A conceptual model of the key components and their relationship is a basic tool for developing or selecting questions. • WG selected the ICF as the conceptual model: • Common point of reference • Provides a common vocabulary, classifies domains within conceptual areas • Does not provide an operational definition or a way to measure the concepts

9. Moving from Concept to Definition to Measurement: The Definitional Paradox • There is no single operational definition of disability • Different operational definitions lead to different estimates • The question you are trying to answer (the purpose) will determine which operational definition to use • Need to understand the choices that are being made when a purpose and an operational definition are chosen • Need to understand the choices that are being made when time, expenses and respondent burden limit number of questions

10. Purpose of Data Collection 3 major classes of purposes at aggregate level • Service Provision • Monitoring functioning in the population • Assess equalization of opportunities 2 criteria for selection of a purpose • Relevance—particularly for policy makers and program officials • Feasibility

11. Purpose: Service provision • Seeks to identify those with specific needs, usually the most serious problems • Requires detailed information about the person and the environment • Influenced by the organization and structure of service organizations within a particular culture

12. Purpose: Monitoring functioning in the population • Seeks to identify all those with activity or participation limitation • Response comparability problematic since participation is culturally and environmentally determined Population reporting work limitation

13. Purpose: Equalization of opportunities • Seeks to identify all those at greater risk than the general population for limitations in activity or participation • Disability as a demographic % Employed

14. Issue 4 –What Population? • Actually, if we identify disability as an outcome of the person’s interaction with his/her environment we are seeking to identify a person with a Greater Risk ofDisability based on health related functional limitations • The Population would be defined as: persons with functional limitations who are at risk for the loss or restriction of opportunities to take part in the normal life of the community on an equal level with others due to physical, social and other environmental barriers.* *Adapted from Scott Brown’s presentation at the ISDS Meeting

15. Locating Risk in the ICF Model Health Condition ACTIVITY ? Body Functions & Structure Participation Environmental Factors Personal Factors Source: ICF, WHO, 2001

16. Body Functions & Structures Activities & Participation Environmental Factors THE ‘At Risk’ POPULATION: In Pure ICF Terms Functions Structures Capacity Performance Barriers Facilitators ICF Components

17. Moving from Concept to Definition to Measurement: Measurement of equalization of opportunities • Locate the definition of disability at the most basic level of activity/participation in core domains • This level is associated with the ability or inability to carry out basic bodily operations at the level of the whole person (i.e. walking, climbing stairs, lifting packages, seeing a friend across the room)

18. Criteria for inclusion of domains • Suitability for self-report • Parsimony • Validity across various methodological modes • WG Perspective – Cross cultural comparability • Possible domains that fulfill these criteria: • Walking • Seeing • Hearing • Cognition

19. Benefits of this approach • Development of a demographic means of understanding disability (can compare persons with and without disability) • Connection between disability and participation can be made during data analysis • Effectiveness of programs / policies to promote full participation can be monitored

20. Intended use of data • Compare levels of participation in employment, education, or family life for those with disability versus those without disability to see if persons with disability have achieved social inclusion • Monitor prevalence trends for persons with limitations in specific basic activity domains

21. Measurement Answers* • Issue 1 - Addressing the problems of the wide ranging variety of questions and wording. • For international comparability we have found it necessary to locate our population of interest within the basic levels of functioning of the whole person. If we want comparability within the US across surveys it is essential to have a similar basic standardized operational measure. • Issue 2 – Nature of the conceptual model serving as a basis for definition and measurement. For example, a theoretical or legislative definition. • Commonly accepted theoretical approaches need to underlie the conceptualization of the components of disability that are then operationalized. Legislative definitions need to be treated as “purpose” of data collection since the varieties of legislation and legislative definitions are directed at specific policy purposes. *Personal Opinion

22. Measurement Answers* • Issue 3 - Purpose of data collection effort may dictate the conceptual component from the model as well as the specific operational definition of disability. • Major disconnect occurs between data collection process, use and interpretation. While we can push for purpose related operationalization, how the measures are used in analysis requires careful planning and interpretation and understanding of the limitations of the measures. • Issue 4 - Is the measurement of a ‘true’ population feasible or desirable? Is there “one” true number? • We can’t represent the ‘true’ population but we can represent a consistent population by using a standardized subset of questions across surveys (or across countries). That is both feasible and desirable and does not preclude inclusion of other measures for specific purposes. *Personal Opinion