Validity and Reproducibility of Physical Activity Questionnaires

1. Validity and Reproducibility of Physical Activity Questionnaires Lisa Chasan-Taber,Sc.D.Patty S. Freedson, Ph.D.

2. Chasan-Taber Physical activity and pregnancy Effect on maternal/fetal disorders Development & validation of measurement tools Reproductive & perinatal epidemiology Women's health Freedson Children's responses to exercise Exercise and women's health Determinants of performance and fitness Development and validation of objective techniques to assess habitual physical activity Research Interests

3. Learning Objectives • To understand the need for evaluation of physical activity questionnaires • To understand the role of subjective and objective measures of physical activity in validity/reproducibility studies. • To understand approaches for analyzing validity/reproducibility studies.

4. Performance Objectives • Students should be able to design a study to assess the validity & reproducibility of a physical activity questionnaire. • Be able to interpret the findings from a validity/reproducibility study. • Be able to present findings from a reproducibility/validity study.

5. Overview • Need for Evaluation of Physical Activity Questionnaires • Reproducibility Studies: Design, Analysis, & Presentation • Validity Studies: Design, Analysis, & Presentation • Future Directions

6. Associations Between Physical Activity and Disease RR Sedentary Occasional Conditioning Exercisers Exercisers Kujala, UM et al., JAMA 279:440-444, 1998

7. Why are Questionnaires used to Measure Physical Activity in Epidemiologic Studies? • Advantages • Practical for large sample sizes • Non-reactive • Tailored to specific populations and time periods • Disadvantages • Precision

8. The Need for Validation of Physical Activity Questionnaires • Physical activity is a complex behavior difficult to measure accurately • Based upon self-report • Includes unstructured activities • Individuals rarely make clear changes in their activity at identifiable points in time. • Activity intensity may vary from person to person

9. Definition of Terms: Reproducibility The consistency of questionnaire measurements • on more than one administration, • to the same people, • at different times.

10. Definition of Terms: Validity The degree to which the questionnaire actually measures the aspect of physical activity that it was designed to measure. • Type • Frequency • Duration • Intensity

11. When to Perform Reproducibility/Validity Studies? • For new questionnaires. • When a questionnaire is modified. • When questionnaires will be used in a different population according to: • Age • Gender • Culture • Physical activity profile

12. Design of Reproducibility Studies Specify time interval

13. Analysis of Reproducibility Studies ICC Total Activity 0.81 By Intensity Light - Intensity Activity 0.64 Moderate - Intensity Activity 0.78 Vigorous - Intensity Activity 0.85 By Type Occupational Activity Recreational Activity 0.78 Household Activity 0.73

14. Interpretation of Results from Reproducibility Studies High reliability Low validity High reliability High validity Figure 1. Figure 2.

15. Validity Studies: Choice of a Comparison Measure • Use superior method as a comparison. • No gold standard exists. • Use a method with different sources of error.

16. Comparison Measures • Subjective • Physical activity logs (PALs) • 24 hour recall • 7 day recall • Previous month recall • Previous year recall • Objective • Accelerometers • Heart rate monitoring • Doubly labeled water • Direct observation

17. Subjective Comparison Measures- Physical Activity Logs (PALs)

18. Advantages and Disadvantages of PALs as Comparison Measures • Advantages • Open-ended • Alleviate memory difficulties • Disadvantages • Subject cooperation and motivation • May alter normal activity • Share error of self report

19. Computation of Summary Estimates from Subjective Measures Frequency 1 times/day Duration 2 hours/day Intensity 5 METs X X Total Energy Expenditure 30 MET-hrs/day

20. Objective Comparison Measures - Accelerometers

21. Example Accelerometer Data

22. Computation of Summary Estimates from Accelerometer <3 METS 3-6 METS 6-9 METS Accelerometer Counts >9 METS METS

23. Example of Summary Estimates from Accelerometer Day Min Min Total X Counts/ Mod Vig Counts Min 1 52 130 1247890 1122 2 16 141 1181263 1238 3 14 138 1135351 1169

24. Design of Validity Studies Questionnaire Questionnaire 2nd Admin. 1st Admin. Comparison Comparison Comparison Comparison measure measure measure measure Winter Spring Summer Fall

25. Design of a Validation Study (Continued) • The number of administrations of the comparison method should be based upon • Questionnaire time frame • Variation in physical activity • Subject burden

26. Data Analysis & Presentation P earson Correlation Coefficients Total Activity 0.50 By Intensity Light 0.45 Moderate 0.55 Vigorous 0.70 By Type Occupational 0.47 Recreational 0.60 Household 0.58

27. Interpretation of Results from Validity Studies • High validity may reflect correlated error • If both omit a source of activity • If both include questions that are misinterpreted • If both rely upon self-report

28. Summary • Evaluation of physical activity questionnaires is critical • Goal is to assess typical or usual activity • A variety of comparison methods are available • Study design should be tailored to the specific population • Interpretation of results should be informed

29. Future Directions • How much physical activity (dose) is needed for health (response)? • Intensity • Duration • Frequency • Improvements and standardization of methods to assess and validate physical activity. • How sensitive are self-report and objective measures in detecting change consequent to physical activity interventions?