Assessing Consumer Health Vocabulary Familiarity: An Exploratory Study

1. Assessing Consumer Health Vocabulary Familiarity: An Exploratory Study Alla Keselman1,2 Tony Tse1, Jon Crowell3 Allen Browne1 Long Ngo3 Qing Zeng3 1 – US National Library of Medicine 2 – Aquilent, Inc. 3 – Harvard Medical School

3. Study Background • Consumers have difficulty with health texts

4. Study Background • Consumers have difficulty with health texts • We would like to provide support • Authoring guidelines; tools; translators • Need a way to evaluate readability • Readability formulas • Health domain is unique • Familiar long words (diabetes); unfamiliar short words (apnea)

5. Term Familiarity Likelihood Regression Model • Computational (regression) model • Each term is assigned 0 – 1 score • Algorithm basis • Empirical data • Term frequency counts from health text corpora • Term score categories • 0.8 – 1.0 score – “likely to be familiar” • 0.5 – 0.8 score – “somewhat likely to be familiar” • 0.0 – 0.5 score – “not likely to be familiar” Source: Zeng Q, Kim E, Crowell J, Tse T. A text corpora-based estimation of the familiarity of health terminology. Proc ISBMDA 2005: 184-92.

6. Objectives • Validate regression model • Test with consumers • Effect of demographic factors on familiarity • Health literacy • Education level • Relate surface-level and conceptual familiarity • Term vs. concept

7. Hypotheses • Significant effect of predicted familiarity likelihood 1. Surface-level familiarity 2. Conceptual familiarity • Significant effect of demographic factors • Surface level familiarity > conceptual

8. Survey Instrument • 45 items – hypertension, back pain, GERD (gastroesophageal reflux) • Random set of terms from MedlinePlus • Two types of test items: • Surface-level – prominent association • Surgery => knife • Concept level • Surgery => removing or repairing a body part • 45 surface questions; 15 concept questions (GERD)

9. Item Format *Modeled on the Short Assessment of Health Literacy for Spanish-speaking Adults (SAHLSA) Lee S-YD, Bender DE, Ruiz RE, Cho YI. Development of an easy-to-use Spanish health literacy test. Health Serv Res. In press.

10. Participants

11. Procedure • Demographic survey • Short Test of Functional Health Literacy in Adults (S-TOFHLA) • Familiarity test

12. Results Decrease

13. Results Decrease

14. Results

15. Predictors of Surface-Level Familiarity • Regression I • DV: surface level term familiarity • IV: Predicted Familiarity Likelihood Level, Gender, English proficiency, Highest Education Level, Age, Race, Health Literacy Level • Significant predictors • Predicted Familiarity Likelihood (P<.001) • Health Literacy (P<.001) • English Proficiency (P=.05) Confirms Hypothesis I Confirms Hypothesis II

16. Predictors of GERD Concept Familiarity • Regression II (GERD) • DV: GERD concept familiarity • IV: Predicted Familiarity Likelihood Level, GERD surface-level familiarity Gender, English proficiency, Highest Education Level, Age, Race, Health Literacy Level

17. Predictors of GERD Concept Familiarity • Regression II (GERD) • DV: GERD concept familiarity • IV: Predicted Familiarity Likelihood Level, GERD surface-level familiarity Gender, English proficiency, Highest Education Level, Age, Race, Health Literacy Level • Significant predictors • Predicted Familiarity Likelihood (P=.009) • GERD surface-level familiarity score (P<.001) • Health Literacy (P.06) - trend Confirms Hypothesis I

18. Predictors of GERD Concept Familiarity • Regression II (GERD) • DV: GERD concept familiarity • IV: Predicted Familiarity Likelihood Level, GERD surface-level familiarity Gender, English proficiency, Highest Education Level, Age, Race, Health Literacy Level • Significant predictors • Predicted Familiarity Likelihood (P=.009) • GERD surface-level familiarity score (P<.001) • Health Literacy (P.06) - trend Addresses Hypothesis III

19. Predictors of GERD Concept Familiarity • Regression II (GERD) • DV: GERD concept familiarity • IV: Predicted Familiarity Likelihood Level, GERD surface-level familiarity Gender, English proficiency, Highest Education Level, Age, Race, Health Literacy Level • Significant predictors • Predicted Familiarity Likelihood (P=.009) • GERD surface-level familiarity score (P<.001) • Health Literacy (P.06) - trend Trend for Hypothesis II

20. Relationship Between Surface Level and Concept Familiarity (GERD) • Gap between surface and concept familiarity (P=.001) • Size of gap greater for “likely” than for “unlikely” (P=.006) • Trend for “somewhat likely” vs. “unlikely” (P=.07)

21. Conclusions • Initial validity evidence for CHV familiarity model • Health readability utility • Ways to improve the model • Allow demographic corrections • Distinguish between knowledge of terms / concepts • Follow-up work • Increase sample and term pool • Education level? • Other predictors? • Work on integrated findings into health readability formula

22. Acknowledgements Intramural Research Program of the US National Library of Medicine, US National Institutes of Health NIH grant R01 LM007222-05 Ilyse Rosenberg for medical expertise Cara Hefner for help with data collection

23. Thank You!