Community Engaged Research: Measurement of Process and Outcomes

1. Community Engaged Research: Measurement of Process and Outcomes Julie E. Lucero MPH, PhD(c) University of New Mexico jelucero@salud.unm.edu January 24, 2012 Community–Engaged Research Outcomes Workgroup

2. Gaps to Fill • Community engaged research (CEnR) is increasing • Gaps remain in how processes work • Gaps remain in how success of partnerships are evaluated. • Lack of conceptual models that are empirically tested and validated.

3. CBPR Model • Wallerstein (2008) proposed a CBPR model • NCMHD funding for 3- year pilot project • University of New Mexico and Washington • Literature review 2004-2008, expanded the 2004 AHRQ • Internet survey with 96 CBPR project participants • Interpretation with National Community Advisory Boord • Model provides evaluation framework • four major components: context, group dynamics, interventions, and outcomes.

4. Examples of items in Matrix • The political and social climate seem "right" for starting a collaborative project like this one. • I have a lot of respect for the other people involved in this collaboration. • I can talk openly and honestly at the Board meetings. • I am comfortable expressing my point of view. • The partnership has been successful at preparing tribal leadership. • To what extent has your partnership strengthened ATOD-related policies and regulations in the community?‘ • The partnership positively influences community health.

5. Instrument Matrix • Attempting to fill the gap of how success of partnerships are evaluated. • The goal was to provide a matrix of available instruments/measures for researchers • to evaluate their own partnerships’ success, and • to point to future research needs to uncover the quality and usefulness of instruments and their measures

6. Methods • Literature search- 258 articles • Instruments measuring the distinct “processes” and “outcomes” • 46 unique studies totaling 224 measures • Instrument = whole tool developed by a single author(s) • Measures = specific parts of the instrument • Detailed description see: • Sandoval JA, Lucero J, Oetzel J, Avila M, Belone L, Mau M, Pearson C, Tafoya G, Duran B, Iglesias Rios L, Wallerstein N. (2011)Process and outcome constructs for evaluating community-based participatory research projects: a matrix of existing measures. Health Educ Res.  PMID: 21940460 • http://hsc.unm.edu/SOM/fcm/cpr/cbprmodel.shtml

7. Example: The Wilder Collaboration Factors inventory (Mattessich, Monsey, Murray-Close 2001) • Collaboration assessment-self guided • 40 item instrument-20 research-tested success factors • CBPR model used to organize • Community Capacity • Favorable political and social climate (2) • History of collaboration (2) • Collaborative group legitimate leader (2) • Group Dynamics • Open and frequent communication (3) • Mutual respect, understanding and trust (2) • Flexibility (2) • Outcomes • Members see collaboration as self-interest (1)

8. Results • 28 measures for Context • least number of identified measures • Community Capacity (11) • 162 measure for Group Dynamics • Highest number of measures • Participatory Decision Making & Negotiation (24) • 22 Outcomes • Empowerment & Community Capacity (22)

9. Examples of items in Matrix • The political and social climate seem "right" for starting a collaborative project like this one. • I have a lot of respect for the other people involved in this collaboration. • I can talk openly and honestly at the Board meetings. • I am comfortable expressing my point of view. • The partnership has been successful at preparing tribal leadership. • To what extent has your partnership strengthened ATOD-related policies and regulations in the community?‘ • The partnership positively influences community health.

10. What did we learn? • Only 25% of the measures evaluated had reliability or validity information. • Longitudinal evaluation is lacking. • Important partnership factors like trust are understudied. • Measures focus on researcher characteristics like dependability, reliability, approachability, communication skills. • CDC partnership trust tool (CDC Prevention Research Centers 2008) • Coalition Self Assessment Survey (Kenney and Sofaer 2000) • Documentation and Evaluation of CBPR Partnerships (Israel, Lantz et al. 2005) • Evaluating dimensions of group dynamics within CBPR (Schulz, Israel et al. 2003)

11. From the literature • Trust is dynamic, not static • Is directly correlated to time • Easier to lose trust than to earn it • Dependent on • Person doing the trusting • The situation (historical context) • The person getting the trust • Current measures focus on this aspect. • Our current research attempts to get at all three aspects and improve on longitudinal measure

13. Research for Improved Health: A National Study of Community-Academic Partnerships • Partners: • National Congress of American Indians Policy Research Center (Sarah Hicks, PI) • University of New Mexico Center for Participatory Research (Nina Wallerstein, co-PI) • University of Washington Indigenous Wellness Research Institute (Bonnie Duran, co-PI) • Funder: NIH/Indian Health Service partnership, Native American Research Centers for Health (NARCH) • 2009-2013

14. CBPR Conceptual Model