What are we learning about training in evidence-based public health among local practitioners?

1. What are we learning about training in evidence-based public health among local practitioners?

2. THANKS!! Ross Brownson Janet Canavese Beth Baker Katie Stamatakis Katie Duggan Carol Brownson Paul Erwin Case Western PRC Robert Fields UNC PRC Carson Smith MI PRC Rodrigo Reis U of WA PRC Carolyn Leep (NACCHO) Jenine Harris Rachel Tabak Chris Casey Amy Eyler Beth Dodson Kathleen Wojciehowski

3. What is Project LEAD? 4 Aims: • Describe the evidence-base for local EBPH in the United States (esp. A-EBPs) • Test the effectiveness of local-level EBPH capacity building in 4 states (Train the Trainer) • Describe a range of local models in EBPH • Translate and disseminate findings to stakeholders

4. Selected findings (Aim 1)

5. Administrative evidence-based practices (A-EBPs) • Agency (health department)-level structures and activities that are positively associated with performance measures (e.g., achieving core public health functions, carrying out evidence-based interventions) Brownson, Allen, Duggan, Stamatakis, & Erwin. Am J Prev Med, 2012

6. Results, 5 EBP domains • Workforce development • QI (82%), EBDM (59%) • Leadership • Staff participation (84%) hire w/ PH degree (36%) • Organizational climate and culture • Life long learning (71%), culture supports EBDM (42%) • Relationships and partnerships • Importance (92%), share resources (68%) • Financial processes • Multi funding sources (96%), QI resources (55%)

7. Predictors of a-EBPs

8. Early Results Patterns of A-EBPs and use of EBDM vary significantly based on: • Population size served by the LHD (strongest predictors after adjustment) • Smaller HDs 3X more likely to be led by a nurse • Governance structure (state governed) • Age group in the 50s

9. Use of resources • 26% of LHDs reported systematic reviews as 1 of top 3 most important resources • 34% were not familiar with the Community Guide

10. A tale of two worlds…

11. Selected findings (Aim 2)

12. Train the Trainer • November 2012, St. Louis Missouri • Recruitment of PRC’s • Geographic distribution • PRCs with established relationships with PHTCs or PBRNs • “Ready to hit the ground running”

13. Training Locations • Case Western Reserve UniversityPrevention Research Center for Healthy Neighborhoods andPBRN • University of MichiganPrevention Research Center of Michigan and Michigan Public Health Training Center • University of North Carolina at Chapel HillCenter for Health Promotion and Disease Prevention and PHTC • University of Washington Health Promotion Research Center and PHTC

14. Train the Trainer Process • Partners from PRC came to St Louis for 3 day training and overview of Evidence Based Public Health course • Conference calls with each site to set up logistics and provide technical assistance • EBPH slides and materials for trainings given to each site • PRC in conjunction with PBRN or PHTC put on training for local health departments in their area

15. Train the Trainer • Review of Module Content • Focus on “Process” for course delivery

16. Recruiting participants for the State Trainings • Emails to various public health listservs • PBRN, LHD commissioners, directors of nursing, etc. • Website postings • PRC and PHTC • Announcements and flyers at conferences • Help from state health department **Each state had waiting lists

17. Participants of Trainings • 57% program manager or coordinator, 22% top executive, administrator, or assistant director • 2% doctoral degree, 24% public health masters, 37% other masters, 9% nursing degree, 28% bachelors degree or less • Mean years in current position = 5.2 years, mean years in public health = 8.8 years • Number of organizations represented at state trainings in person = 101 (OH = 24, MI = 21, NC = 26, WA = 20)

18. Course Format Two locations used traditional in-person training and two used mixed modes for course delivery i.e. in-person and webinars for distance learning capabilities • WA – 3 one-hour webinars with 2 days of in-person training • MI – 2 onsite training days, 3 modules each day, 3 modules by webinar, 2 of which had multiple days

19. 11 EBPH Skills Evaluated (pre-training course survey) I Can… • develop a concise statement of the issue • conduct a community assessment • describe the issue in a quantitative way • discover what is known about the issue through the scientific literature • prioritize program or policy options that address the issue • understand economic evaluation to use findings from others to • assist with determining which intervention would be best suited for my community and population of interest • understand economic evaluation to assist in the design and implementation of an economic evaluation • develop an action plan for the program or policy • qualitatively evaluate the program or policy • quantitatively evaluate the program or policy • economically evaluate the program or policy

20. Pre/Post Skill Abilities(course survey) • Data show participants felt more capable of all 11 skills after the training • Differences in ability scores between pre and post evaluations were all statistically significant using p=.01 for sites evaluated

21. An Example: Skills Ability for Case Western and NC combined Pre Pre Post Post Number of responses Number of responses I can prioritize program or policy options that address the issue I can qualitatively evaluate the program or policy

22. Views Related to EBDM (pre-training survey) • I feel that I have the skills necessary for developing evidence-based interventions (No. 76, 59.4%) • I feel that I can effectively communicate information on evidence-based strategies to policy matters (No. 54, 42.2%) • I feel evidence-based interventions are packaged in a way that I can use them (No. 34, 26.6%) • I feel evidence-based interventions are designed in a way to be self-sustaining (No. 30, 23.6%) • I feel that I need to be an expert on many issues in order to effectively make evidence-based decisions (No. 22, 17.2%) • My fears about job security prevent me from using EBDM (No. 6, 4.7%)

23. Which of the following would most encourage you to utilize EBDM in your work? (pre-training survey) • Placing a high priority on EBDM in your work • Trainings on EBDM • Positive feedback for EBDM use

24. Which of the following would most encourage you to utilize EBDM in your work? • Director’s Survey • Trainings on EBDM • Placing a high priority on EBDM by leaders in my agency • Positive feedback for EBDM use • Programmatic Survey • Placing a high priority on EBDM in your work • Trainings on EBDM • Positive feedback for EBDM use

25. Gap between importance and availability of skills(pre-training survey) • Economic Evaluation 4.1 • Communicating research to policy makers 3.5 • Evaluation designs 3.3 • Adapting interventions 3.2 • Prioritizing health issues 2.7

26. What participants most liked about the course (post-training course survey) • The breadth of the topics truly encouraged use of evidence across all functions of a LHD, not just programs • Being able to concentrate on learning and having a systematic approach to follow • Presents a whole new way of thinking about public health practice

27. Lessons LearnedWebinars – advantages and disadvantages Advantages • Quicker and easier for participants • Less expensive • Flexibility of combined format was appreciated by participants Disadvantages • Opportunity for both formal and informal interaction among participants and between participants and instructors is much less • Onsite instruction was more effective

28. Take away messages from Aim 2 • All surveys point to the need for more training in EBPH • The pre/post evaluations received from the trainings indicate that participants feel more competent in EBPH skills after training • EBPH trainings = improved skills, ability and increased potential for change in local health departments

29. Ideas going forward… • Continue analyses, featuring more complex data linkage • Explore context and meaning of Aim 1 gaps • Develop new approaches for these populations (e.g., findings for nurses) • Explore new approaches for information seeking • Enhance Aim 2 additional technical assistance (beyond the EBPH courses) • Scaling up findings from Aim 2 • Use Aim 3 findings to find and test new leverage points • Link more closely with AcademyHealth for dissemination