Evaluating the Added Value of Offering Technical Assistance Following a Training Program

1. Evaluating the Added Value of Offering Technical Assistance Following a Training Program Dana Keener, Ph.D. ICF Macro 2009 AEA Annual Meeting November 12, 2009

2. Overview • Background/context of the study • What is technical assistance? • Intervention description • Methods • Results • Conclusions

3. Study Context • CDC’s Rape Prevention and Education (RPE) Grant Program in the Division of Violence Prevention • Four regional trainings offered in summer 2006 to sexual violence prevention professionals • RPE wanted to explore the added value of offering TA after a traditional training session

4. What is Technical Assistance? • What  Intentional, individualized, tangible help, aid, or assistance • For whom  provided to an individual, organization, or community • Why  for the purpose of increasing knowledge, skills, and abilities towards the successful achievement of a particular end-goal or product

5. Four Dimensions of Technical Assistance in the Literature • Relationship quality • Individualization • Proactive design • Sufficient dosage

6. The Intervention • Purpose: to promote the application of effective training practices among sexual violence prevention professionals • Two components: • A full-day training component • A subsequent telephone-administered technical assistance component

7. Desired Outcomes of Intervention • Satisfaction with the training and TA • Motivation to use effective training practices • Knowledge and self-perceived ability to use effective training practices • Application of effective training practices

8. Training Component • 6 hour in-person training • Repeated in 4 locations • Included PowerPoint, stories, worksheets, exercises, and group discussion • Small group sizes • Atlanta = 4 participants • Chicago = 3 participants • San Diego = 9 participants • Hartford = 11 participants

9. Technical Assistance Component • Conducted via scheduled telephone conferences within 6 months of training • Two different intensity levels • Designed to extend and reinforce the training component • No new material presented • Focused on application of training content

10. Technical Assistance Component • 1-60 min group call • Scheduled 90 days after training • n=14 • 3-60 min group calls • 1-60 min individual call • Scheduled every 30-45 days after training • n=13 Low-Intensity TA (Chicago and Hartford) High-Intensity TA (Atlanta and San Diego)

11. Research Design and Methods • Longitudinal, quasi-experimental • Data collected via surveys and observations

12. Participants • 27 participants in total • Self-selected into the training session • Professional, well-educated • Involved in training others as part of their jobs

13. Group Assignment • Based on training city • Chicago and Hartford  low-intensity TA • Atlanta and San Diego  high-intensity TA • No systematic differences observed between groups at pre-test

14. Points of Measurement

15. Response Rates • Pre-Training  100% • Post-Training  100% • 6 Month Follow-up  96%

16. Measures • Demographics/Participant Characteristics • Organizational Characteristics • Satisfaction with Training and TA • TA Engagement • Effective Training Practices • Attitudes/Motivation • Self-Perceived Ability (general and specific) • Knowledge (knowledge score) • Improvement • Application

17. Results RESULTS

18. Training Outcomes • Participants were highly satisfied w/the training • Participants reported significant increases in: • Perceived ability to plan, implement and evaluate training programs • Knowledge scores • Outcomes did not vary based on size of training group • Outcomes did not diminish 6 months after the training regardless of participation in TA

19. TA Engagement • Defined as participating in at least 50% of calls offered to them • 15 engaged participants • 9 from high-intensity group • 6 from low-intensity group • Participants from the smallest two trainings were more likely to be engaged in the TA component • 6 of 7 from smallest trainings were engaged (86%) • 9 of 20 from largest trainings were engaged (45%)

20. Technical Assistance Outcomes No changes in self-perceived ability was observed from post-training to 6 month follow-up No differences found between high and low intensity groups

21. Technical Assistance Outcomes • Participants in the high-intensity TA group reported more improvement on training tasks than the low-intensity group • Engaged participants reported more improvement in training tasks than unengaged participants at six months • However, the engaged participants already had greater self-perceived abilities before the TA component

22. Technical Assistance Outcomes • Participants who were engaged in the TA fared better than participants who were not engaged at six months follow up. • BUT, most of the differences between engaged and unengaged participants preceded the TA intervention. • The results say more about who was inclined to participate in TA than they do about the impact of the TA component on the outcomes we measured.

23. Conclusions • Full-day, interactive, skills-based training with small groups (fewer than 12) can be very effective! • Brief TA that is one-size-fits-all is unlikely to contribute very much to outcomes, particularly if it follows training that was already effective. • Don’t throw in the towel on TA though! Need to test tailored TA programs under other conditions. One size DOES NOT FIT ALL!

24. Recommendations for Future TA • Conduct a brief needs assessment among training participants to identify and select appropriate participants of a follow-up TA program. • Develop individualized TA goals and objectives for each participant. • Develop an individualized TA plan for each participant to meet their goals. • Schedule TA proactively, rather than waiting to be asked for assistance.