Focusing Your Evaluation Activities

1. Focusing Your Evaluation Activities Chapter Four cont…

2. Review • Needs assessment • Formative evaluation • Implementation evaluation • Process evaluation • Outcome evaluation • Cost analysis

3. Outcome Evaluation • Is the program effective? • What are effects of participation? • What are negative effects? • Do clients benefit or improve? • Are clients any better than they would have been?

4. Outcome Evaluation cont… • Intended outcome is positive change • Or maintaining functional ability • Or prevention programs • Consider total outcomes both pos and neg • Outcome eval in a sequential manner

5. Timing • Most programs need time to refine • Staffing and program design will change • Revision is a normal part of a new program • Outcome eval is done when stability is present • Outcome eval after one year of operation

6. Credibility • Outcome eval in a self report format • Memory loss about the program • Reporting bias on forms • Personal testimonials may be good for reports • They are not seen as highly credible • Assessment should be prior and after exercise in order to assess program impact

7. Credibility cont… • Administer multiple measures of client outcome • Compare effectiveness of different programs • Run two programs and measure each • Compare results at conclusion

8. Sequential Approach • Look at individual programs first • Look at short term first then long term • Use new clients for assessment as old ones may have already changed • Tracking is key to assessment • Determine “WHY” program is not working • Make modifications to the program

9. Cost Analysis • Program efficiency • What are costs of overhead • What is the cost per case • What is cost to produce outcomes • Capital expenses are equipment at beginning • Operating expenses are salaries and custodial

10. Cost Analysis cont… • Advertisement cost analysis • Cost per case is total cost/membership # • Cost-outcome is direct benefits to exercisers • Cost required to produce outcomes?

11. Program Goals and Objectives • What does your program want to accomplish • Goals are larger in scope • Objectives are individual items to achieve goals • Table 4.2 page 55 • Your rationale or mission statement is a program goal • Implementation objectives are proposed events

12. Goals and Objectives cont… • Service objectives are what is happening to members • Output objectives are what you want to accomplish • Outcome objectives are the most important • Outcomes are inputs vs results • Do people benefit from the program? • Positive results are outcomes

13. Performance Indicators • Results are the bottom line • Client satisfaction is not an outcome • What will you accomplish in the future? • Targets for performance • Weight loss • Fitness improvement • Return to work

14. Performance Indicators cont… • Future targets are based on past performance • Other programs can serve as benchmarks • Tracking is the best way to set targets • Don’t set too low or too high • Success is relative to objectives

15. Program Logic Model • A diagram showing objectives and performance indicators and linkages • Figure 4.2 page 58

16. Lab 2 • Visit a fitness center: • Objectives • Outcomes • Cost