Goals, Outcomes and Program Evaluation

1. Goals, Outcomes and Program Evaluation Community Memorial Foundation March 5, 2014

2. Agenda • Introductions • Logic Models • Goals, Outcomes, Baselines and Results • Implementation and Outcomes Evaluation • Q and A

3. Session Objectives • To define terms used in CMF responsive grants application regarding program outcomes and evaluation • To introduce tools and frameworks to facilitate the learning of program evaluation

4. Presenters • Tom Fuechtmann Program Officer, Community Memorial Foundation tfuechtmann@cmfdn.org • Jon Korfmacher, PhD Associate Professor, Erikson Institute jkorfmacher@Erikson.edu

5. Where Program Evaluation is helpful • Understand, verify or increase the impact of services on clients • Improve delivery mechanisms to be more efficient and less costly • Verify that you're doing what you think you're doing • Facilitate management really thinking about what their program is all about, including goals, how it meets goals and knowing if it has met its goals or not.

6. Where Program Evaluation is helpful • Produce data or verify results that can be used for public relations and promoting services in the community. • Produce valid comparisons between programs to decide which should be retained, e.g., in the face of pending budget cuts. • Fully examine and describe effective programs for duplication elsewhere

7. Logic Models A way to summarize how your program works: a “picture” of your program. Note: There is no one “correct” way to develop a logic model * Logic models must be submitted for requests over $25,000

8. How is a logic model used? • Logic Models have multiple functions: • Program planning • Program management • Evaluation • Communication • Within your agency • With people outside your agency

9. Logic Model Components Outcomes: Short-term Outcomes: Medium Outcomes: Long-term Inputs (What we invest) Outputs Activities (What we do) Participants (Who we serve) (Expect) (Want) (Hope) Assumptions: What is necessary for this to happen? External Factors: What else may impact outcomes?

10. Inputs (What we invest) Logic Model Components • Resources dedicated to or consumed by the project • Staff or volunteer resources • Funding • Equipment • Materials or other resources

11. Outputs Logic Model Components Inputs (What we invest) Activities (What we do) Sometimes described as activities accepted by participants Participants (Who we serve) Ex: 10-week parent training session for teen parents Ex: Teen parents attend at least 7/10 weeks of parent training

12. Logic Model Components Outcomes: Short-term Outcomes: Medium Outcomes: Long-term Inputs (What we invest) Outputs Activities (What we do) Participants (Who we serve) (Expect) (Want) (Hope) • What changes do we expect to see right away? • What changes would we want to see after that? • What changes would we hope to seeafter that?

13. Ex: College HIV prevention Short Term College students have knowledge of how HIV/AIDS is spread Intermediate College students practice safe sex Long Term Lowered incidences of HIV cases on college campuses Closer in Time Easier to Measure More attributable to Project More distant in time Harder to measure Less attributable to project

14. Congratulations!You now have a logic model! (Now what?)

15. Remember… • Logic models are working documents • Responsive programs need evolving models • They can and should be open to change • The process is as important as the product • Your logic model is only the first step of an evaluation

16. CMF Application

17. Application continued

18. Application continued

19. Goal: What you hope to achieve with this program (ex. To Reduce Homelessness) • Outcomes: Changes that will occur as a result of your work (short term, intermediate, long term). Ex: 20% of those receiving services will be in permanent housing 6 months after graduating from program. • Baseline: What is the current level of care? Ex. Only 2% are in permanent housing after a period of 6 months.

20. Evaluating Outcomes: What changes have occurred? • Outcomes: the changes you expect to see as a result of your work • Indicators: the specific way you measure an outcome. In other words, “How will I know it?”

21. Outcome Examples

22. New knowledge Increased skills Changed attitudes, opinions or values Changed motivation or aspirations Changed decisions Modified behavior Changed policies Changed conditions Outcomes: Common types of Indicators

23. Ex: HIV/AIDS Prevention Outcome Intermediate: Students practice safe sex Long Term: Number HIV/AIDS cases decrease on college campus Indicator Poor: Number of college students in program over time. Better: Percent students who talk about illness with friends Strong: Percent students reporting regular condom use

24. Evaluating Outcomes: Indicator Statements Elements of a Strong Statement: • How much? • Who? • What ? • When?

25. Evaluating Outcomes: Indicator Statements Example: “ 75% of program participants find part-time employment within six months of enrollment.” But… Have a rationale for the indicator (e.g., why 75%?)

26. Collecting Evaluation Information Questions to Ask Yourself • What resources do you have? • Quantitative or Qualitative? • Standardized or non-standardized? What makes the most sense for your program and the outcome in question?

27. CMF and Evaluation Outcome Objectives and Program Review • What is the impact and how will this move CMF’s mission forward? • What are the chances those results will be achieved? • Given the opportunities before us, is this the best use of the Foundation’s assets?

28. CMF and Evaluation (continued) • Grantees as Partners • Trust • Open Dialogue • A Learning Foundation

29. Questions and Answers

30. Additional Resources • Innovation Network www.innonet.org • Free Management Library www.managementhelp.org/evaluatn/fnl_eval.htm • University of Wisconsin- Extension (Logic Model) http://www1.uwex.edu/ces/lmcourse/.