Making a Difference: Measuring Your Outcomes

1. Making a Difference:Measuring Your Outcomes Montgomery County Volunteer Center February 4, 2014 Pam Saussy and Barry Seltser, Consultants

2. Today’s Goals • Help you think about outcomes • Give you an opportunity to develop your own outcomes • Reduce anxiety

3. Measuring Success • “At the end of the year how do we know we accomplished our goals?” • Outputs, such as: • Number of program offerings • Materials provided to clients • Outcomes, such as: • Increased client knowledge • Living status improvement

4. Why do outcomes matter? • Your funders care • Your program needs them to get better • You care about them

5. What do your stakeholders need? • Executive Director and staff • Board members • Current and potential funders • Partners • Clients Handout 1—Stakeholder Needs

6. The Logic of Outcomes • Logic Model approach to understanding your program • For each program, connect potential outcomes to: • Goal of the program (why does it exist?) • Inputs/Resources (who and what is being invested?) • Activities(what are you doing?) • Outputs(what is being produced, what services are provided, what products are offered or delivered?)

7. Logic Model Ladder

8. Questions to Help Generate Outcomes • Handout 2– Questions to Help Generate Outcomes

9. Outcome Indicators • Indicators need to be: • Specific and measurable • Relevant to mission and goals • Feasible to collect • Useful • Accurate, valid, and reliable If possible, make your indicators attributable to your program outcomes

10. Classifying Outcomes • Handout 3: “A Classification of Outcome Types” • Handout 4: A Logic Model Framework for Outcomes • The value of “sequenced” outcomes

11. Exercise: Identifying Outcomes • Use Handout 5 to identify 2-3 specific sequenced outcomes you use or would like to use • Discuss some examples with the full group

12. Setting Standards/Targets • Try to find a balance between “realistic” and “stretch” standards • Use your targets to track improvements or gaps Are you meeting them? If so, what changes could you make to meet even higher ones? If not, why, and what could you change to get closer?

13. Evidence • What information will you need to collect and analyze? • How will you do this? • How strong is the evidence likely to be? • Is it feasible to collect and analyze it?

14. Measurement and Collection Issues • Measurement Challenges (Handout 6) • Tips for Collecting Information (Handout 7)

15. Evidence and Belief • “…often enough our faith beforehand in an uncertified result is the only thing that makes the result come true.” – William James • The power and temptation of belief Outcomes Measurement

16. How Much Evidence of Success Do You Need? • Certainty or Strong Evidence • Usually requires control or comparison group, pre-and post-measures, and many resources • Likelihood of Success • Usually requires pre- and post-measures, a strong logic model, and preferably a comparison group • Examples or Suggestions of Success • Can rely on anecdotal stories, personal reports, evidence of post-program conditions

17. Assessing Your Measures • Indicator Assessment Instrument (Handout 8)

18. Summary of Key Points • Start where you are, and develop stronger measures over time • Think sequentially about your outcomes • Outcome information should help you improve—outcomes are your tools, not your masters

19. Other Help and Resources • Consult experts in measurement and evaluation design as needed • Use internet, libraries, other organizations to find examples and measurement tools • Handout 9

20. Thank you for your participation! For information or help, please contact: Kathleen.stobie@ montgomerycountymd.gov