Key Performance Measures, Evaluation Plans, and Work Plan

1. Key Performance Measures, Evaluation Plans, and Work Plan Shelley Kuklish Epidemiologist Arizona Nutrition Network

2. Session Objectives By the end of this session, participants will be able to: • Complete the Key Performance Measures, Evaluation Plans, and Work Plan Sections of the application • Write S.M.A.R.T. objectives • Write succinct action steps • Identify an evaluation plan for each objective

3. Application • It is recommended that you: • Complete Work Plan (Appendix B) before completing Key Performance Measures and Evaluation Plans on the “Project Overview” sheet

5. Housekeeping • We will be working on a ‘sample’ work plan one step at a time. • Goal is that you leave here with at least one objective, one action step and one evaluation plan for that objective. • Can be used as an example for writing the rest of your FY10 work plan • During the practice, Network staff will walk around to help as necessary. • Be polite and tolerant of other’s opinions. • The goal of this training is to teach you HOWto write your work plan, not WHAT you are doing to teach SNAP-ED.

6. Work Plan

9. Writing Objectives

11. Goals vs. Objectives • A GOAL is: • A broad statement of intent • Non-specific, non-measurable • Example: • Increase the proportion of SNAP recipients and eligibles aged two years and older who meet dietary recommendations for fruit and vegetable consumption.

12. Goals vs. Objectives • An OBJECTIVE is: • A realistic target for the program • Answers: Who will do how much of what by when? • Example: • By the end of the class series, 50% of low income 3rd grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption.

13. S.M.A.R.T. Objectives • S.M.A.R.T. objectives are: • Specific • Measurable • Achievable • Relevant • Time Bound

14. Specific Measurable Achievable Relevant Time Bound What are you going to do, with or for whom? Focus on results, not how they are achieved.

15. Specific Measurable Achievable Relevant Time Bound Is the objective measurable? Quality Resources Quantity Time

16. Specific Measurable Achievable Relevant Time Bound Can you achieve this objective? Set objective after considering the baseline Keep objectives within reach Is this objective realistic? Can we do this? Why are we doing this?

17. Specific Measurable Achievable Relevant Time Bound Is this objective logical as it relates to SNAP-Ed goals?

18. Specific Measurable Achievable Relevant Time Bound What is the timeframe for this objective? For the Network purposes, the time frame generally is by September 30, 2010

19. Objectives Apply to 3 Areas • Process (Operational) • Impact (Knowledge) • Outcome (Behaviors)

20. Process Objectives(Operational) • Tells what you are doing and how you will do it • Tracks participants, interactions and activities • Example: • By September 30, 2010 at least 5,000 Fun Food News will be distributed to SNAP-Ed participants at community events.

21. Impact Objectives(Knowledge) • Tells how you will change attitudes and knowledge • Example: • By the end of the class series, 80% of low income 3rd grade students participating in the BBB classes will increase their knowledge of activities that help build strong bones.

22. Outcome Objectives(Behavior) • Tells what the long term implications of your program will be. • Example: • By the end of the class series, 50% of low income 3rd grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption.

23. Easy Way to Write a SMART Objective • Answer the following question: • Who will do • How much • Of What • By When? • Example: • By the end of the class series, 50% of low income 3rd grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption.

24. Your Turn!Write An Objective Statement • Make sure that it meets the SMART requirements • Specific • Measurable • Achievable • Relevant • Time Bound • Answer the following questions: • Who will do • How much • Of What • By When?

25. By the end of the class series, 50% • of low income 3rd grade students • participating in the FVMM classes • will report an increase in fruit • and vegetable consumption. • (FVMM survey baseline 48%) • By September 30, 2010, teach the • importance of eating fruits and • vegetables through the FVMM classes • to at least 500 low-income 3rd grade • students.

26. Writing Action Steps

28. Action Step Focus on the activities Provide the ‘How’ Objective Focuses on the results Provide the ‘What’ Action Steps vs Objectives

29. Objectives Action Steps Action Steps • Should work towards the objective. • Should provide how you will attain your objective. • Be brief and succinct.

30. Example: Objective: • By the end of the class series, 50% of low income 3rd grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption. Action Step: • 1.Teach FVMM classes to low income 3rd grade students in 18 schools. • 2. Conduct pre-/post-surveys

31. Your Turn!Write an Action Step • Using the objective statement you wrote in section 1, write at least 1 action step for this objective. • Remember: • Action steps should work towards the objective. • Should provide how you will attain your objective. • Be brief and succinct.

32. Teach FVMM classes to low income 3rd grade students in 18 schools. • By the end of the class series, 50% • of low income 3rd grade students • participating in the FVMM classes • will report an increase in fruit • and vegetable consumption. • (FVMM survey baseline 48%) Conduct pre/post-test evaluations. Coordinate with 18 schools to conduct FVMM classes. • By September 30, 2011, teach the • importance of eating fruits and vegetables • through the FVMM classes to at • least 500 low-income 3rd grade • students. Meet with teachers to schedule FVMM classes. Teach FVMM classes to low income 3rd grade students in 18 schools.

33. Writing Evaluation Plans

35. Evaluation Measures • Formative • Collects data for improvement of program/processes • Process • Evaluates that the program is being delivered as planned • Impact/Outcome • Measure change in knowledge, attitudes or behaviors

36. Formative Evaluation • Formative research is conducted before a program is designed and implemented, or while a program is being conducted. • It helps you: • Decide on and describe your target audience, • Understand the needs of the audience and the factors which influence their behavior • Helps you to design programs, services, and products to address those needs.

37. Examples of Formative Evaluation • Needs assessments • Focus group discussions • In-depth interviews • Market assessments • Teacher evaluations • Pilot programs where content of nutrition education is tested and evaluated, etc.

38. Process Evaluation • Process evaluation focuses on how the program operates, the services it delivers, and the function it carries out. • It addresses whether the program was implemented and is providing services as intended.

39. Examples of Process Evaluation • Tracking: • Number of materials distributed • Number of clients viewing a food demonstration • Number of classes held • Number of participants reached

40. Outcome/Impact Evaluation • An outcome or impact evaluation assess the change in knowledge or behaviors as a result of a particular program or intervention. • The goal of an outcome or impact evaluation is to learn how effective the intervention was in changing the target populations' attitudes, awareness, and behaviors.

41. Examples of Outcome/Impact Evaluation • Impact • Pre-/post-test showing an increase in knowledge • Outcome • Pre-/post-surveys showing an improvement in self-reported behaviors • Point-in-time surveys, tracked over time

42. Example: • Objective: • By the end of the class series, 50% of low income 3rd grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption. • Action Steps: • 1. Teach FVMM classes to low income 3rd grade students in 18 schools. • 2. Conduct pre/post-test evaluations. • Evaluation Plan: • Conduct pre- and post-surveys using FVMM consumption survey. (Baseline 48%)

43. Baselines • Clearly defined starting point from where implementation begins, improvement is measured, or comparison is made • The first data point for your evaluation • Allows you to create achievable objectives by providing a starting point. • Allows for tracking trends over time.

44. Evaluation Plans Should Tie Directly to The Objective • Process objective=process evaluation • Example: • Objective: Teach FVMM curriculum to 1,000 students in FY 2010 • Evaluation: Track number of students participating via attendance records • Impact objective=impact evaluation • Example: • Objective: 5th graders increasing knowledge of osteoporosis prevention • Evaluation: Pre/Post-test measuring knowledge • Outcome objective=outcome evaluation • Example: • Objective: 3rd graders increasing fruit and vegetable consumption • Evaluation: Pre/Post-test measuring consumption

45. Your Turn!Write An Evaluation Plan • Remember: • Evaluation plans tie to the OBJECTIVE • Process objective=process evaluation • Impact objective=impact evaluation • Outcome objective=outcome evaluation

46. Teach FVMM classes to low income 3rd grade students in 18 schools. • By the end of the class series, 50% • of low income 3rd grade students • participating in the FVMM classes • will report an increase in fruit • and vegetable consumption. • (FVMM survey baseline 48%) Conduct pre/post-test evaluations. Conduct pre- and post- surveys using FVMM consumption survey Coordinate with 18 schools to conduct FVMM classes. • By September 30, 2011, teach the • importance of eating fruits and vegetables • through the FVMM classes to at • least 500 low-income 3rd grade • students. Meet with teachers to schedule FVMM classes. Teach FVMM classes to low income 3rd grade students in 18 schools. Track number of students using attendance records

47. Things to Note • You can include evaluation plans for each action step, however at least ONE evaluation plan must directly apply to the objective.

48. Project Overview