1 / 86

Evaluation Based Program Development

Based on work by Lydia B. Blalock, Ph.D., Youth Development Specialist New Brunswick, New Jersey. Evaluation Based Program Development. Karen Barale, Pierce County Renee Overath, Kitsap County Christy Price, Grant County. From Assessment to Impact: Day 1. Introduction. Why are you here?

orsin
Télécharger la présentation

Evaluation Based Program Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Based on work by Lydia B. Blalock, Ph.D., Youth Development Specialist New Brunswick, New Jersey Evaluation Based Program Development

  2. Karen Barale, Pierce County Renee Overath, Kitsap County Christy Price, Grant County From Assessment to Impact: Day 1

  3. Introduction • Why are you here? • When I hear Program Evaluation I think . . . • Education Philosophy • Experiential: I can’t teach anything, but I can provide opportunities to learn! • There’s no such thing as a stupid question!

  4. How We Got . . . Here • Various backgrounds • Program Evaluation Philosophy • We can’t develop or perform an evaluation if the program wasn’t built upon a strong foundation! • “Train the trainer” Evaluation-Based Program Development

  5. Process Objectives If I: • Develop and presentseminar on EBPD • Develop and distribute appropriate handouts • Engage participants in group activities • Provide participants an opportunityto practice • Encouragequestions THEN

  6. ProgramDevelopment Understand how to clearly define issue program will address. Use program theory to plan activities. Write clear outcome and process objectives. Use EBPD Model to plan programs. Outcome Objectives Participants will:

  7. Program Evaluation Develop skills to evaluate programs. Select appropriate evaluation methods and designs. Select criteria to evaluate objectives. Use EBPD Model to plan evaluation. Outcome Objectives Participants will:

  8. Scholarship of Engagement Understand scholarship opportunities in EBPD process. Identify several avenues to share work. Use EBPD to plan scholarly activities. Cultural Aspects of EBPD Understand influence of culture on EBPD (personal & audience). Understand importance of learning culture of target audience/s. Select evaluation methods appropriate for diverse audiences. Outcome Objectives Participants will: Commit to putting new knowledge and skills into practice!

  9. BUT FIRST: Is it a Program or a program • Stakeholder leadership • Faculty leadership • Multi-disciplinary • Sustainable • Academic department • Student component • Research, teaching, outreach

  10. Group Assignments • Form groups of 4 • Sit with new group • Select bag • Develop program based upon contents of bag • Develop 10-15 minute (tops) program presentation

  11. Program Development Project: Requirements • Define problem • What method/s did you use • Identify stakeholders/collaborators • Develop outcome objectives • Target audience • Resources needs • Short, medium, long term objectives

  12. Program Development Project: Requirements • Plan program activities (process objectives) • Develop evaluation plan • Success indicators • Research design and data collection methods • Identify scholarship opportunities

  13. Evaluation-Based Program Development: By the Numbers

  14. Step 1. Define the Problem • Describe Current Situation • How does the world look today? • Why a Problem • Clearly articulate why a problem • Ask why until get to root of problem • Symptom vs. Cause

  15. Step 1. Define the Problem • Identify Stakeholders • Population: Who is affected? How many? • Stakeholder Leadership: interest in solving problem? • Potential Collaborators: Already working on problem?

  16. Step 1. Define the Problem • How: Evidence? Learn more? References & Facts • Learn all you can about the issue/problem – • “prove” the problem exists • Identify potential solutions • “Needs” Assessment • Need: difference between actual and target “state” • A.K.A.: Asset, Gap, etc.

  17. Small Group Activity:Needs Assessment Needs Assessments • Purpose of “needs” assessment is to clearly define or clarify issue. • What kinds of information might you need? • Who or what can provide the information? • What methods could you use to gather information? • Where can you gather or find the information?

  18. Needs Assessments • How will we gather the information? • Surveys • Interviews • Focus groups • Card sort • Archival/secondary data • Criteria for selecting methods • Consider characteristics of target and respondent groups • Choose combo of methods to gain different types of info • When • Develop target dates and timelines

  19. Using Results • Set Priorities • Factors that influence priorities • Difference between actual and target states • Degree of difficulty in addressing needs • Cost of implementing solutions • Political and other factors • Derive priorities from several data source • Priorities without resources

  20. Using Results • Select Solution Strategies to Meet Needs • Consider alternative solutions for each need • Evaluate alternatives • Select solutions to implement • Scholarship Opportunities?

  21. 15 Minute Break

  22. Step 2. Develop Outcome Objectives Ï • Who do you want to change? • What do you want to accomplish? • Do you have resources to accomplish changes?

  23. Step 2. Develop Outcome Objectives Ï • Describe Ideal Situation • How would world look w/o problem (perfect world)? • Define Target Audience/s • Who/what can we change (youth, families, policies, etc.)? • What do we know about culture/s of audience?

  24. Step 2. Develop Outcome Objectives Ï • Identify Outcome Objectives • What changes can I influence in my target audience/s? • Identify Resources • What do we need? • What do we have?

  25. Step 2. Develop Outcome Objectives Ï

  26. Writing Objectives • Specific • Will increase knowledge vs. Will know 3 sources of Vitamin C • Time delineated • At the end of the program, in 6 weeks, etc. • Measurable • 75%, all, etc.

  27. Writing Objectives • Example • Participants will increase knowledge of nutrition. • 75% of program participants will know 3 sources of Vitamin C 6 weeks after the program.

  28. Step 3. Plan Program Use “Program Theory” to develop process objectives!

  29. Program Theory: The Key  to Unlocking  Change      

  30. Small Group Activity:Process Outcome Challenge

  31. Template: Process Objectives

  32. WORKING LUNCH • Define the problem • Outcome objectives • Process objectives (program activities)

  33. Step 4. Plan Program Evaluation

  34. I dunno I think I knew why–once They make me do it Department chair Funding organization My mom I don’t Why Evaluate?

  35. Demonstrate success Understand failure Provide feedback for program improvement Process Outcome Nip problems in the bud Why Evaluate?

  36. Why We Don’t Evaluate • Too hard • Not qualified • Costs too much • Takes too much time • Don’t think about it “in time” • No one ever reads evaluation reports • Everyone “knows” it’s a good program

  37. Establish criteria What are criteria (benchmarks, indicators) for judging your item? Construct standards How well should your item perform on benchmarks (standards)? Develop criteria rating system Small Group Activity: Setting Criteria

  38. AFTER developing criteria: measure performance and compare with standards. How well does each type of item measure up against criteria? Small Group Activity:

  39. Template: Develop Success Indicators

  40. Process Indicators • How will you know you’ve successfully accomplished your process objectives?

  41. Step 5. Implement Program and Conduct Evaluation • You planned your work. • Now is the time to work the plan. • If all goes well – great! • If problems arise – check plan, revise as needed

  42. 15 Minute Break

  43. Cultural Aspects of EBPD • What is culture? • “Behaviors and beliefs characteristic of a specific social, ethnic, or age group.” • Product of our culture • Culture influences (unconsciously) what we think of people from other cultures. • Recognize cultural biases towards participants and strive to eliminate them.

  44. Cultural Aspects of EBPD • Neutralize cultural biases and prevent embarrassing (or worse) EBPD design flaws is: • Know your audience (Step 2 – Develop Outcome Objects) • Be aware of how differences between cultures may impact EBPD • Develop cross-cultural efficacy!

  45. Cross Cultural Efficacy

  46. Small Group Activity:Cultural Influences

  47. Step 6. Dissemination of Results -Extension Scholarship

  48. Step 6. Dissemination of Results - Extension Scholarship • Tenure track faculty engage in appropriate scholarly activities • Scholarship • Development and implementation of creative concepts . . . • . . . broadly communicated and validated by appropriate peers

More Related