100 likes | 218 Vues
This document outlines the essential steps for initiating evaluations, focusing on the evaluation questions, roles of evaluators and sponsors, and strategies for fostering strong relationships. It emphasizes the importance of clearly defining the evaluation criteria, engaging stakeholders, and addressing ethical considerations. Michael Scriven’s concepts of goal-free evaluation and insights on evaluation responsibilities are discussed. The document also highlights critical tasks involved in proposal writing and monitoring evaluation quality. Practical scenarios encourage critical thinking about assessment responsibilities in various contexts.
E N D
PA5462 February 2009 Getting the evaluation started Focusing the evaluation question Evaluator-sponsor responsibility & relationship
Naming Names • Michael Scriven - goal-free evaluation & “Key Evaluation Checklist” (mentioned by Kirkland on wikispaces under “articles”) consider looking a section II Preface
Naming Names • Michael Scriven - goal-free evaluation
Before we begin • What observations made in the readings • You would like explained or • Are important to remember/discuss or • You disagree with or • Surprised you
OCPS Case • What is the evaluation question? • If you were the evaluator what questions would you ask at your first meeting with the associate director? What is the purpose of these questions? • Would you encourage her to identify a criteria for judging the success of the initiative? Why or why not? • Would you meet with the cooperating organizations prior to writing the formal proposal? Why or why not? If you met with them what would you ask? • How important is it to include interviews with program consumers? What would you ask them?
OCPS Commentators • Both commentators write about the relationship between the evaluation sponsor & the evaluator. What points do they make that you agree with/disagree with? • Which ethical standards (pp. 212 ff) do you find most relevant to this case? Why? • What is the obligation of an evaluator or policy analyst to represent the point of view of marginalized populations?
Evaluator-Stakeholder Relations • What are responsibilities of the evaluator/stakeholders in • Identifying the evaluation question • Specifying the evaluation’s [potential] use • Identifying criteria and measures • Identifying subjects (who will be interviewed/surveyed) • Consider the following complicating factors • Stakeholder has no evaluation experience • Evaluation in being done for a funder (funder is evaluation sponsor) • Evaluation is being done to satisfy a funder • Can you imagine a request or an event that would cause you to withdraw for ethical reasons
Bell’s Evaluation Tasks • Writing a rational proposal: links work plan, budget, & sponsor’s requirements • Clarifying mandate: purpose, scope, resources, method, work plan, schedule • Staffing: qualifications needed & available • Getting the work done • Productive assignments • Monitoring progress • Ensuring quality & useful products
Critical Terms • Implementation Failure & Theory Failure • Criterion/standards • Content of objectives • One aim/activity per objective • One outcome/per objective • Designate a time period & place • Dietary education of pregnant women in Memphis will decrease underweight births by 20 percent by January 2010
Up-coming in February • February 9 – Tentative Work Session • February 16 – Proposals due (Presentations scheduled for 2/23, but may move things along if done on 2/16)