1 / 20

Evaluation research

Evaluation research. Using research methods in combinations Policy analysis. Evaluation. Examining the effects of programs, policies and laws Programs: providing services to clients Policies: rules for handling clients, actions taken i.e., policy on deadly force for fleeing felons

jamil
Télécharger la présentation

Evaluation research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation research Using research methods in combinations Policy analysis

  2. Evaluation • Examining the effects of programs, policies and laws • Programs: providing services to clients • Policies: rules for handling clients, actions taken i.e., policy on deadly force for fleeing felons • Laws: what are the real impacts

  3. Evaluation • People assumed that particular effects would take place • However, those effects may or may not happen • There may be unanticipated side effects to a change, both good and bad

  4. Policy • Usually some issue is raised • Demands are made, and there is support and opposition • Goals should be formulated and ways established to attain those goals (objectives) • Policy output: the program, resources allocated

  5. Policy • Then the impact is determined • Did the changes in policy outputs have an impact on the problem? • Hypothesis: If some policy action is taken, then we expect a particular result • To test the hypothesis, must do an evaluation

  6. Evaluation components • Process evaluation and impact • Process • Needs assessment and monitoring part of the process evaluation • The first step should be needs assessment • Frequently this is not done

  7. Needs assessment • Is the program/policy/intervention needed? • Is the problem serious? Will it “get better” on its own? • Who are the members of the target population? (those most in need of services, deterrents, etc.) • What are their characteristics?

  8. Needs assessment • Programs will have little impact if they don’t affect the target population • Programs will have little effect if they do not take into account the characteristics of the target population

  9. Needs assessment • i.e., a school based program will not help chronically truant delinquents • Adding 11th and 12th grade classes at a reform school will not be useful if very few delinquents at the school are functioning at even a 9th grade level • Welfare to work requires child care

  10. Needs assessment • There is a need to gather information to determine the characteristics of the target population • May not be readily apparent, as the target population might not be in the system • Assessment methods: records, key informants, gatekeepers, data collection

  11. Needs • If the target population is not reached, even good programs, policies and laws may not have an impact • Weed and Seed example • Adolescent drug offender example

  12. Needs • Establishment of goals and objectives • Exercise: what are the goals of a prison • Mechanisms (objectives, activities) for meeting those goals • Develop a list

  13. In an evaluation • The research must ask • What does the program intend to accomplish? • How do existing staff determine whether they have attained their goals? • Have formal goals and objectives been identified and prioritized?

  14. Evaluation • What performance measures are currently used? • Are they adequate, or must additional measures be developed as part of the evaluation? • It would be very useful if evaluators were included at the start of a program to establish measures, seldom happens

  15. Monitoring (process) • What is actually happening in the intervention? • What happens on paper, theoretically, may be different from what actually happens • Who is actually served by the program? • Are they members of the target population?

  16. Monitoring • If they are not, the program will probably have little impact • how are people selected for inclusion, or how do they decide to participate? • How do participants differ from members of the target population? • How do potential participants differ from nonparticipants?

  17. Monitoring • How do staff feel about the program or policy? • Example: laws and prosecutors • Juvenile judges and juvenile justice programs • Is the program or policy actually being implemented?

  18. Monitoring • Are all the components being addressed? • Example of drug courts • Program components and benchmarks, means of assessing whether program components are being met

  19. Monitoring • Are there problems that were not anticipated? • i.e., lack of community support, resistance to the program or policy, reactions of criminal justice personnel, unexpected obstacles

  20. Monitoring • Unanticipated side effects • Could be positive or negative • Positive: conditions improve for people not in the target population • Mental health centers, nutrition requirements for WIC • Negative: example of net widening • Program may do something else

More Related