400 likes | 567 Vues
MODEL ACADEMIC CURRICULUM MODULE 13. Assessing and Evaluating Responses. Module 13 Components. Assessment and Evaluation Conducting Community Surveys. Introduction. Did the problem decline? If so, did the response cause the decline?.
E N D
MODEL ACADEMIC CURRICULUMMODULE 13 Assessing and Evaluating Responses
Module 13 Components • Assessment and Evaluation • Conducting Community Surveys
Introduction • Did the problem decline? • If so, did the response cause the decline? The purpose of assessing a problem-solving effort is to help you make better decisions by answering two specific questions:
Reviewing the SARA Model • SCANNING • ANALYSIS • RESPONSE • ASSESSMENT
Types of Evaluations • Process Evaluations • Impact Evaluations
Conducting Process Evaluations • Observe response implementation • Interview relevant parties • Conduct focus groups • Conduct surveys
Conducting Process Evaluations • Record decisions and progress in meeting minutes • Develop a timeline of important achievements • Document everything!!
Problem Solving Project - Process EvaluationCalifornia Highway Patrol Problem - Traffic Collisions • Environmental surveys - Personal inspection of 20 miles of roadway. • Key recommendation - 24-hour headlights-on policy.
Conducting Impact Evaluations Measures • Quantitative Measures • Qualitative Measures • Measurement Validity • Selecting Valid Measures
Criteria for Claiming Causality • There Is a Plausible Explanation of How the Response Reduces the Problem • The Response and the Level of the Problem Are Related • The Response Occurs Before the Problem Declines • There Are No Plausible Alternative Explanations
Practical Limitations to Interrupted Time Series Designs • Measurement is expensive or difficult. • Data are unavailable for many periods before the response. • Decision-makers cannot wait for sufficient time to elapse after the response. • Data recording practices have changed, making inter-period comparisons invalid. • Problem events are rare for short time intervals, forcing you to use fewer, longer intervals.
Spatial Displacement of Crime or Disorder, and Spatial Diffusion of Crime Prevention Benefits
Problem-Solving Evaluation Checklist • Early Considerations • Process Evaluation • Impact Evaluation • Evaluation Conclusions • Overall Impact Evaluation Conclusions
I. Early Considerations • What will the evaluation help you decide? • Do you know the problem? • Do you know how the response works?
II. Process Evaluation • Did you implement the response? • Did you implement enough of the response?
III. Impact Evaluation • Do you need a control group? • How often can you measure the problem? • What type of evaluation design should you use? • What type of control group do you need?
IV. Evaluation Conclusions • What are your findings from the process evaluation? • What are your findings from the impact evaluation?
V. Overall Impact Evaluation Conclusions • Did the problem decline after the response? • If the problem did decline, did it do so at a faster rate after the response than before the response? • If the problem did decline, can you rule out all other plausible explanations for the decline, other than the response? Use your list of differences between the response and control groups to help answer this question.
General Use of Surveys CJ Researchers • to better understand crime and public fear of crime Social Scientists and Political Pollsters • to learn about social relations and predict future events Government Agencies • to predict economic trends and how people react to policy Police • to measure public opinion and operational effectiveness
Specific Uses for Community-based Surveys • Gather information about public attitudes regarding police or crime • Detect and analyze problems in neighborhoods • Evaluate problem solving efforts and other interventions by taking baseline measures • Control crime and fear of crime
Survey Process Basic Sampling Designs Simple random sampling – ensures that everyone in the population has a chance to be included in the sampling bias Nonrandom sampling – sometimes is the best option available given the time and other limits How many will be sampled? - Sample size, sampling error
Methods for Contacting Respondents • Mail surveys • Telephone surveys • In-person interviews • Internet-based surveys • Officer/Deputy or volunteer delivery • Surveys are filled out at a designated location
Asking Questions • General considerations • How to ask questions • Open ended vs. close ended questions • Designing the questionnaire
Analysis of the Data • Characteristics of the sample • Does the sample represent the population? • Making inferences about the population • Estimating relationships • Significance testing