Evaluation Challenges in Chile

1. Evaluation Challenges in Chile

2. Challenges in Prevention Evaluation • Prevention evaluation • Types of evaluation • Standard strategies • Indicators • Evaluation outcomes, effects and impact • Prevention process

3. ANTICIPATE STRENGTHEN MANAGE the risks PROCESS People and context In the commitment to measuring future outcomes, it is important to have evolution and a control group. Prevention programs strike along many variables that are multi-determinate upon several other factors which are simultaneously present. It is important to isolate targeted programs from other educational, family, and overlapping interventions (self-esteem, decision-making, life planning and vision, etc.) To measure values, attitudes, knowledge and skills Prevention seeks medium- and long-term effects, and implies a long educational process. Follow-up. Evaluation of people and contexts. Family, school, neighborhood. PREVENTION AND THE EVALUATION PROCESS

4. WHAT PROGRAM? • To distinguish actions, both of projects and of programs • Program. Defining a systematic, consistent intervention with long-term vision. • Project. Limited in time. • All projects should include, from their initiation, a proposal for evaluation. • In programs, it is worthwhile to invest additionally in an external evalution, that enables you to: • Validate the program from the very beginning (materials, training processes, etc). • Evaluate the process • Evaluate short- and medium-term outcomes

5. STUDIES CARRIED OUT BY CONACE • GENERAL POPULATION STUDIES (BIANNUALLY, PAIRED SINCE 1994) • SCHOOL POPULATION (BIANNUAL, UNPAIRED SINCE 1995) • STUDIES WITH TEACHERS • STUDIES WITH PRISON POPULATION • EMERGENCY ROOM STUDIES • MORGUE STUDIES • STUDIES WITH HIGH-RISK YOUTH (SENAME) • STUDIES AMONG TRANSPORTATION WORKERS (METROPOLITAN REGION) • OTHER STUDIES: TEXTUAL ANALYSIS METHODOLOGY • QUANTITATIVE: Social surveys and metabolism tests • QUALITATIVE: Textual analysis: In-depth interviews, key informants and focus groups. TYPES OF SURVEYS • Self-applied • In-person interviews

6. EVALUATIONS CARRIED OUT IN THE CHILEAN EDUCATION SYSTEM Outcome evaluations at the level of effects and impact: • Identification: Evaluate the pilot use of the pre-school level educational material “On a Treasure Hunt,” 2002. • Executed by: Center for Development and Psychosocial Stimulation Studies (CEDEP). • Identification: Evaluation of the Prevention Program “I Want to Be,” 2002-2003.Executed by: the University of Chile School of Public Health. • Identificación : Fifth National School Study, 2003 (a two-year program). Executed by: the University of Chile School of Public Health. Process Evaluations:Identification: Evaluation of standard drug prevention strategies in primary and secondary educational settings, 2000-2001. (Marori and Tutibú; To Have a Good Time; To Be and Grow; Alive and Free; Telegram to Earth). Executed by: the Bolivarian University. Evaluation of user satisfaction:Each program in the Prevention Continuum incorporates a system to gather teacher views regarding their satisfaction with the program and its implementation strategy.

8. EVALUATION OF THE “I WANT TO BE” PROGRAM The program was positively assessed by students in terms of its contents and activities, although with a lower assessment by 8th graders The program demonstrates a lower degree of acceptance by those who have already used drugs.The program contributes to an increase in the perceived risk incurred by habitual use of marijuana It helps to diminish students’ belief in myths that favor use It increases the rejection of habitual use of marijuana, and sometimes of alcohol Marijuana use is shown to be slightly lower among students who have participated in the program Students have shown significant progress in their understanding of drug issues and problems The program helps to improve some “life skills”

10. WHAT WE HAVE LEARNED • Programs must be consistent and constant over time (prevention is a process). • The usefulness of validating materials used in prevention programs. • The studies do not only allow you to measure trends, but also to research protective factors associated with protective factors and program impacts. Ex., School prevention. • To homogenize indicators used in evaluations across different programs. • A good evaluation will permit you to improve the effectiveness of the: program design, implementation methodology, and training process.

11. CHALLENGES • Incorporating evaluations from the moment when initially designing the program. • Actually utilizing the results from the evaluation to improve the scientific basis for prevention, to re-write the programs based on the measured outcomes, and to improve intervention processes. • To use control groups, to follow-up on a regular basis, and to advance in multivariate analysis that permits you to intervene more specifically in those variables or factors that best assure desired program impact. • To share successes and failures among specialists. • To learn from the evaluations. • To measure the impact of a gradual, ongoing and systematic prevention process.

12. EVOLUTION OF ILLICIT DRUG USE IN THE GENERAL POPULTION 12 TO 64 YEARS OLD, ACCORDING TO PAST-YEAR PREVALENCE Fuente: CONACE, Estudio Nacional de Drogas en la Población General de Chile, 2002

13. MORE INVOLVED PARENTS DO PREVENT DRUG USE PAST-YEAR PREVALENCE FOR USE OF ANY DRUG AMONG SCHOOLCHILDREN, ACCORDING TO PARENT HABITS

14. www.conacedrogas.cl www.conace.cl Chilean drug portal Agustinas 1235 p. 9 Santiago de Chile Tel: (56-2) 510.08.00 tizquierdo@conace.gov.cl