ESTABLISHING RECIDIVISM BASELINES AND BENCHMARKS G. Brown J. Wright K. Underhill

1. ESTABLISHING RECIDIVISM BASELINES AND BENCHMARKS G. Brown J. Wright K. Underhill Ministry of Correctional Services Program Effectiveness Unit

2. CONTEXT

3. The Ministry of Correctional Services values very strongly the inclusion of recidivism as an outcome measure for multi-measure and source quasi-experimental evaluations of new offender test programs (e.g. Project Turnaround, core programs emanating from Program Rationalization Projects). Additionally, and equally important, the Ministry has recently begun to include recidivism in Published and Internal Performance Indicators as part of the government planning process.

4. The Canadian Centre for Justice Statistics (CCJS) • began a multi-year project to develop a national • definition and formula for the measurement of • recidivism in 1998. • It is hoped that the CCJS project's end-result will • be a definition which will lead to fairly • ‘apples’ to ‘apples’ comparison of recidivism • between the different provinces and territories. • It is unlikely however that the CCJS project will be • completed satisfactorily in the near future.

5. At a May 2000 Correctional Executive Committee (CEC) • meeting, senior managers were unanimous regarding • their interest in having a ‘straight forward’ recidivism • benchmarking system put into place, as soon as possible. • The CEC also expressed an interest in having • correctional administrators use the • Offender Management System (OMS) and • later SYSCON to aggregate the recidivism rates for • the offenders under their custody and control, • for comparison against the established benchmarks.

6. There is no generally accepted formula for measuring • recidivism, or for establishing recidivism baselines • or benchmarks. • There are significant differences across • jurisdictions in the type of data available to measure • recidivism, and in the methods used to establish • baseline and benchmark measures.

7. RECIDIVISM BASELINES AND BENCHMARKING: CONCEPTUAL APPROACH

8. Recidivism • Recidivismis defined as a return to correctional supervision following a conviction for a criminal offence committed either during or after correctional supervision. Recidivism is distinguished from serious non-compliance, serious violations of the terms of probation, a conditional sentence, or a conditional release order. • Recidivism can be tracked for any length of time. Operationally, follow-up periods of two or three years are the norm.

9. Baseline • A baselineis a measure of performance taken prior to an intervention or treatment. The purpose of the baseline measurement is to determine the quantity and/or quality of a thing as it is normally or typically observed, before any change or intervention is initiated. • There are two ways to measure a behavioural baseline. A sample can be taken at one point in time, and an estimate can be constructed, or a measure of the behaviour can be taken at multiple points in time.

10. Benchmarking • Benchmarkingis a method for identifying an achievement standard against which to measure progress. • Competitive benchmarking involves drawing comparisons between an organization’s characteristics and performance with other similar organizations that are considered to be the ‘best’ in the field.

11. Benchmarking (cont’d) • The outcome of a benchmarking exercise is the identification of a standard of performance the organization should strive to achieve, and a plan for implementing ‘best practices’ to achieve the standard.

12. RECIDIVISM BASELINES AND BECHMARKING METHODOLOGICAL APPROACH

13. Calculatingthe recidivism rate • A variety of techniques have been used to calculate the rate of recidivism. Survival analysis is a technique for measuring recidivism that is being used with increasing frequency. • The population-specific recidivism rate is the rate calculated for sub-groups in the offender population

14. Calculating the recidivism rate (cont’d) • The system-wide recidivism rate is the rate calculated for the entire offender population. The system-wide rate is calculated by estimating a ‘weighted’ average of sub-population rates. • The standard recidivism rate is calculated at two-year post-release.

15. Calculating the baseline recidivism rate • Establishment of a baseline measure is the first step in the benchmarking process • A simple ‘moving average’ will be used to calculate the baseline recidivism rate. The baseline will be calculated by averaging the standard recidivism rate over three periods (six years). Each time a new two- year standard rate is available, the baseline will be updated. • A retrospective random sample of adult and young offender cases has already been drawn from the OMS database, and is currently being tracked.

16. Establishing the recidivism benchmark • Units of analysis and measurement methodology are key issues to address in constructing benchmarks. • The recidivism benchmarks will be constructed using both quantitative and qualitative strategies. Baseline information will be combined with information from other jurisdictions, and with relevant research and government studies. Operational expertise will be called on to contribute to and advise on the setting of benchmarks.

17. Key Interpretations • The recidivism rate is an estimate, not a precise count. • The baseline recidivism rate is an estimate of the 'normal' or typical rate of recidivism characteristic of the system. Attention should be focused on interpreting longer-term trends in the baseline rate, and evidence of abrupt, severe changes in the baseline rate that can indicate important changes in recidivism dynamics.

18. Key Interpretations (cont’d) • The recidivism rate is only one measure of correctional performance. It is a ‘global’ measure of the rehabilitative effect of the correctional system on offenders, a product of a variety of policies, programs and activities designed to have an (indirect) impact on recidivism. • The recidivism rate can also be interpreted as one measure of the cost efficiency of the correctional system in reducing the financial impact of re-admissions.

19. Key Interpretations (cont’d) • The recidivism benchmark is an ideal standard that • the correctional system should strive for. Achievement • is not measured by whether or not the benchmark is • reached, but by the progress made. Best practices • benchmarking is a continuous improvement process. • To measure, to monitor, to continuously improve.

20. RECIDIVISM BASELINES AND BENCHMARKING: OPERATIONAL APPROACH

21. Reference Populations • Population specific - sex - age - Aboriginal status - type of correctional intervention - offence type • System wide

22. Reference Populations • Correlate specific (LSI-OR & possibly other) - prior recidivism - time under supervision - education level - unemployment - family/marital - pro-criminal/anti-social companions - pro-criminal/anti-social attitudes - substance abuse - leisure/recreation problems - anti-social behaviour patterns

23. Sampling Strategy • A retrospective random sample of adult and young offender cases has been drawn fromthe OMS database according to definitional criteria and specific populations. Continuous monitoring of the sample from 1992/93 cases will occur. Additional update samples will be drawn at two-year intervals to capture changes in offender characteristics or system functioning. Over time, the total sample size will grow, making recidivism estimates more 'accurate' from a statistical perspective. In addition, as the total sample size under study grows, it will be possible to test for increasing 'fine-grained' differences in offender populations.

24. Offenders whose recidivism rates will be tracked are • defined as adults and young offenders (male and female) • who are sentenced to a period of supervision of • no less than six months. This sentence length “cut-off” • generally allows for a minimum program dosage of 3 • months. The offender treatment literature indicates that • for any kind of desirable program effect to occur, which • might be attributable to some favorable change in • recidivism, approximately 3 to 9 months of program • dosage is required. Remand offenders will therefore not • be tracked because of issues of limited days stay and • inaccessibility to programming.

25. Recidivism rates for offenders who will be tracked will be reported for different survival periods. Survival analysis involves the reporting of recidivism rates for several time periods post intervention. With this model, survival periods will include: 3 months, 6 months, 9 months and 12 months post intervention.

26. RECIDIVISM BASELINES AND BENCHMARKING: ORGANIZATION REQUIREMENTS

27. Role of Statistical Services Unit • The Statistical Services Unit will be responsible for: • Initial Set-up • programming to extract data from the Statistical Research Service (SRS) to generate individual site recidivism follow-up reports; • discharge list maintenance, set-up and conversion;

28. completing a verification process of the efficacy of the report code (e.g. culling out confounding information such as offender deaths, inter-provincial transfers, dual-status offences, etc.); • hiring and training 1 full-time staff person who would be required to perform all statistical liaison, coordination, analysis and reporting.

29. Ongoing • Post implementation, and on an ongoing basis, the • Statistical Services Unit would be responsible for: • collecting discharge lists from individual program sites, on an ongoing basis, and maintaining, updating and converting the lists; • conducting regular quality assurance of the efficacy of the report code; • running quarterly recidivism follow-up reports for each of the program sites;

30. disseminating recidivism follow-up reports; • working closely with the Program Effectiveness Unit and individual site managers to develop action plans for corrective action where anomalous recidivism findings occurred; and, • conducting a simple moving average calculation of both the system-wide and population – specific baseline recidivism rates every two years.

31. Role of Program Effectiveness Unit • The Program Effectiveness Unit will be responsible for: • Initial Start-up • facilitating the setting of recidivism benchmarks; • sharing Ministry recidivism benchmarks with field and senior managers; • developing policy and procedures for recidivism performance measurement;

32. marketing and educating the field and senior managers about the purpose, model and policy and procedures for recidivism performance measurement; and, • hiring and training 1 full-time staff person to work with the field to understand and correct for operational variables that may be impacting negatively upon desirable recidivism rates.

33. Ongoing • Post implementation, and on an ongoing basis, the • Program Effectiveness Unit would be responsible for: • analyzing reasons for individual site and aggregate level recidivism rates in excess of established benchmarks; • developing and monitoring action plans to address systemic variables that might be negatively impacting aggregate recidivism rates;

34. developing, in cooperation with individual site managers, action plans to correct for variables that might be negatively impacting upon an individual site’s recidivism rate; • sharing information through the Program Effectiveness Newsletter, of evidence and reasons for program successes as well as suggestions for program enhancement; and, • reconvening panel groups to re-set recidivism benchmarks every two years.

35. Role of the Field • Individual site managers and staff would be responsible • for complying with established policy and procedures for • recidivism performance measurement including: • providing discharge lists to the Statistical Services Unit on a quarterly basis; and, • cooperating with the Program Effectiveness Unit, and fully participating in a process of program correction/understanding resulting from negative or anomalous recidivism follow-up findings.

36. Role of Regional Offices • Regional Offices would be required to initiate and • maintain appropriate methods of ensuring compliance by • individual program sites with the recidivism performance • measurement policy and procedures.

37. PRELIMINARY RESULTS

38. 1992/93 Release Sample: Adult Sentences to Incarceration Cumulative Percentage of Cases Reconvicted

39. 1992/93 Release Sample: Adult Probation Cumulative Percentage of Cases Reconvicted

40. 1992/93 Release Sample: Young Offender Custody Cumulative Percentage of Cases Reconvicted

41. 1992/93 Release Sample: Young Offender Probation Cumulative Percentage of Cases Reconvicted

42. 1992/93 Release Sample: Adult Sentences to Incarceration People Reconvicted Within 5 Years of Release

43. 1992/93 Release Sample: Adult Probation People Reconvicted Within 5 Years of Release

44. 1992/93 Release Sample: Young Offender Custody People Reconvicted Within 5 Years of Release

45. 1992/93 Release Sample: Young Offender Probation People Reconvicted Within 5 Years of Release