Community-Oriented Defense Performance Indicators

1. Community-Oriented Defense Performance Indicators A Conceptual Overview Michael Rempel Center for Court Innovation Presented at the Community-Oriented Defense Network Conference, “Performance Measures - Making the Case for Community-Oriented Defense,” New York, NY, July 23, 2009

2. Project Goals • Goals: define the overall mission or purpose of the project; provide a “definition of success.” • Community-Oriented Defense Examples: • Rehabilitation: address defendant’s underlying problems • Collateral Consequences: mitigate adverse effects of conviction • Community Engagement: seek community input in programming

3. Project Objectives Objectives: support the goals and explain exactly how they will be accomplished. Objectives are SMART! Specific Pertain to a certain task or program Measurable Quantifiable Achievable Doable within constraints (e.g., financial or staffing resources) Results-Oriented Focused on short-term activities to attain longer-term goals Time-bound Include a date by which the objective must be completed

4. Performance Indicators Objectives translate into Performance Indicators. • Quantitative: • Number (#) • Percent (%) • Yes or no (y/n): Something happened or not • Feasible: • Relevant data can be captured/tracked • Appropriate control group can be identified (if necessary)

5. Process vs. Impact Indicators • Process Indicators: Did the intended program activities take place? • Failure of Implementation: program model not implemented as designed (few clients, intended services not delivered, best practices not followed, compliance not monitored, etc.) • The Bronx Juvenile Accountability Court • Impact Indicators: Did the program have the intended effects? • Failure of Design: program model implemented as designed, but model’s theory of change was flawed • Batterer programs for domestic violence offenders

6. Role of Control Groups • Process Indicators: Control group unnecessary (just measure whether program included the intended elements – volume, services, staffing, etc.) • Impact Indicators: Control group essential (cannot determine impact in absence of comparison) • Did program reduce recidivism if participants were arrested less often than before participation began? • Did program reduce recidivism if completers were arrested less often than dropouts? • Did program reduce recidivism if participants were arrested less often than a control group?

7. Sample Goal: Rehabilitation • Assessment: Determine each client’s individual needs • Percent (%) of all clients screened or assessed for problems • Referrals: Refer more clients to treatment/services: • Total # and % of clients referred for onsite or outside services • Breakdowns by service type: e.g., substance abuse treatment, mental health treatment, employment services, GED classes • Dosage: Increase treatment dosage that clients receive: • Show-up rate: Percent (%) of referrals at first appointment • Program completion rate (%) • Crime/Delinquency: Produce recidivism reduction: • Percent (%) re-arrested in one year: participants vs. controls

8. Drug Courts: Framework • Mission: Link drug-addicted defendants to court-supervised treatment as alternative to incarceration. • Content (typical): • Referral to community-based treatment • Ongoing judicial oversight (frequent judicial status hearings, drug testing, case management, sanctions, rewards) • Program duration at least one year • Major Goals (typical): • Recidivism reduction • Offender rehabilitation (primarily via reduced drug use) • Cost savings (to criminal justice system, crime victims, etc.)

9. Drug Courts: Key Process Indicators • Volume: Indicates “reach” (how many can benefit?) • # cases referred to drug court • # and % of referrals becoming program participants • Processing Speed: Indicates “immediacy” (are addicted defendants rapidly placed and engaged?) • Average # days from arrest to intake • Average # days from intake to first treatment placement • Retention: Indicates engagement (what % became invested in recovery?) and dosage (was it sufficient?) • One-year retention rate (%): percent of participants that graduated or were still active one year after beginning

10. Drug Courts: Key Impact Indicators • Recidivism Rate: Did the program ultimately reduce recidivism? • Re-arrest rates after 1, 2, or 3 years • Re-conviction rates after 1, 2, or 3 years • Drug Use: Did the program ultimately foster recovery? • Drug test results or self-reported use after 1, 2, or 3 years

11. Impacts #1: NYS Drug Courts on Recidivism Source: Rempel et al (2003).

12. Impacts #1: USA Drug Courts on Drug Use Source: Rossman and Rempel (2009).

13. Funding Considerations • General Rules: • Do anecdotes help? No • Must quantitative indicators be included? Yes • Common Types of Indicators: • Bean Counting: Actual vs. target volume • Fidelity Measures: % implemented of all proposed activities • Completion Rates: Actual vs. target completion rates • Recidivism Rates: Actual vs. target re-arrest rates • The Court Perspective: • Recidivism, recidivism, recidivism • Cost savings

14. Exercise: Goals to Indicators • Group Exercise: • Identify goals of Community-Oriented Defense Programs • State which objectives follow from the identified goals • Develop specific and quantifiable performance indicators • Tip: Begin each indicator with words like: “percent,” “number,” or “average” (i.e., make sure it is quantitative) • Reality Check: Do your identified goals, objectives, and indicators relate to your actual program activities? • Bonus Questions: What data do you need to obtain your indicators? How can you obtain it?