appraising risk of violence: is there a role for clinical judgement

1. 1 Appraising Risk of Violence: Is There a Role for Clinical Judgement? Marnie E. Rice, Ph.D., FRSC riceme@mcmaster.ca http://www.mhcp-research.com/present.htm Grand Rounds McMasterUniversity Dept. of Psychiatry & Behavioural Neurosciences March 24, 2004

2. 2 Appraising Risk of Violence: Is There a Role for Clinical Judgement? Historical overview of violence risk assessment Predictors of violence and of sexual recidivism Current approaches to violence risk assessment Combining predictors into risk assessment tools Evidence in favour of each approach What is a �dynamic risk variable�? Evidence? The role of clinical judgement in violence risk assessment

3. Components of Expertise Experts should make different judgements than laypersons Experts should make more accurate judgements than laypersons (the amount of accuracy being limited by the amount of agreement shown in their judgements) Experts should use specialized assessments or measurements in arriving at their judgements (Einhorn, 1974)

4. 4 History of Prediction of Violence Prediction of violence before mid-80�s Baxstrom (Steadman, 1973) Quinsey & Ambtman, 1979 Pasewark, Bieber, Bosten, Kiser, & Steadman, 1982 Monahan (1981)

5. 5 Predictors of Violent Recidivism Among Mentally Disordered Offenders Big predictors objective risk assessment, antisocial personality, violent history, juvenile delinquency, age* Medium predictors nonviolent criminal history, adult criminal history, substance abuse, marital status Small predictors clinical judgement, psychosis*, offense seriousness

6. 6 Nonpredictors of Violent Recidivism: Psychological distress Remorse Insight

7. 7 MacArthur Risk Assessment Study �More research demonstrating that the outcome of unstructured clinical assessments left a great deal to be desired seemed to be overkill: that horse was already dead.� (Monahan et al., 2001)

8. 8 Predictors of Violence Among Psychiatric Patients Big predictors: Psychopathy (PCL:SV); Adult arrests, Antisocial personality disorder, Major mental disorder without substance abuse*, drug or alcohol abuse, anger (Novaco) Moderate predictors: Violent arrests, schizophrenia*, child abuse, threat-control over-ride symptoms* delusions at the time of admission*

9. 9 MacArthur Risk Study Small predictors persecutory delusions*, male, BPRS hostility, BPRS thought disturbance, medication nonadherence on admission* Nonpredictors mania, depression, any delusions, hallucinations, command hallucinations, grandiose delusions, GAF, BPRS Total

10. 10 Summary of Predictors Large and medium predictors Major mental disorder* Antisocial personality, psychopathy, age* violent history, juvenile delinquency, adult criminal history substance abuse never married threat-control over-ride symptoms* delusions at the time of admission*

11. 11 Summary of Predictors Nonpredictors mania, depression, any delusions, hallucinations, command hallucinations, grandiose delusions, GAF, BPRS Total Psychological distress Remorse Insight Why clinical judgement may not have worked very well

12. 12 Combining Predictors into Risk Assessment Tools Clinical Structured Clinical Variables derived from empirical and clinical literature: e.g. HCR-20, SVR-20, SARA Actuarial Derived from actual followup; provides norms and numerical risk estimates e.g. VRAG, MacArthur ICT, SORAG, RRASOR, Static-99, ODARA Adjusted actuarial Adjust actuarial score up or down (e.g., VPS)

13. 13 Recent Example of Clinical Judgement Huss, Odeh, & Zeiss, 2004 20 psychologists, 20 psychiatrists, 20 social workers, and 20 nurses working in clinical psychiatric settings Clinicians reviewed admission evaluations and clinical notes from the first 24 hours of actual hospital stays for 2-12 different patients No relationship between risk cues clinicians used and actual violence whether presence/absence and severity Very low inter-rater reliability

14. 14 Structured Clinical e.g. HCR-20 Variables derived from empirical and clinical literature 20 items Score each 0, 1, or 2 according to manual Use as guide or aide-memoire combined with clinical judgement to categorize patient as low, medium, or high risk

15. 15 HCR-20 Items Previous violence Young age at first violent incident Relationship instability Employment problems Substance use problems Major mental illness Psychopathy Personality disorder Early maladjustment Lack of insight Negative attitudes Active symptoms of mental illness Impulsivity Unresponsive to treatment

16. 16 Performance of the HCR-20 Many studies of civil & forensic populations, correctional & mixed samples ROC areas range from .63-.80 Most research studies look at total score and treat it as a continuum Most evidence for validity of historical items One study showing that clinical judgement can do better than unadjusted score (ROC area of .74 vs. .70); one showing it does no better

17. 17 Summary of Structured Clinical Judgement When scored as a numerical scale it does much better than unaided clinical judgement Unclear how it does when clinicians are free to adjust score as manual suggests

18. 18 Combining Predictors Using an Actuarial Approach We decided to construct an actuarial tool to predict violent recidivism among mentally disordered offenders The Violence Risk Appraisal Guide or VRAG

19. 19 Development of the VRAG Construction Sample 618 �mentally disordered offenders� Candidate Predictor Variables Demographic Criminal Psychiatric Childhood

20. 20 Development of the VRAG 7 years average time at risk 31% committed a new violent offense Definition of violent offense Analyses Multiple regression Divided sample into halves Univariate analyses Weighting system

21. 21 Violence Risk Appraisal Guide Psychopathy Checklist Score Elementary school maladjustment Age at index offense* DSM III personality disorder Separation from parents before age 16 Failure on prior conditional release History of nonviolent offenses

22. 22 Violence Risk Appraisal Guide Never married DSM III schizophrenia* Victim injury in index offense* History of alcohol abuse Male victim in index offense

23. 23 VRAG- Psychometric PropertiesQuinsey, Harris, Rice, & Cormier (1998).Violent Offenders: Appraising and Managing Risk. Washington, D.C.: American Psychological Association. Range of scores: -26 to +38, often divided into 9 �bins� Mean score in construction sample= .91 (SD=12.9) IRR= .90 SEM = 4.1 (Means that 95% confidence interval is approx. +/- 8 or 1 �bin�)

24. Performance of the VRAG

25. 25

26. Receiver Operator Characteristic

27. 27 Length of followup and predictive accuracy What if we change the followup period? 3.5 years Baserate = 15% 7 years Baserate = 31% 10 years Baserate = 43% What if we predict time until violent failure?

28. Harris, Rice, & Cormier, LHB, 2002 28 Prospective Study of the VRAG in Predicting Violent Recidivism Among Forensic Patients 406 male forensic patients in Ontario in 1990 who had an opportunity to fail before Sept. �98 All scored on VRAG in 1990 Mean length of opportunity was 85.2 mos. for 172 with known length of opportunity ROC area for men= .75 Fixed 5 yr. risk period, ROC area = .80

29. 29 Prospective validation of the VRAG For men, ROC area= .75 For 133 cases with known 5 years followup, ROC area= .80 For 6 month followup, ROC area= .80 VRAG related to �serious� recidivism, homicide, speed of recidivism VRAG unrelated to recidivism for women Clinician�s ratings did not improve on VRAG alone

30. 30 Performance of the VRAG on Cross-validation

31. Illustrative ROCs

32. 32 The VRAG for Psychiatric Patients Applied VRAG to the MacArthur Violence Risk Assessment Study data set 1136 male and females admitted to acute psychiatric wards Had to approximate many of the variables ROC area= .72 Worked as well for women as for men Outcome was mostly self-reported violence, not criminal violence Followup was short- 20 weeks

33. Replications of VRAG/SORAG (n=26)

34. 34 Clinical decisions using only current actuarial instruments Considerable expertise Different (and more accurate) judgments than other �experts� (and laypersons) More reliable than laypersons Use special instruments- DSM diagnosis, PCL-R , VRAG, MacArthur ICT

35. 35 Can We Improve on the VRAG or other Risk Assessment Tools by Allowing Clinical Adjustment? Hilton & Simmons (LHB, 2001) No association between actuarial risk score and clinicians� opinions even when VRAG score was made available Recidivism related to VRAG score (r=.42), but not to clinician recommendation (r=.14)

36. 36 Another example Krauss, 2004 Federal judges in U.S. have discretion to over-ride federal sentencing guidelines based on an actuarial instrument for predicting recidivism�The Salient Factor Score They are given what the Federal Sentencing Guidelines recommend and then can over-ride so long as they justify reasons Gathered followup data after offenders were released

37. ROCs* After Krauss, 2004

38. 38 Summary No evidence that clinical judgement combined with actuarial can do a better job than actuarial alone

39. 39 Hanson & Morton-Bourgon (2004) meta-analysis: Sexual recidivism Clinical assessment .60 .61 Empirically guided .60 .61 With outlier .60 .64 Actuarial risk scale (sex) .67 .67

40. 40 Hanson & Morton-Bourgon meta-analysis: Violent recidivism Clinical assessment .66 With outlier .59 Empirically guided .64 With outlier .59 Actuarial risk scale (sex) .66 VRAG .71 VRAG (with outlier) .73 SORAG .70 SORAG (with outlier) .72

41. 41 Can We Improve Actuarial Instruments by Including Dynamic Predictors? VRAG and other actuarial instruments includes only static variables Scores don�t change with time or treatment Could they be improved by including measures of change over time or change due to treatment?

42. 42 What Is a Dynamic Predictor?

43. Prospects for Dynamic Prediction of Who Is Likely to Be Violent Performance of an instrument (MASORR) incorporating possible dynamic predictors ROC area Serious Sexual Pre-Treatment .58 .61 Post- Treatment .54 .61 Barbaree, Seto, Langton & Peacock, CJB (2001)

44. 44 Prospects for Dynamic Prediction Quinsey, Coleman, Jones, & Altrose, 1997 �Dynamic antisociality� Complains about staff ; Shows no remorse for crime; Takes no responsibility for own behavior; Ignores or passes over previous violent acts; Has more antisocial attitudes and values; Shows no empathy or concern for others; Has unrealistic discharge plans; Psychiatric symptoms are not in remission; Has made threats aimed at specific victims Quinsey & Jones, in preparation Dynamic antisociality variables predict WHEN a high-risk offender is likely to be violent

45. Challenges for Dynamic Prediction of Who is Likely to be Violent Static predictors alone yield very high effect sizes (ROC areas up to .85) SO...Not much room for improvement on static predictors given noise in outcome measure Dynamic predictors can likely make only a very modest contribution to the prediction of who is likely to reoffend Main hope for dynamic prediction may be in predicting when recidivism is likely to occur

46. 46 Summary Still no evidence for expertise in clinician�s unaided clinical judgements Structured clinical judgement tools scored as numerical scales do much better than unaided clinical judgement Actuarial instruments tend to yield even higher effect sizes No evidence that clinical over-rides can improve on actuarial risk estimates and some evidence they make them worse

47. 47 Conclusions Best place for clinical judgement is inside an actuarial instrument