New Findings on Jurors’ Evaluations of Forensic Science

1. New Findings on Jurors’ Evaluations of Forensic Science William C. Thompson Department of Criminology, Law & Society University of California, Irvine Dayton, August 19, 2007

2. Overview • Jury simulation studies • Undergraduates • Real jurors • Evaluating performance • Sensitivity to important variables • Susceptibility to fallacious reasoning • Consistency with principles of logic, such as Bayes’ theorem

3. Evaluating negative findings • When is absence of evidence treated as evidence of absence? According to Locard, “every contact leaves a trace.” • When is failure to find a trace treated as evidence that no contact occurred?

4. The normative question Q: Doctor Lee, when should absence of evidence be treated as evidence of absence? A: (Henry Lee): Depends. You must ask a forensic scientist. We tell you.

5. The Bayesian Answer • Let E signify “no trace was found” • Let C indicate “contact occurred” and NC that “no contact occurred” • E provides evidence of NC to the extent p(E|NC) > p(E|C) • If p(E|NC)=1.00, thenE supports NC unless p(E|C) is also 1.00

6. Translation • If there is no chance of finding a trace in the absence of contact, then • Failure to find a trace always supports an inference of no contact • unless there is no chance a trace would be found in any case (and then the evidence is worthless) • Absence of evidence is evidence of absence to the extent absent evidence is more likely given absence than presence (which is almost always the case).

7. Example • No GSR was found on defendant’s hand • Is that evidence he wasn’t the shooter? • Yes, to the extent GSR was likely to be found if he was the shooter. • If the probability of finding GSR on a shooter is: • 100%--absence of GSR is definitive proof he didn’t shoot • 0%--absence of evidence proves nothing • 50%--absence of evidence should double our estimate of the odds he wasn’t the shooter (or decrease by 50% our estimate of the odds he was the shooter)

8. Jury Experiments

9. Confusion with Deductive Logic Q: Does the failure to find GSR on the defendant’s hand prove he didn’t fire a gun? A: No. A negative test doesn’t prove anything one way or the other. There are too many possible explanations for a negative test. Because you cannot rule out all the other explanations, you cannot say for sure that the person didn’t fire a gun. It is a basic principle of science. In order to prove a theory you have to rule out all the alternative theories.

10. Evaluating a Forensic Match • Probability of Coincidental Match—RMP • Probability of False Positive—FPP

11. Normative Models How should people evaluate information on the RMP and FPP? Prior Odds x Likelihood Ratio = Posterior Odds p(match|same source) LR = ------------------------------ p(match|different source) 1 LR = ------------------------------ RMP + [FPP(1-RMP)] See, Thompson, Taroni & Aitken, J.Forensic Sci. 2003

12. Effect of a Reported DNA “Match” on the Odds That an Individual Is the Source of the Matching Sample From Thompson, Taroni & Aiken (2003)

13. Effect of a Reported DNA “Match” on the Odds That an Individual Is the Source of the Matching Sample From Thompson, Taroni & Aiken (2003)

14. The False Positive Fallacy “If the probability of a false positive is one in a thousand that means there are 999 chances in 1000 we have the right guy.”

15. Previous Research on Jurors’ Evaluations of Forensic ID Evidence • People are “conservative”—give less weight to evidence of a forensic match than Bayes’ theorem says they should. • But… • Examined cases with relatively high prior odds of guilt • Possible scaling problems with 0-100% probability of guilt scales

16. Experiment 1

17. Percentage Voting Guilty—Undergraduates (N=360)

18. Check One Chances that Defendant is Guilty Certain to be Guilty About 999,999,999,999 chances in 1 trillion that he is guilty About 999,999,999 chances in 1 billion that he is guilty About 999,999 chances in 1 million that he is guilty About 99,999 chances in 100,000 that he is guilty About 9,999 chances in 10,000 that he is guilty About 999 chances in 1,000 that he is guilty About 99 chances in 100 that he is guilty About 9 chances in 10 that he is guilty One chance in 2 (fifty-fifty chance) that he is guilty About 1 chance in 10 that he is guilty About 1 chance in 100 that he is guilty About 1 chance in 1,000 that he is guilty About 1 chance in 10,000 that he is guilty About 1 chance in 100,000 that he is guilty About 1 chance in 1 million that he is guilty About 1 chance in 1 billion that he is guilty About 1 chance in 1 trillion that he is guilty Impossible that he is guilty 21 10 1

19. Estimated probability of defendant’s guilt False Positive Probability

20. Intuitive vs. Bayesian posteriors for LR=100 Bayesian Posterior False Positive Probability

21. Intuitive vs. Bayesian posteriors for LR=10,000 Bayesian Posterior False Positive Probability

22. Experiment 2: Actual Jurors

23. Percentage Voting Guilty– Actual Jurors (N=241)

24. Estimated probability of defendant’s guilt

25. Intuitive vs. Bayesian Posteriors for LR=100 Bayesian Posterior

26. Intuitive vs. Bayesian Posteriors for LR=10,000 Bayesian Posterior

27. Conclusions • Only a minority made judgments exactly consistent with fallacious reasoning • College students were sensitive to FPP and made judgments consistent with Bayesian norms • Actual jurors were insensitive to FPP and gave too much weight to the DNA evidence when FPP was high (1 in 100) • Well educated and mathematically-trained jurors look like college students

28. Legal Implications • When FPP is high, look for well-educated jurors, or • Consider excluding the evidence on grounds it will be over-valued by jurors

29. Acknowledgement Supported by NSF Grant SES 0617672 to William Thompson