Exploring Cognitive Biases 17 October 2007

1. Exploring Cognitive Biases17 October 2007 Team NSA Mark Blair Dan McNulty Brittany Monteparte Andreea Neagu Adrienne Sluga

2. Agenda • Key Findings • Definitions • Anchoring Effect • Availability Heuristic • Conjunction Fallacy • Video • Confirmation Bias • Groupthink • Application

3. Key Findings • Cognitive Biases Are Not An Indictment Of An Individual’s Analytical Abilities • Two Types Of Analysts • Those Who Are Aware Of Their Own Biases • Those Who Are Not Aware Of Their Own Biases • Memory Is Not Localized

4. Cognitive Biases • “Cognitive biases are mental errors caused by our simplified information processing strategies.” • Psychology of Intelligence Analysisby Richard J. Heuer • “Strategic decision making is particularly vulnerable to the influence of cognitive biases, misguided thought patterns that result from the simplifying strategies or decision-making shortcuts known as judgment heuristics. These common errors in judgment can lead to disastrous consequences and are often the result of either insufficient information or not enough of the right kind of information.” • Cognitive Bias in Everyday Strategic Planning by Loren Gary

5. Source: http://www.eyetricks.com/illusions.htm Cognitive Biases • Fundamental Limitations In Human Mental Processes • Simple Rules Of Thumb Leading To Faulty Judgments • Similar To Optical Illusions • Physical Images That Appear Misleading • The Brain Carries Weaknesses • Awareness Of The Bias Does Not Produce A More Accurate Perception Source: http://www.au.af.mil/au/awc/awcgate/psych-intel/art12.html

6. Biases • “Any systematic deviation from the truth that affects the conclusions you make based on your data.” • The National EMSC Date Analysis Resource Center • “When a point of view prevents impartial judgment on issues relating to the subject of that point of view.” • MedicineNet, Inc • “Bias, or systematic error, favors particular results.” • Annenburg Foundation

7. Fallacy • “A fallacy is a general type of appeal (or category of argument) that resembles good reasoning, but that we should not find to be persuasive.” • http://www.cuyamaca.edu/brucethompson/Fallacies/discussion.asp • “Fallacies are mistakes of reasoning, as opposed to making mistakes that are of a factual nature.” • http://philosophy.hku.hk/think/

8. Heuristics • “A heuristic is a mental shortcut that allows people to solve problems and make judgments quickly and efficiently. The rule-of-thumb strategies shorten decision-making time and allow people to function without constantly stopping to think about the next course of action. While heuristics are helpful in many situations, they can also lead to biases.” • Psychologist, Kendra Van Wagner

9. Anchoring Effect

10. Anchoring Effect • Also Known As Focalism • Tendency To Rely Too Heavily On One Trait Or Piece Of Information • Bias Toward A Certain Value • Emphasis Based On When The Information Is Received Rather Than On The Strength Of The Information Source: http://changingminds.org/explanations/theories/anchoring_adjustment.htm

11. Question #1 • Is the Mississippi River longer or shorter than 500 miles? • Give estimate of how long the Mississippi River actually is. • Is the Mississippi River longer or shorter than 5000 miles? • Give estimate of how long the Mississippi River actually is.

12. Relevance To Intelligence • One Of The Most Dangerous Biases For Intelligence Professionals • Analyst Must Be Aware Of Possible Shortcuts • Assessment Of Variables In The Decision Source: http://www.au.af.mil/au/awc/awcgate/psych-intel/art15.html

13. How To Prevent It? • Three Methods: • Decision Support Tools • Training • Devil’s Advocate • Having one individual search for refuting evidence is likely to lead to better analysis • Rethink Problem From Beginning • Avoid Any Prior Judgment As A Starting Point Source: http://findarticles.com/p/articles/mi_m0IBS/is_1_28/ai_82351480/pg_3

14. Availability Heuristic

15. Availability Heuristic • Tendency To Overestimate The Probabilities Of Events Associated With Memorable Occurrences • Base Decisions On What Is Remembered Rather Than Complete Data • Information Mentally Available • Media And News Reports Have A Significant Effect • Demonstration Source: http://changingminds.org/explanations/theories/availability_heuristic.htm

16. Questions #2, #3, & #4 • Are there more words that begin with ‘R’ or are there more words that have ‘R’ in the third place in the English language? • Are there more words that begin with ‘K’ or are there more words that have ‘K’ in the third place in the English language? • Which is a more likely cause of death in the United States: being killed by falling airplane parts or being killed by a shark? • Do more Americans die from a) homicide and car accidents, or b) diabetes and stomach cancer? • Which claims more lives in the United States: lightning or tornadoes?

17. Relevance To Intelligence • The Soviet Break Up • Difficulty In Imagining The Break-up Due To Lack Of Experience • Not As Difficult To Imagine A Return To A Communist Regime In Russia Because Of Familiarity Source: http://www.au.af.mil/au/awc/awcgate/psych-intel/art15.html

18. How To Prevent It? • Be Aware Of All The Diverse Factors That Influence A Decision • Considering What Information Is Truly Relevant Rather Than Simply Available • ACH

19. Conjunction Fallacy

20. Conjunction Fallacy • Tendency To Assume That Two Or More Specific Conditions Are More Probable Than A Single General One • Associating Quantity Of Events With Quantity Of Probability Source: http://changingminds.org/explanations/theories/conjunction_fallacy.htm

21. Question #5 • Linda is 31 years old, single, outspoken and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. • Is she more likely to be a bank teller or a bank teller active in the feminist movement?

22. Video Kahneman & Tversky

23. Confirmation Bias

24. Confirmation Bias • Tendency To Search For Information That Confirms One’s Fixed Ideas • When Building A Hypothesis: • More Likely To Use Information That Supports That Decision • Avoid Things That Are Contradictory To The Hypothesis Source: http://changingminds.org/explanations/theories/confirmation_bias.htm

26. Confirmation Bias • Test The Following Hypothesis: • All Cards With Vowels Have An Even Number On The Back • Select Two Cards To Flip Over:

27. Disconfirming Data • Confirming Data • This Choice Provides No Indication • Of Whether The Hypothesis Is True • This Choice Provides No Indication Of • Whether The Hypothesis Is True Confirmation Bias • Correct Answers: • A: Confirming Data • 7: Disconfirming Data

28. Relevance To Intelligence • Major Chris Tatarka – US Military Academy • Enemy Course Of Action (ECOA) • Tendency To Look For Information Or Reports That Support It • Likely To Deploy Assets Only Along The Most Expected Enemy Axis Of Advance • Discount Reports That Do Not Confirm The ECOA • General Ulysses S. Grant • The Battle Of Shiloh (1862) • Ignored Information And Reports Source: http://findarticles.com/p/articles/mi_m0IBS/is_1_28/ai_82351480/pg_2

29. How To Prevent It? Three Methods: • Decision-Support Tools • Analysis Of Competing Hypotheses (ACH) • Training • Devil’s Advocate Source: http://www.fas.org/irp/eprint/pirolli.pdf http://findarticles.com/p/articles/mi_m0IBS/is_1_28/ai_82351480/pg_2

30. Groupthink

31. Groupthink • “A mode of thinking that people engage in when they are deeply involved in a cohesive in-group, when the members’ striving for unanimity override their motivation to realistically appraise alternative courses of action.” • Irving L. Janus Victims of Groupthink Source: http://tataworld.com/pdf/Biases%20and%20Prejudices%20in%20Intelligence%20AnalysisJohn%20Nolan.pdf

33. Groupthink • Characteristics Of Groupthink • Idea Of Invulnerability • Collective Efforts To Rationalize Or Discount Warnings • Unquestioned Belief In The Group’s Moral Correctness • Stereotypes Of Those Not In the Group • Self-Censorship • Pressures To Conform • Illusions Of Unanimity • Protect The Group From Contrasting Viewpoints Source: http://changingminds.org/explanations/theories/confirmation_bias.htm

34. Relevance To Intelligence • Senate Intelligence Committee Report - 2004 • Intelligence Community And Iraq’s Weapons Of Mass Destruction Program • John F. Kennedy And Bay Of Pigs Invasion • Group Cohesion • Mind Guarding Source: http://www.msnbc.msn.com/id/5403731/ http://www.globalsecurity.org/intell/ops/bay-of-pigs.htm

35. How To Prevent It? • Select A Devil’s Advocate • Divide The Group Into Sub-Groups • Discuss What Is Happening With Others Not In The Group • Gather Anonymous Feedback Source: http://www.valuebasedmanagement.net/methods_janis_groupthink.html

36. Application

37. Bias And Structured Analysis • Most Dangerous In The Beginning (Organizing) Stages Of Analysis • Confirmation Bias • Anchoring Effect • Availability Heuristic • Conjunction Fallacy • Gambler’s Fallacy • Overweighing Historical Data To Reduce The Probability Of Repeated Experiences

38. Combating Biases With The Structured Approach • Focus On Context • Focus On Most Indicative Information • Ignores Extraneous Information • Identifies Relationships Of Data • The “Structure” Is Non-Assumptive And Not Susceptible To Bias

39. Female Astronauts When Harry Met Sally Ride Movies/Actors Field Famous Sallys Struthers Hemings Information Dominance And The Availability Heuristic • Modern Research: Memory Is Not Localized • Information Recall Is Likely The Result Of The • Activation Of Neural Networks teacher Disaster Query: Who was the teacher who died in the Challenger Disaster? Challenger Cue: Sally... Sally

40. Who Was On The Challenger? • The Answer Is: Christa McAuliffe • Sally Ride Gained Notoriety As: • The First US Female Astronaut • Assigned To The Rogers Commission To Investigate The Challenger Disaster • Sally Ride Is Still Alive Today

41. A Google Search Using: Sally+Challenger+Disaster+Teacher Returns: • 9 Mentions Of The Name “Sally Ride” • Most Near The Top Of The List • All Containing The Full Name “Sally Ride” (“Ride” Was Not A Search Parameter) • 4 Mentions Of The Name “Christa McAuliffe” 1 Mention Of “McAuliffe” • Most Near The Bottom Of The List However, The Search Engine Worked Perfectly. It Did Exactly What It Is Designed To Do!

42. Data Retrieval • Excluding The Term “Sally” Only Returns Two Mentions Of “Christa McAuliffe” • The Inclusion Of “Sally” Actually Returned More Mentions Of “Christa McAuliffe” • “Sally” Is A Powerful Informational Cue And Demonstrates An Informational Dominance

43. Data Retrieval • Structured Analysis Combats This Bias By Organizing And • Weighing Data In The Exclusive Context Of The Intelligence • Query Specific Contextual Data Required Parameter Mismatched cue and data! Heavily Weighted Info Astronaut Teachers “Teachers in Space” Anecdotal Data Barbara Morgan Pam Leestma No Indication Dick Scobee Arthur Walker, Jr Richard Feynman Chuck Yeager Michael J. Smith Eugene Covert Christa McAuliffe Alton G. Keel, Jr Challenger Disaster (Rogers) Commission Astronauts Killed in Challenge Disaster Robert Hotz Judith Resnik Neil Armstrong Sally Ride David Acheson Ellison Onizuka Robert Rummel Ronald McNair Albert Wheelon Gregory Jarvis Joseph Sutter Donald Kutyna William Rogers

44. Blink And Info Weighting • Malcolm Gladwell’s Blink • Cited Cook County Hospital’s Dilemma Of Predicting Heart Attacks Over Next 72 Hours • Lee Goldman’s 4-Factor Algorithm More Accurate Than Seasoned Cardiologists • Conclusion • Cardiologists Considered Information Not Relevant To The Context Of The Query • A Layman (Or A Computer For That Matter), With An Hour Of Training, Could Learn To Be More Accurate At Predicting Heart Attacks In Next 72 Hours Than A Seasoned Cardiologist

45. Context And The Query • Structured Approaches Focus Specifically On Indicative Contextually Exclusive Data • Context Is Paramount Required Parameter Heavily Weighted Info Anecdotal Data No Indication Specific Contextual Data (Goldman’s Algorithm) Non-Contextual Data Diet Cholesterol Medical History Fluid In Lungs ECG Evidence Risk Factors For Heart Attack Over Lifetime Indicators Of Heart Attack In Next 72 Hours Diabetes Unstable Angina Systolic Blood Pressure > 100mm Hg Exercise Habits Family History Of Heart Attacks

46. Thank You For Your Attention • Are There Any Questions? • Contact Information: • Mark Blair: mblair81@mercyhurst.edu • Daniel McNulty: dmcnul91@mercyhurst.edu • Brittany Monteparte: bmonte80@mercyhurst.edu • Andreea Neagu: aneagu16@mercyhurst.edu • Adrienne Sluga: asluga69@mercyhurst.edu