Tobi Saulnier - Cognitive Bias Training Game Valuable for Everything from Law Enforcement to Teen

1. Cognitive Bias Training for Everyone! Tobi Saulnier, CEO 1st Playable Productions

2. What is CYCLES? What is CYCLES? • Part of IARPA program, designed to train Intelligence Analysts • Serious game that teaches bias recognition, discrimination, mitigation • Two games – Phase 1: Fundamental Attribution Error, Confirmation Bias, and Bias Blind Spot – Phase 2: Anchoring, Projection, and Representativeness •Proven to train effectively

3. Game Video Links • Trailer Game 1 https://www.youtube.com/watch?v=5QqY4KmX1S4&feature=youtu.be • Trailer Game 2 • https://www.youtube.com/watch?v=-St0x9a5uq4&feature=youtu.be 3

4. CYCLES: Basic Lesson We (our brains) are and always will be biased BUT There are simple thinking tools that can help Always remember your thinking tools CYCLES games let players experience this lesson 4

5. What are Cognitive Biases? Confirmation Bias – People search for or interpret information in a way that confirms their preconceptions. Often preceded by priming. Fundamental Attribution Error – People over-emphasize personality-based explanations for behaviors observed in others while under-emphasizing the role and power of situational influences on the same behavior. Bias Blind Spot – Individuals are unaware of their own cognitive biases, even when they can recognize cognitive biases in others. • • •

6. CYCLES Game CYCLES Game - - Phase 1 Phase 1 • Design: 30 minute, casual 3rd person puzzle-style game • Series of “challenge rooms”, “review rooms” Confirmation Bias Fundamental Attribution Error Bias Blind Spot

7. Phase 1 Variables Repetition and Duration Character Customization RIch/Minimalist Narrative/Art Do details in narrative or art impact efficacy? Relative impact of longer game experiences Player investment in custom characters versus no choice 7

8. Implementation: Challenge Types Challenge Hub Review • Learn and observe bias • Core bias training • Examples of bias • Walk-through complex concepts • Review knowledge • Real world examples • Reiterate lessons Real world text-based quizzes at end of levels 8

9. Repetition vs Length Bias Reduction Immediately after and 8-weeks after playing Immediate Immediate Immediate Immediate 8-weeks 8-weeks 8-weeks 8-weeks Video avg. post 11% Exp 3 Repetition Exp 4 Length All conditions have sig. pre to post change; sig. different than video

10. Bias Knowledge More than Doubled Best game: Repeated Play Exp 3a Knowledge Improvement Video avg. post 99% Immediate Immediate 8-weeks 8-weeks Video avg. 8-week 33% Pre to Post N = 335 8-week N = 198 All conditions statistically significant (p<.05) difference with pre-test

11. CYCLES Game - Phase 2 • ~60 minute, 1st person puzzle adventure game • Premise: Mr. Q will train you to survive decision-making challenges • A survival experience set in a remote carnival Anchoring Bias Projection Bias Representativeness Bias 11

12. Phase 2 Variables SATs Rewards Interactivity Practice with Structured Self-Critique- like system vs. without Sounds, visuals, prizes, bronze-silver-gold ratings accompanying feedback vs. feedback alone Choices in content and instruction vs. straight- through play 12

13. Implementation: Challenge Types Tents Shops Booths • Learn and observe bias • Core bias training • Examples of bias • Walk-through complex concepts • Practice decision- making • Based on carnival games • Point-and-click puzzles • Apply decision- making • Familiar settings (shopping) • Point-and-click puzzles Real world text-based quizzes at end of levels 13

14. CYCLES Design Learnings • Narrative Alignment – Align player role with concepts – Trustworthiness of feedback, player coaching paradigm • Bias Elicitation – Players may believe they are not biased • Simplify Interactions – Phase 1 is a more guided experience than Phase 1 – Simplified language, mitigations • Infographics for Complex Material 14

15. Design Challenges • Emphasize the learner is not at fault • Challenge difficulty “sweet spot” • Text density balance • Dynamic sensitivity to learning progress • Value of failure and replay • “Slow down” mechanics • Humor is important! • Practice needs to be built in • Literacy supports 15

16. Challenges for General Adaptation • Content easily updated to be applicable for target audience – Quiz questions • Target platform – Flexible code base • People could feel like there was something wrong with them when introduced to the topic. – Solution: Optical illusion ● Cognitive biases are difficult to demonstrate on an individual scale 16

17. New Audiences ● Law Enforcement ● Teens and Empathy ● Business 17

18. Design challenges for LEO ● Targeting ○ Law Enforcement Officers ○ First Responders ● Feedback ○ Narrative Context - Superhero training Target audience think of themselves as superheroes ○ Order - •Phase 2 game (superhero training classes) •Phase 1 game (superhero mission) bonus, this training ends on Bias Blind Spot ● Tradeoff between muscle memory training and the “slow down” message 18

19. Design challenges for empathy Cognitive Bias and thought processes affecting empathy are similar Biases we proposed teaching: • Projection Bias • Fundamental Attribution Error • Confirmation Bias • Bias Blind Spot Some existing CYCLES content was less applicable for CYCLES of Empathy Explaining to students why empathy was important to learn 19

20. Challenges for Teen audience Complex language and concepts (“Representativeness”), amount of reading Younger audience have more limited attention span than a college or professional audience Player motivation - original audience was more intrinsically motivated 20

21. Key Adaptations • Went from a 9th grade reading level to 5th / 6th on Flesch Kincaid. • Real-world quiz questions were made more audience centric. • Used the more direct mechanics from CYCLES Phase 2 • Narrative: “School for not being a jerk” 21

22. Example from Game 22

23. Bias -> Empathy Representativeness Use to get buy in on bias (with simpler problems like coin flips and probability) Anchoring Bias (Feelings sub-bias) to talk about how our strong emotional association with an event can prevent us from seeing the whole event clearly Projection Bias (renamed Similarity) to describe how others like different things than they do Bias Blind Spot Used learnings from our other game to inform the design for adding Bias Blind Spot to Ph 2 (New) Used existing teaching structure and functionality for new content Feelings Bias 23

24. Design challenges for Business ● Forecasting ● Hiring ● Artifical Intelligence ● Risk Assessment ● Communication 24

25. Change Management Q x A = E Even though some design variables were established as not affecting teaching effectiveness, they do affect marketing/perception Example: Minimalistic art in Phase 1 game 25

26. Making it Actionable 26

27. Dr. Tobi Saulnier, tobi@1stplayable.com