Deceptive Speech

1. Deceptive Speech Frank Enos • April 19, 2006

2. Defining Deception • Deliberate choice to mislead a target without prior notification (Ekman‘’01) • Often to gain some advantage • Excludes: • Self-deception • Theater, etc. • Falsehoods due to ignorance/error • Pathological behaviors

3. Why study deception? • Law enforcement / Jurisprudence • Intelligence / Military / Security • Business • Politics • Mental health practitioners • Social situations • Is it ever good to lie?

4. Why study deception? • What makes speech “believable”? • Recognizing deception means recognizing intention. • How do people spot a liar? • How does this relate to other subjective phenomena in speech? E.g. emotion, charisma

5. Problems in studying deception? • Most people are terrible at detecting deception — ~50% accuracy (Ekman & O’sullivan 1991, Aamodt 2006, etc.) • People use subjective judgments — emotion, etc. • Recognizing emotion is hard

6. People Are Terrible At This

7. Problems in studying deception? • Hard to get good data • Real world (example) • Laboratory • Ethical issues • Privacy • Subject rights • Claims of success • But also ethical imperatives: • Need for reliable methods • Debunking faulty methods • False confessions

8. 20th Century Lie Detection • Polygraph • http://antipolygraph.org • The Polygraph and Lie Detection (N.A.P. 2003) • Voice Stress Analysis • Microtremors 8-12Hz • Universal Lie response • http://www.love-detector.com/ • http://news-info.wustl.edu/news/page/normal/669.html • Reid • Behavioral Analysis Interview • Interrogation

9. Frank Tells Some Lies An Example…

10. Frank Tells Some Lies Maria: I’m buying tickets to Händel’s Messiah for me and my friends — would you like to join us? Frank: When is it? Maria: December 19th. Frank: Uh… the 19th… Maria: My two friends from school are coming, and Robin… Frank: I’d love to!

11. How to Lie (Ekman‘’01) • Concealment • Falsification • Misdirecting • Telling the truth falsely • Half-concealment • Incorrect inference dodge.

12. • Concealment • • Falsification • • Misdirecting • • Telling the truth falsely • • Half-concealment • • Incorrect inference dodge. Frank Tells Some Lies Maria: I’m buying tickets to Handel’s Messiah for me and my friends — would you like to join us? Frank: When is it? Maria: December 19th. Frank: Uh… the 19th… Maria: My two friends from school are coming, and Robin… Frank: I’d love to!

13. Reasons To Lie (Frank‘’92 ) • Self-preservation • Self-presentation • *Gain • Altruistic (social) lies

14. How Not To Lie (Ekman‘’01) • Leakage • Part of the truth comes out • Liar shows inconsistent emotion • Liar says something inconsistent with the lie • Deception clues • Indications that the speaker is deceiving • Again, can be emotion • Inconsistent story

15. How Not To Lie (Ekman‘’01) • Bad lines • Lying well is hard • Fabrication means keeping story straight • Concealment means remembering what is omitted • All this creates cognitive load  harder to hide emotion • Detection apprehension (fear) • Target is hard to fool • Target is suspicious • Stakes are high • Serious rewards and/or punishments are at stake • Punishment for being caught is great

16. How Not To Lie (Ekman‘’01) • Deception guilt • Stakes for the target are high • Deceit is unauthorized • Liar is not practiced at lying • Liar and target are acquainted • Target can’t be faulted as mean or gullible • Deception is unexpected by target • Duping delight • Target poses particular challenge • Lie is a particular challenge • Others can appreciate liar’s performance

17. Features of Deception • Cognitive • Coherence, fluency • Interpersonal • Discourse features: DA, turn-taking, etc. • Emotion

18. Describing Emotion • Primary emotions • Acceptance, anger, anticipation, disgust, joy, fear, sadness, surprise • One approach: continuous dim. model (Cowie/Lang) • Activation – evaluation space • Add control/agency • Primary E’s differ on at least 2 dimensions of this scale (Pereira)

19. Problems With Emotion and Deception • Relevant emotions may not differ much on these scales • Othello error • People are afraid of the police • People are angry when wrongly accused • People think pizza is funny • Brokow hazard • Failure to account for individual differences

20. Bulk of extant deception research… • Not focused on verifying 20th century techniques • Done by psychologists • Considers primarily facial and physical cues • “Speech is hard” • Little focus on automatic detection of deception

21. Modeling Deception in Speech • Lexical • Prosodic/Acoustic • Discourse

22. Deception in Speech (Depaulo ’03) • Positive Correlates • Interrupted/repeated words • References to “external” events • Verbal/vocal uncertainty • Vocal tension • F0

23. Deception in Speech (Depaulo ’03) • Negative Correlates • Subject stays on topic • Admitted uncertainties • Verbal/vocal immediacy • Admitted lack of memory • Spontaneous corrections

24. Problems, revisited • Differences due to: • Gender • Social Status • Language • Culture • Personality

25. Columbia/SRI/Colorado Corpus • With Julia Hirschberg, Stefan Benus, and colleagues from SRI/ICSI and U. C. Boulder • Goals • Examine feasibility of automatic deception detection using speech • Discover or verify acoustic/prosodic, lexical, and discourse correlates of deception • Model a “non-guilt” scenario • Create a “clean” corpus

26. Columbia/SRI/Colorado Corpus • Inflated-performance scenario • Motivation: financial gain and self-presentation • 32 Subjects: 16 women, 16 men • Native speakers of Standard American English • Subjects told study seeks to identify people who match profile based on “25 Top Entrepreneurs”

27. Columbia/SRI/Colorado Corpus • Subjects take test in six categories: • Interactive, music, survival, food, NYC geography, civics • Questions manipulated  • 2 too high; 2 too low; 2 match • Subjects told study also seeks people who can convince interviewer they match profile • Self-presentation + reward • Subjects undergo recorded interview in booth • Indicate veracity of factual content of each utterance using pedals

28. CSC Corpus: Data • 15.2 hrs. of interviews; 7 hrs subject speech • Lexically transcribed & automatically aligned  lexical/discourse features • Lie conditions: Global Lie / Local Lie • Segmentations (LT/LL): slash units (5709/3782), phrases (11,612/7108), turns (2230/1573) • Acoustic features (± recognizer output)

30. Columbia University– SRI/ICSI – University of Colorado Deception Corpus: An Example Segment SEGMENT TYPE Breath Group LABEL LIE Obtained from subject pedal presses. um i was visiting a friend in venezuela and we went camping ACOUSTIC FEATURES max_corrected_pitch5.7 mean_corrected_pitch5.3 pitch_change_1st_word -6.7 pitch_change_last_word-11.5 normalized_mean_energy0.2 unintelligible_words 0.0 Produced automatically using lexical transcription. Produced using ASR output and other acoustic analyses LEXICAL FEATURES has_filled_pauseYES positive_emotion_wordYES uses_past_tense NO negative_emotion_wordNO contains_pronoun_iYES verbs_in_gerund YES PREDICTION LIE

31. CSC Corpus: Results • Classification (Ripper rule induction, randomized 5-fold cv) • Slash Units / Local Lies — Baseline 60.2% • Lexical & acoustic: 62.8 %; + subject dependent: 66.4% • Phrases / Local Lies — Baseline 59.9% • Lexical & acoustic 61.1%; + subject dependent: 67.1% • Other findings • Positive emotion words deception (LIWC) • Pleasantness  deception (DAL) • Filled pauses  truth • Some pitch correlation — varies with subject

32. Example JRIP rules: (cueLieToCueTruths >= 2) and (TOPIC = topic_newyork) and (numSUwithFPtoNumSU <= 0) and (wu_ENERGY_NO_UV_STY_MAX__EG_ZNORM-ENERGY_NO_UV_STY_MIN__EG_ZNORM-D <= 5.846) => PEDAL=L (231.0/61.0) (cueLieToCueTruths >= 2) and (numSUwithFPtoNumSU <= 1) and (wu_ENERGY_NO_UV_STY_MAX__EG_ZNORM-ENERGY_NO_UV_STY_MIN__EG_ZNORM-D <= 5.68314) and (wu_ENERGY_NO_UV_RAW_MAX-ENERGY_NO_UV_RAW_MIN-D >= 8.41605) and (wu_F0_SLOPES_NOHD__LAST >= -2.004) => PEDAL=L (284.0/117.0) (cueLieToCueTruths >= 2) and (wu_F0_RAW_MAX >= 5.706379) and (wu_DUR_PHONE_SPNN_AV <= 1.0661) => PEDAL=L (262.0/115.0)

33. CSC Corpus: A Perception Study • With Julia Hirschberg, Stefan Benus, Robin Cautin and colleagues from SRI/ICSI • 32 Judges • Each judge rated 2 interviews • Judge Labels: • Local Lie using Praat • Global Lie on paper • Takes pre- and post-test questionnaires • Personality Inventory • Judge receives ‘training’ on one subject.

34. By Judge 58.2% Acc. By Interviewee

35. Personality Measure: NEO-FFI • Costa & McCrae (1992) Five-factor model • Openness to Experience • Conscientiousness • Extraversion • Agreeability • Neuroticism • Widely used in psychology literature

36. Neuroticism, Openness & Agreeableness correlate with judge performance WRT Global lies.

37. These factors also provide strongly predictive models for accuracy at global lies.

38. Other Perception Findings • No effect for training • Judges’ post-test confidence did not correlate with pre-test confidence • Judges who claimed experience had significantly higher pre-test confidence • But not higher accuracy! • Many subjects used disfluencies as cues to D. • In this corpus, disfluencies correlate with TRUTH! (Benus et al. ‘06)

39. Our Future Work • Individual differences • Wizards of deception • Predicting Global Lies • Local lies as ‘hotspots’ • New paradigm • Shorter • Addition of personality test for speakers • Addition of cognitive load