Jennifer M. C. Vendemia Michael Jay Schillaci Department of Psychology

1. Neuroscientific Modeling of Deception with HD-ERP and fMRI Data:Experimental and Computational Problems Jennifer M. C. Vendemia Michael Jay Schillaci Department of Psychology University of South Carolina

2. Outline • Theoretical Framework • An Overview of Deception Research • A Cognitive-Physio Model of Deception • Experimental Procedure • Studies Performed & Standard Analysis • Main Results • Data • Collection & Problems • Analysis • Software • Proprietary & Research • In-House • Continuing Research • Current Efforts • Wish List

3. Theoretical Framework • Detection of Deception is a Multi-disciplinary Problem

4. Person A asks question. Based on intensity and prosody and other stimulus relevant information this question has a certain level of salience A motivation to deceive (gain, defense) can occur either before or after recall of the information. However, in most cases motivation occurs post recall. Motivation Memory Salience Workload Salience of the question is adjusted based on Person B’s motivation, personality characteristics, and learning history. Attention Emotion Decision Inhibition Orienting related to salience results in changes in the peripheral nervous system Attention is directed towards the question, and the truthful information is pulled from memory. Changes in arousal affect attention, specifically working Memory. This arousal will either Facilitate or impair deception. At the point a decision is Made, the truthful response must Be inhibited and a deceptive response must be generated. The time is takes to generate This response is determined by how far this response will deviate from the truth and how complex this response is. Arousal

5. A Cognitive-physio Model of Deception Motivation Memory Salience Workload Attention Emotion Decision Inhibition Arousal fMRI ERP PET Resp GSR HR BP MEG CNS Measures Physiological Measures

6. What is testable at the cognitive neuroscience level? • Certain aspects related to deception are visible in the HD-ERP signal • Attention: P3a • Memory: P3b • Deception Complexity: N4 and LPC • Salience: P3a P3a: Early attention component with an anterior distribution and positive deflection. Occurs when one switches tasks such as from telling the truth to telling a lie. P3b: A late component that is related to decision making, workload, inhibition, and attention, and context updating. N400: A component that occurs when what we’ve heard, said, or seen does not match the contents of our semantic (and possibly) episodic memory. Anterior distribution, negative deflection 0 250 500 750ms

7. The Directed Lie Paradigm Participants respond truthfully to one color and lie to the other. Response Agree Response Disagree Response Agree Response Disagree 750ms Example of someone in “Blue True” condition.

8. Stimulus 1 (2500ms) Fixation Prompt (750 ms) Stimulus 2 (Response Termination)

10. Semantic Paradigm Results: P3a When deception was predictable but congruity was not and when neither were predictable, the PCA component of the early frontally-distributed waveform was greater for deceptive responses than truthful responses F(1, 42) = 4.79, p = .034 and F(1, 27) = 4.44, p = .045.

11. Congruent Incongruent Deceptive Truthful Deceptive Truthful Results: P3b Exp. 2 Exp. 3 Topographic distribution of Principal Component scores When congruity was predictable but deception was not and when neither were predictable, the P3b component was significantly smaller in the deceptive condition than in the truthful condition, F(1, 42) = 5.37, p = .028 and F(1, 27) = 6.63, p = .028.

12. Congruent Incongruent Deceptive Truthful Results: N4 When deception and congruity were predictable, and when only deception was predictable, the mean PCA scores for the N4 were significantly more negative in the incongruent condition than in the congruent condition [F(1, 33) = 22.59, p < .0001, and F(1,42) = 46.75, p < .0001], respectively. However, when neither were predictable the N4 waveform was not observed in the data. The PCA scores for deceptive responding were significantly greater than those for truthful responding when both congruity and deception were predictable F (1, 33) = 5.33, p = .027. The relationship between the conditions was similar when congruity was predictable; however, the effect only appeared as an interaction with the congruity effect.

13. HD-ERP Results for Participant 2506-HDuring Deception H Lie P3a P3b N4 Left N2 N4

14. Participant 2506 (H): Dipole for P3a corresponds to fMRI Activation in Anterior Cingulate Electrical Activity on Scalp At 304 ms Dipole in Anterior Cingulate fMRI Activation in Anterior Cingulate

15. P3b Participant 2506-H 93%

16. N2 Participant 2506-H 90%

17. N4 Participant 2506-H 86%

18. Standard Analysis • Standard Waveform Analysis • Averaging and Statistical Comparison • Amplitude and Latency Measures • Topographic Mapping • Principal Components Analysis (PCA) • Averaging and Statistical Testing • Spatial and Temporal Components • Topographic Mapping • Dipole Source Analysis • Averaging and Amplitude and Latency Measures • PCA, Independent Components Analysis (ICA) • Source Localization • Individual Response Analysis • Data Replacement • Independent Components Analysis • Cluster Analysis • fMRI Seeded Dipole Analysis • fMRI Parametric Mapping and Cluster Identification • HD-ERP Averaging and Amplitude and Latency Determination • Localization of fMRI Locations at Max Amp/Latency Along ERP Waveform

19. Data Collection • Basic Procedure • Subject Preparation • Data Amplification • Data Acquisition • Data Visualization

20. If All Goes Well!

21. Data Problems • Hardware • Amplifier “Buzz”

22. Artifacts • Eye Blinks • QRS (Heart Beat) • Muscle Tension (Skull)

23. Subject Variation • Psychology • Physiology

24. Data Analysis

25. Proprietary Software • EMSE (Source Signal Imaging) • An integrated solution for brain electromagnetic source estimation • FEATURES • Data Editor • Source Estimator • Source Visualizer • MR Viewer

26. Research Software • Human Brain Project (LANL) • MEGAN • MEG and EEG Analysis & Visualization • MRIVIEW • An Interactive Tool for Brain Imaging • FEATURES • 2D Mode • 3D Mode • Source Localization • Model Viewer

27. In-House Software • JAVA Based Solutions • Area and Latency Analysis of ERP Data • Phase Space Plots of ERP Data

28. VB Based Solutions • Talairach to MRViewer Coordinate Conversion

29. Model Development and Enhancement

30. Continuing Research Working Memory Semantic Information

31. Current Efforts • Modeling Deception as a Two State System • Building a Continuous Potential Model • Workload, Salience and Deception

32. Wish List • A “Halo” System (Across Subject) • A Unified Analysis Environment (Across Platform)