Virtual Experiences Research Group

1. Virtual Experiences Research Group • PhD Students • Kyle Johnsen, Aaron Kotranza, John Quarels, Andrew Raij, Xiyong Wang, Brent Rossen • Undergraduates • Joshua Horton, Harold Rodriguez • Funding • National Science Foundation (CAREER, REU), University of Florida Colleges of Engineering and Medicine, Medical College of Georgia, Keele University, School of Pharmacy

2. Graduating Students • Kyle Johnsen • http://www.cise.ufl.edu/~kjohnsen • Andrew Raij • http://www.cise.ufl.edu/~raij • Harold Rodriguez • Undergraduate

3. Immersive Virtual Humans for Educating Medical and Pharmacy Communication Skills K. Johnsen, A. Raij, B. Rossen, A. Kotranza, X. Wang, B. Lok Computer and Information Science and Engineering M. Cohen, A. Stevens Surgery J. Cendan Community Health and Family Medicine R. Ferdig Education A. Deladisma, D. S. Lind Surgical Oncology S. Chapman, L. Bracegirdle Pharmacy

4. Can Virtual Humans Enable…

5. VOSCE Project Overview • Started Spring ’04 • n > 250 students • Medical • Nursing • Physician Assistant • Pharmacy • One of the most popular VH/VR experiences • Three institutes • University of Florida • Medical College of Georgia • Keele University (U.K.) • Team • VR/HCI/CS – 1 PhD, 6 grad students, 2 undergrads • Medicine – 6 MDs, 2 medical students • Education – 2 PhDs • Research focus on interfaces

6. Play Video

7. Why Virtual Humans? • Students • Repetition • Feedback • Longitudinal learning • Educators • Standardization • Dynamic • Abnormal findings • Cultural competency • Aggregate performance • Researchers • Study the extent of impact of VHs • Easy to run studies (Twiddle one thing)

8. What can a Virtual Human do? • Respond to a sneeze • Show empathy • “I’m scared, can you help me?” • “Could this be cancer?” • Sneezing • Evoke emotion (e.g. anxiety) • Physiological effects • Social conventions • Similar to a human • So what happens with VH’s of different backgrounds?

9. VHs and Bias • VHs elicit racial/ethnic bias • Working to mitigate effects of bias

10. Mixed Reality Humans • Virtual humans have limitations • Open research problems (e.g. AI, speech) • No tactile feedback • Merge the real and virtual spaces • Real tools • Real simulators • Complete patient interaction • Physical Exams (e.g. eye exam) • Point at eye chart • Ophthalmoscope • Follow my finger • 1 or 2 fingers?

11. Breast Simulator Integration • Breast simulator integrated • Dr. Carla Pugh, Northwestern • Student does a patient history • Asks to remove gown (physiological measures) • Performs a breast exam • VP winces at too much pressure • Future work • Pelvic simulator • Central line simulator • [Kotranza (submitted) VR2008]

12. Potential Applications for VHs • Training Interpersonal Scenarios • Employee Training • Social situations • Sexual Harassment • Cultural Sensitivity/Competency • Business Dealings • Patron diversity (physical, emotional, mental) • Park Visitor Experience Enhancement • Kiosks for patrons to interact with their own culturally knowledgeable guide • Personalized “ride” that allows patrons to have immersive experiences

13. IPS System • Inputs • Natural speech • Tracking data (head, hands, chair, tools) • Video • Physiological measures • Outputs • Speech and animation • Life-size projection (or HMD) • Perspective correct rendering • Reactive virtual human • COTS components • 2 PCs • 2-4 video cameras • Data projector or HMD • Wireless microphone • Bodymedia Sensewear • < $10,000(USD) • Potential: • Every Hospital

14. Behavior Cue Analysis Student interaction with VH1 Δ? After-Action Visualization Student interaction with VHn Studying diversity issues with VHs Bias • Study (n=9), people reported that the African-American VH had less education and money • Identical: animations, words spoken • Differences: skin-tone, voice • Other biases to study • Race/Ethnicity • Gender • Age • Weight • Social ‘hot topics’

15. Abnormal Findings • Conditions difficult to represent with existing education methods • Conditions • Psychomotor • Neurological • Age/race/ethnicity dependent conditions • Blurry vision scenario • Cranial Nerve III (due to brain tumor) • Corneal ulcer • Retinal detachment • Benefits • Curricular planning of medical student exposure • Supplement SP experiences • Leverage the dynamic nature of VPs • [Wang ISMAR 2007, submitted VR 2008] • Play Video

16. Virtual Patients Level of Interactivity Standardized Patients Physical Simulators (Harvey, HPS, etc.) Textbooks and Journal Articles Abnormal Normal Patient Condition Current Work • Classroom incorporation • Communication course • 2nd year MS • Surgical rotation 3rd year MS • Pharmacology 1st year • n>120 per year • Potentially 30,000 interactions a year @ existing institutions • Virtual Instructor • Real-time response to tracked cues • Physiological measures • Posture cues • Verbal cues

17. Scenarios and VPs • Acute abdominal pain • Breast mass • Dyspepsia • Sexual history • Eye exam • Patients • DIgital ANimated Avatar (DIANA) • Elderly Diana (Edna) • Manniquin Diana (Mandi) • Building the following VPs • Male • Personality (e.g. irate) • Intelligence (e.g. mentally retarded) • Appearance (e.g. disfigurement, limbs, burns)

18. Join Us! • Application domain outside medicine • Additional locations to install and test system • Support • Research directions • Students • Thank you! • http://www.cise.ufl.edu/research/vegroup • Questions?

19. Virtual Experiences Research Group • PhD Students • Kyle Johnsen, Aaron Kotranza, John Quarels, Andrew Raij, Xiyong Wang, Brent Rossen • Undergraduates • Joshua Horton, Harold Rodriguez • Funding • National Science Foundation (CAREER, REU), University of Florida Colleges of Engineering and Medicine, Medical College of Georgia, Keele University, School of Pharmacy

20. After Action Review of VH Interactions • Taking a cue from flight simulators • End-Users • Self-reflection • Understanding • Feedback • Educators • Quantitative analysis • Identify trends and outliers • Researchers • Capturing H-VH interaction • Analysis • Process, Filter, Visualize • [Raij (submission) VR 2008, SIGCHI 2008]