Dmitriy Babichenko, Jonathan Velez - To Scope or Not To Scope: Challenges of Gamifying Clinical Procedures Training

1. To Scope or Not To Scope Challenges of Gamifying Clinical Procedures Training

2. Who are we? Dmitriy Babichenko Jonathan Velez Kailani Bailey William O’Toole Ravi Patel ● ● ● ● ●

3. Acknowledgements While we take full responsibility for any errors and shortcomings of this presentation, we would like to thank the following people for their constant support of this project, for offering their medical, educational, and gamification expertise, and for providing indispensable advice on all aspects of design and implementation. Lorin Grieve, PharmD, Instructor, Pharmacy and Therapeutics, University of Pittsburgh School of Pharmacy John Lutz, Director of Information Technology and Co-Director of Research, Peter M. Winter Institute for Simulation, Education, and Research (WISER) Phillip Lamberty, MD, Director, Medical Intensive Care Unit UPMC Presbyterian Hospital, Director, Pulmonary and Critical Care Ultrasonography, Medical Director, Select Specialty Hospital Pittsburgh Deborah Farkas, PhD, Director of Educational Development, Peter M. Winter Institute for Simulation, Education, and Research (WISER) Timothy Meehan, Student, University of Pittsburgh School of Information Sciences Taylor Winn, Student, iSchool Inclusion Institute (I3) Morgan Freeman, Student, iSchool Inclusion Institute (I3) Michael Depew, Director, iSchool Inclusion Institute (I3) ● ● ● ● ● ● ● ●

4. Who are you?

5. Challenges (should you choose to accept them) 1. Structure an in-house game or simulation development project 2. Connect learning objects to system requirements, game mechanics, and technologies 3. Establish a framework that facilitates learner evaluation

6. ScopingSim

7. What is bronchoscopy? (Medline - https://medlineplus.gov/ency/images/ency/fullsize/23232.jpg)

8. Motivation #1 - Cost Medical simulators & task trainers are EXPENSIVE (and creepy) 1. 2. 3. High cost of purchase High cost of maintenance Low simulator to student ratio

9. Motivation #2 - Feedback High-end simulators & task trainers provide limited feedback based on anatomy & physiology ● Difficult to determine whether a learner performed well because of skill / knowledge or just luck ● No user model / learner model ●

10. Motivation #3 - Debriefing Students cannot practice on their own without supervision of domain expert ● Domain experts’ time is expensive ● Low domain expert to student ratio ● Without debriefing complex procedure simulations are virtually pointless ● Image source: https://cphp.org/critical-incident-debriefing-services/

11. Motivation #4 - Generalizability ● Common procedure(s) - bronchoscopy, endoscopy, colonoscopy ● Good generalizability ● Number of interested stakeholders Laerdal SimMan (http://www.laerdal.com/us/doc/86/SimMan)

12. Let’s Roleplay Education/Game Expert IT Expert Medical Expert

13. Mission Impossible I: Define Requirements

14. What did the stakeholders want? ● Something that doesn’t require an expensive mannikin & easily distributable ● Provides real-time feedback & just-in-time learning ● Debriefs learner without the help of a domain expert

15. What did the stakeholders want? (Learning Goals) ● Something that doesn’t require an expensive mannikin & easily distributable Deliberate Practice ● Provides real-time feedback & just-in-time learning - Reinforcement Learning ● Debriefs learner without the help of a domain expert - Reflective Learning

16. Organize Your Ideas Mission Impossible II

17. Affinity Diagram Organizes a large number of ideas into their natural relationships Taps a team’s creativity and intuition When to use ○ You are confronted with many facts or ideas in apparent chaos ○ Issues seem too large and complex to grasp ○ Group consensus is necessary ● ● ● http://asq.org/learn-about-quality/idea-creation-tools/overview/affinity.html

18. Affinity Diagram Procedure Look for ideas that seem to be related in some way. Place them side by side. ● Form notes into “ idea” clusters ● Discuss any patterns, especially reasons for moving controversial notes, and potentially split or merge clusters ● Repeat ● http://asq.org/learn-about-quality/idea-creation-tools/overview/affinity.html

19. Affinity Diagram Procedure Now it’s your turn http://asq.org/learn-about-quality/idea-creation-tools/overview/affinity.html

20. Mission Impossible III: Priorities.. Priorities..

21. Make the Hard Decisions Recognize that each idea in the pool may be competing ideas ○ How will committing to one idea influence the feasibility of other ideas in its cluster? ○ How will committing to one idea influence the feasibility of ideas in other clusters? Analysis & Design ○ Select and prioritize requirements ○ Requirements should support learning objectives ○ Be agile..! Requirements and feasibility can change at any moment ● ●

22. ScopingSim with Scope Video Version 1 Setareh Sarachi, Pavitraa, Faris Obaid Alotibi, Abhishek Mukherjee, Dimple Varma

23. ScopingSim with Scope Video Version 2 Siying Zhang, Yue Su, Hanwei Cheng, Zihan Xie

27. Failed Controllers

30. Flex Sensor Ambu Scope Arduino Uno Accelerometer / Gyroscope

32. Mission Impossible Ghost Protocol: User Modeling

33. Building the User Model We have no idea what that might look like at this point...

34. Mission Impossible - Let’s talk about models How do we define performance? ● How do we classify “expertise”? ● When and how do we give feedback? ●

35. How do we define performance? User model constrained by system design Raw measures: ● x, y, z acceleration Time Spatial position in virtual environment Completed objectives Number of failures ... ● ● ● ● ● ● System design constrained by user model ●

36. How do we classify “expertise” ? Ask the experts! ○ How is expertise determined in the domain? ○ Pedagogies for knowledge/skill acquisition Qualify expertise from quantified performance ○ Heuristics vs Analytics Choose your pedagogy: ● Fitt and Posner’s Theory of Motor Acquisition (1967) ○ Cognitive → Integrative → Autonomous ● ● Parameterize your raw data: Changes in direction (avg and std dev) Time in motion (avg and std dev) Time at rest (avg and std dev) Ratio of time at rest to time in motion Velocity of motion (avg and std dev) Accomplishment of game objectives … ● ● ● ● ● ● ●

37. When and how do we give feedback? Feedback elements constrained by system design ○ Environmental cues (e.g., visual, audio, haptic) ○ Implementation examples: mini-map, flash or buzz on environment collision events, servomotor resistance Scaffold learning based on the model of expertise ○ Suggestion mechanisms (helpful or distracting?) ○ Implementation examples: next-step model Contextualize deviations in performance to encourage reflective learning ○ Facilitate debriefing; ask “how” and “why” ○ Detect where learners’ performance can improve and provide enough info for open-ended self-evaluation ● ● ●

38. Mission Impossible: Where are we today?

39. Dmitriy Babichenko - dmb72@pitt.edu To Scope or Not To Scope Thank you! Questions/Answers?

40. Dmitriy Babichenko - dmb72@pitt.edu To Scope or Not To Scope Thank you! Questions/Answers? The Floor is Failure

42. Dmitriy Babichenko - dmb72@pitt.edu To Scope or Not To Scope Thank you! Questions/Answers?