Establishing a Worldwide Driving Simulator Scenario Library Matthew Rizzo

1. Establishing a Worldwide Driving Simulator Scenario Library Matthew Rizzo Division of Neuroergonomics University of Iowa, Department of Neurology Iowa City, Iowa USA HFES, New Orleans September 22, 2004

2. Introduction • This brief talk outlines advantages of developing a standardized shared library of driving simulator scenarios -- and hopefully serves as a call to action. • Having standards for driving simulator a common library of standardized driving simulator scenarios is essential. • Why?

3. Introduction • Simulators are an important means for safe and objective assessment of performance capabilities in normal and impaired automobile drivers. • Yet, substantial problems that limit the utility of simulator research.

4. Introduction • Simulation is advancing, yet it remains a cottage industry of home grown devices with little overall standardization. This is a major hindrance to research. • Key problems facing simulator users are related to: • simulator graphics, audio, and movement • scenario design and validation • subject adaptation and comfort • selection of performance measures • technical standards for reporting experimental set-up and results.

5. We Need Standard Scenarios • Standards for simulation and and simulator scenarios can facilitate comparisons of operator performance across different simulator platforms. • Availability of standards for simulation and simulator scenarios can facilitate collection of large amounts of data at different institutions with greater power for addressing worldwide public health issues related to driving. • Comparisons against standards can clearly reveal missing descriptors and other weaknesses in ever mounting numbers of driving simulator research reports.

6. Issues in Standardization • How well can most simulator scenarios be described within a common framework? • Should researchers be expected to implement a key scenario just one way? • Can levels of fidelity be adequately specified for cross-platform comparisons? • How can we overcome vested interests and entrenched opinions on simulators?

7. Clinical Trials • We need simulator standards to implement clinical trials in drivers with a variety of medical impairments. • NIH definition: “A clinical trial .. is a research study in human volunteers to answer specific health questions. [Clinical trials] are the fastest and safest way to find treatments that … improve health. Interventional trials determine whether experimental treatments .. are safe and effective under controlled environments. Observational trials address health issues in large groups of people or populations in natural settings.” • http://www.clinicaltrials.gov/ct/info/whatis#whatis • A clinical has "some formal structure of an experiment, particularly control over treatment assignment by the investigator" (Piantadosi, 1997, "Clinical Trials: A Methodologic Perspective”).

8. Many dimensions to consider • Dr. Jeff Greenberg, Chief of VIRTTEX at Ford's Research Laboratory, reviewed issues in simulator fidelity relevant to standardization (SUG @ TRB, 1/13/04): • http://www.uiowa.edu/~neuroerg/index.html). • Simulator fidelity and infidelitycan be characterized along 21 key dimensions along 3 key domains • Cueing systems: Visual, Auditory, Haptic or Tactile, and Vestibular • Physical environment: Cab, Laboratory, Virtual environment • Visual world: Terrain and roadways, and Vehicle Models

9. There is hope • Some differences in physical fidelity may not affect psychological fidelity • The driver in the loop may behave similarly across a range of small physical differences in simulation • Meaningful measures may even be had in a surreal simulation • Standards are within our reach, with a promising payoff in new research.

10. Clinical trials in driving safety: examples • Cervical immobility • Reduced visual fields • Ecstasy, marijuana and other illicit drugs • Parkinson’s Disease • Shoulder (rotator cuff) surgery with sling, leg casting • Dialysis • Attention Deficit Disorder • Cirrhosis • Detrussor spasm in women • Low back disorders • Ambulatory surgical/anesthesia restrictions • Post abdominal surgery • Anti-hypertensive medication effects • NPH pre and post-shunting • Post ambulatory anesthesia

11. Things to agree on • Scenarios • Measures • Standards of reporting in journals • Subject selection criteria • Reporting of independent measures • Standards for physiologic recording • Visuals, audio, haptic cues • Training criteria • Simulator adaptation • What is a simulation • Criteria for validation

12. HASTE effort • Mandatory driving performance measures •  Speed • Speed variation • Lateral position • Lateral position variation • Lane exceedences • Time to line crossing • Reversal rate • Time To Collision, Time headway and Distance headway Brake reaction time

13. HASTE effort • Optional driving performance measures • High frequency component of steering wheel angle variation   • Steering entropy • High risk overtakings • Abrupt onsets of brakes • Post Encroachment Time

14. HASTE effort • Mandatory workload measures • Rating Scale Mental Effort (RSME) •  Glance frequency • Glance duration • S-IVIS performance • Situation Awareness Global Assessment Technique (SAGAT) • Optional workload measures • Heart Rate (HR) and Heart Rate Variability (HRV) • Respiration • Skin Conductance

15. Rationale for selecting preliminary scenarios for a standard library • Importance in the epidemiologic record -- focus on key situations conveying potential high crash risk, e.g.: • Run off the road on curves • Rear end collision • Intersection incursion avoidance • Interaction with emergency vehicles/pedestrians • Merging with the potential for side impact collisions • Scenarios that address behavioral effects of using in-vehicle devices, e.g., cell phone, information displays

16. Sample Format presented at SUG 1/13/04 • Sample Format for Describing a Simulator Scenario: Lane Change with Slow Traffic (SUG 1/13/04) • Specifications • Script • Cognitive constructs stressed • Dependent measures • Independent variables • Implementation challenges • Testing Validity of Scenario • Bibliography

17. Sample Scenario Description • Script: The driver is traveling on a road with two lanes of traffic, each moving at different speeds. At different times, one lane of traffic is moving more advantageously (faster) although overall this may be the slower lane. The driver’s task is to pass through traffic as quickly as possible. • This task would be similar to the “Gambling Task” (Bechara et al, 1994) in which an individual has to overcome an immediate reward to ensure long-term benefit. • Variations can be made on this script: the two lanes of traffic could average the same speed or could even be moving at the same speed. The driver’s perception that one lane is moving faster may be a visual and cognitive illusion.

18. Sample Scenario Description • Cognitive constructs stressed: Attention, perception, and decision-making. • Dependent measures: Time it takes the driver to maneuver through the traffic to arrive at a destination, number of navigation errors, and number of moving violations or safety errors (e.g., excessive speed, near misses, and collisions) could be recorded. • Independent variables: The number of vehicles involved, number of lanes, speed of the different lanes and final destination, and travel contingencies could be manipulated (e.g., the driver could be instructed to get off at a specific exit for a hospital).

19. Sample Scenario Description • Implementation challenges: • It may be difficult to create a realistic feel. The problem may be lessened by giving the driver external instructions, thus altering driver expectations and rewards or incentives. For instance, the instructions could be to drive as if you were taking a pregnant woman or a critically sick person to the emergency room. • Is it possible for the driver to be able to change lanes when desired? • How will the surrounding cars respond to the driver?

20. Sample Scenario Description • Measurement challenges: • Some drivers will not try to get in the “faster” lane, if they think it makes little difference in the long run. • A questionnaire following the task may be helpful in assessing how fast the driver felt each lane was moving, and whether or not they the driver have changed lanes if given the opportunity. • Variations in personalities would have to be considered.

21. Sample Scenario Description • Testing Validity of Scenario: An instrumented vehicle could be used to study lane change behavior during times of high- density traffic, yet the environmental variables could not be easily controlled. • Bibliography • Bechara , A., Damasio, A. R., Damasio, H., & Anderson, S. W. Cognition , 50, 7-15 (1994) • Redelmeier, D.A. & Tibshirani, R.J. Nature 401, 35 (1999).

22. Sample Scenario Description • Testing Validity of Scenario: An instrumented vehicle could be used to study lane change behavior during times of high- density traffic, yet the environmental variables could not be easily controlled. • Bibliography • Bechara , A., Damasio, A. R., Damasio, H., & Anderson, S. W. Cognition , 50, 7-15 (1994) • Redelmeier, D.A. & Tibshirani, R.J. Nature 401, 35 (1999).

23. Value of scenario plots for visualizing driver response What would you do if you saw a police car parked on the side of a 2-lane highway with no vehicles coming in the other lane?

24. Value of task analysis:Response to darting dog ahead, honking SUV behind e-dog Brake application

25. Honking SUV in rear view Gas reaction 0=Nothing, 1= Release, 2=Press; Frame At-fault safety errors Quantitative vehicular control measures 1s Brake reaction 0=Nothing, 1= Release, 2=Press; Frame Dog darts to road from right Parkinsonism 0=No change, 1=Worse, 2=Better; Frame Frame, speed Gas reaction 0=Nothing, 1= Release, 2=Press; Frame At-fault safety errors Quantitative vehicular control measures Brake reaction 0=Nothing, 1= Release, 2=Press; Frame Steering reaction 0=Straight, 1= Right, 2=Left; Frame Parkinsonism 0=No change, 1=Worse, 2=Better; Frame Outcome (Frame, speed) 1=Avoids dog, no crash with oncoming traffic 2=Hits dog, no crash with oncoming traffic 3=Avoids dog, but crashes with oncoming traffic 4=Hits dog and crashes with oncoming traffic

26. Summary • General standards are lacking for scenario design and dependent measures. • Lack of standards limits research and testing replication. • Representations of simulator scenarios in a task analysis format help to clarify the logical structure of scenarios and related issues such as operational definitions and measurements of variables. • Joint consideration of these issues will aid in the creation of a proposed driving scenario library. • The effort will contribute to an infrastructure for worldwide research on critical public health issues concerning driver performance and vehicle design, with the support of major funding agencies such the NIH and NHTSA.

27. How shall we proceed? • We for your expert opinions and advice: • Which key scenarios are most worth implementing across different simulator platforms for clinical trials. • How should these scenarios be represented? • Please Contact • Matt Rizzo (matthew-rizzo@uiowa.edu) or • Jeff Caird (jkcaird@ucalgary.ca)

28. Action Items • Continue to examine key topics on standardization, physical and psychological fidelity, reporting of methods, selection of variables and scenarios, and simulator discomfort in simulation research. • Identify funding sources for addressing these concerns in anticipation of future research trials (e.g., involving impaired drivers or patients and on board safety devices). • Prepare a white paper outlining suggestions for standards • Continued efforts to spread the word • TRB Simulator Workshop 2005 (Fisher, Mourant, Rizzo) • Driving Assessment 2005, including SUG meeting • (Rockport ME, June 27-30).