Non-Prescriptive Tools for Effective Fatigue Management DOT Human Factors Coordinating Committee June 23, 2004

1. The Sleep, Activity, Fatigue, and Task Effectiveness (SAFTE™) Model and Fatigue Avoidance Scheduling Tool (FAST™) Non-Prescriptive Tools for Effective Fatigue ManagementDOT Human Factors Coordinating CommitteeJune 23, 2004 Steven R. Hursh, Ph.D. Program Manager, Biomedical Modeling and Analysis Science Applications International Corporation, 443-402-2701 Professor, Johns Hopkins University School of Medicine Hurshs@saic.com

2. Outline • FRA role and objectives in fatigue management initiatives • The SAFTE Model of fatigue • The Fatigue Avoidance Scheduling Tool (FAST) • Update on calibration for accidents and incidents • On-the-Job Performance Study • Fatigue Assessment Dashboard • The Incident Fatigue Assessment Protocol (IFAP) • FAST Schedule Design Wizards and XIMES Interoperability

3. The FRA Role • Partnership between Human Factors R & D Program (Volpe) and Office of Safety. • Supported and coordinated with DOT human factors coordinating committee • Facilitate development of fatigue management tools for the railroad environment • Make available funds to advance this tool • Facilitate and finance necessary studies to calibrate the tool for the railroad environment • Encourage system-wide assessments combined with site specific solutions • Maintain a hands-off approach to applications of the tool • Ensure the confidentiality of information • Disseminate information about fatigue management solutions

4. FRA Office of Safety Objectives • Advance a non-prescriptive approach to fatigue management. • Help develop a fatigue and performance model as a non-prescriptive tool to be used at the discretion of management and labor. • Help develop methods to collect and analyze information (data) that might be used to assess fatigue contributions to accidents and incidents. • Integrate tools so they can be mutually supporting: FAST, Actigraphs, XIMES RAS, PERCLOS • Continue to promote cooperative approaches to fatigue mitigation solutions.

5. Comparative Status of the SAFTE Model • The DOD and DOT sponsored a comparison of six fatigue models from around the world in Seattle. • All models attempted to predict the results from four standard scenarios. • While all models can be improved, the SAFTEmodel had less error than any model tested and was combined with a convenient and logical user interface, the fatigue avoidance scheduling tool - FAST.

6. Model Comparison – Independent EvaluationScenario 2 – Restricted Sleep (data from Dinges Lab – U. Penn) * “Just prior to the Fatigue and Performance Workshop, SAFTE Model was revised and optimized using data from Scenario 5, which is similar to scenario 2. The ….RRMSE value [was] 70.91%...This constitutes a substantial improvement with respect to the earlier predictions from this and all the other models. (page 27)”

7. Estimate of prediction error – population variance. Representation of individual differences. Representation of countermeasures. Translation into task or job performance and risk assessment. Now included in FAST Proposal to Army Ongoing with AF and Army Lapse Index added, other measures are part of DOT effort. General Model Deficiencies Gap Remedy

8. Fatigue Avoidance Scheduling Tool (FAST™) • FAST™ is a fatigue assessment tool based on the SAFTE™ model • Developed for the US Air Force and the US Army. • NTI and SAIC in a Phase III SBIR program. • DOT/FRA sponsored work has lead to a enhancement for transportation applications. • Auto Sleep algorithm • Schedule Grid data entry tool • Wizards and Dashboard - funded • DOT field calibration underway.

9. SAFTE Model and FAST Predict Performance Effectiveness • Effectiveness is a measure of speed of making correct responses. • Effectiveness estimates are at a 1 min. resolution. • Effectiveness is highly correlated with other measures of fatigue: • Lapse likelihood • Reaction time • Average cognitive throughput • Driving simulator performance • Tool predicts both average person and population variance estimate.

10. Adjustable Criterion Line Lower Percentile (e.g. 20%) FAST Graphical Screen Options Effectiveness Sleep Periods in Blue Work Periods in Red

11. Lapses Increase with Decreasing EffectivenessfromFAST (revised)Sleep Dose Response Study – Experimental & Recovery Days - WRAIR Data Days of Increasing Sleep Debt High Low

12. Lapse Index

13. Commercial Applications of FAST™ FAST™ is optimized for the average person, including truck drivers, studied under a range of schedules. It has a variety of applications: • Problem Definition and Assessment • Work Schedule Design and Evaluation • Generic Schedules (shift-schedules) • Individualized Schedules (work assignments) • Safety and Accident Investigation Tool • Training and Awareness • Voluntary Self-management • DOT Experience: Schedule evaluation and accident fatigue assessment underway

14. Complements Other Existing Tools • FAST provides an objective fatigue and performance estimate that can be used in conjunction with other available tools. • Other Fatigue Risk Management Tools: • Fatigue monitoring devices – actigraphs and PERCLOS • Return on Investment Models • Operational/Terminal Management Models • Staffing Tools • Team decision-making tools • Other Fatigue Estimation Tools

15. FRA Initiatives • Calibrate FAST for railroad environment • Analysis of accidents and incidents with two railroad partners, pilot study • Analysis of accidents and incidents with two railroad partners, large sample study • Analysis of locomotive engineer performance • Enhancement of tool for accident and incident investigations • Fatigue indicators “dashboard” based on NTSB workshop • Input wizards for irregular and shift schedules • XIMES interoperability • Incident fatigue analysis protocol

16. Fatigue Risk Management Tools • FAST ™ – Fatigue Avoidance Scheduling Tool • Prospective forecasting of fatigue risk under proposed work/rest schedules. • Retrospective assessment of fatigue leading up to an event. • Uses the SAFTE model of fatigue developed by the DOD. • IFAP™ – Incident Fatigue Assessment Protocol • Questionnaire and schedule assessment software to aid event analysis. • Integrates data into FAST for fatigue assessment.

17. Validation Schematic Components of the Model Intrinsic Process Validation: Laboratory Cognitive Performance Predictions Primary Predictive Validation: Confirming Validation: Task Performance Predictions Safety & Accident Predictions Subjective Fatigue Predictions DOD & DOT Projects will contribute to these three areas of validation

18. SAFTE MODELPredictions and Data Total Sleep Deprivation (WRAIR 72 hr Study)

19. Decline of Performance with Total Sleep Deprivation

20. SAFTE Model (Revised)Walter Reed Army Institute of Research Sleep Restriction Study Sleep duration is total observed according to EEG measurement. Model parameters are constant across conditions of the experiment.

21. Congruence of SAFTE Model to Sleep Deprivation Data

22. Missed Horns AnalysisITRI Locomotive Simulator Chart 3

23. SAFTE/FAST Analysis of Incident DataPilot Study • Pilot study of 50 events from two railroads. • Report here data for only one railroad. • For each event, there were usually two operators: engineer and conductor. • Results are tentative because of small sample size; large study of 500 events is pending railroad approval. • Results illustrates the analysis approach.

24. Work starts and stops Call times Commute times Particular sleep habits Actual sleep times Quality of sleep environment Schedule predictability Considered Considered Considered Not considered Not considered Not considered Not considered Important Data for FAST

25. Important Data Not Considered • Schedule delays or misinformation • Training and experience • Medical conditions and sleep disorders • Medications and/or drug use • Observations of operator performance and appearance • Concurrent stress, family issues, or work demands • Crew resource management (communication problems)

26. Fatigue Analysis Overview • FAST is a tool for evaluating and combining fatigue factors in a schedule and providing an objective and impartial performance prediction. • The model is designed to predict fatigue, not incidents and accidents. • Many human factors accidents occur without fatigue. • Some non-human factors accidents may have a fatigue component that was not recorded. • However, our hypothesis was that the model should indicate an increased probability of events at the extremes of predicted fatigue (low effectiveness).

27. RR-X Incident Effectiveness(Least Effective Crewmember)

28. RR-X Incident Effectiveness Probabilities(Least Effective Crewmember)

29. FAST Can Discriminate Human Factors vs Nonhuman Factors Events Receiver Operator Curve FAST with effectiveness criterion at 67% Percent correct is 68% A’ = .77 Comparisons: Materials Testing (detection of cracks in airplane wings)     Ultrasound - 0.68     Eddy Current - 0.93  Medical Imaging (detecting tumors)         Ultrasound, adrenal gland - 0.83  Weather Forecasting     Extreme cold - 0.89     Rain    - 0.82     Fog - 0.76     Storms - 0.74     Temperature Intervals - 0.71  Polygraph Lie Detection     Real crimes - 0.70 to 0.92 Promising but more data needed.

30. On-the-Job Performance StudyPurpose and Process • To determine if the FAST software can make valid predictions of operator performance changes that result from fatigue and circadian variations. • Analysis Steps: • Collaborate with two or more Class 1 railroads to collect event recorder data. • Correlate performance measures from engineers along with work schedule information and actigraph measures of sleep. • Determine if the FAST predictions are consistent with general variations in performance resulting from sleep patterns and time-of-day effects. • Determine if interventions reduce performance changes. • Status: Looking for collaborators – could use historical data, if available. • FAA-CAMI (Tom Nesthus) is planning a study of fatigue in coordination with the International Flight Inspection Office to test the predictions of FAST. • Actigraphs, Logbooks, ARES Performance Battery, and perhaps oral temperature. • The IFIO missions are typically 3-wks out (Europe, Far East, or wherever).

31. Supported by Transportation Research Board Research Recommendations • (1) Measurements of effectiveness of safety interventions of any sort. Relationship between fatigue and operator performance. • (7) Sensitivity of fatigue models to performance-related measures? Relationship between fatigue and operator performance. • (8) Incidence of fatigue in train crews based on work schedule data.

32. NTSB Workshop on Fatigue Analysis in Transportation Accidents • Rosekind and Dinges – Expert Consultants • Recommended in-depth assessment of operator schedules, sleep need and habits, medical history, and medications. • Recommended calculation of fatigue indicators. • FRA now sponsoring work to add fatigue indicators to FAST, along with performance prediction. • Feature will provide immediate fatigue explanation of low predicted performance to aid the accident investigator.

33. Other Initiatives • Fatigue Indicators Dashboard • Incident Fatigue Assessment Protocol - Update • Schedule Design Wizard • FAST – XIMES RAS interoperability (Details follow)

34. All of the indicators can be computed from variables available within FAST. All the performance indicators are computed as transforms from Effectiveness shown on the graph. • Fatigue factors within a dangerous range are indicated by a red flag based on criteria established at NTSB workshop. • The performance indicator will change color from green to yellow to red. When performance is less than green, the flagged fatigue factors will provide an explanation. • All these values would be continuously updated as the cursor is moved along the time line of the schedule, giving an instantaneous assessment at any moment or at some critical event.

35. Schedule Name: Date:Time When activated, the dashboard will appear as a window within the schedule window and update with the movement of the mouse. A left mouse button depression within the schedule area will clear the table; release the mouse button and it will rewrite with new values appropriate for the new cursor position. The “dashboard” can be dragged to any convenient place within the graphical window. A right mouse click will allow the user to print or copy the dashboard to the clipboard for paste into another application.

36. Incident Fatigue Assessment Protocol • Computer-based investigation questionnaire of operators and witnesses • Computerized version of Technical Bulletin (proof of principal) • Keppen Associates initiative to test and improve questionnaire • Computer-based schedule recording and analysis • Access database creation • Automatic porting to FAST • Potential to be ported to PDA

37. Incident Fatigue Assessment Protocol(IFAP) Schedule Entry Screen

38. Schedule Design Wizards – Funded (Volpe Project) • Shift-schedule wizard • Irregular schedule wizard • Dialog data entry with options to branch to tabular or grid entry (“TurboTax” metaphor) • Descriptive results output • Graphical • Tabular • Fatigue factors • Narrative

39. FAST and XIMES RAS Interoperability – Funded(Volpe Project) • XIMES RAS is an Austrian developed shift schedule design tool. • Assists in design and descriptive analysis of shift schedules • Contains rule of thumb schedule evaluation • Does not assess fatigue potential • Initiative will create standard schedule file format that can be shared by these two tools and an other schedule analysis software. • Schedules created with XIMES RAS can be imported into FAST for sleep assessment and fatigue analysis.

40. Recommendations • Continue to advance our knowledge and understanding of software modeling of fatigue as one tool to address fatigue. • Continue to seek cooperation of railroad partners in collecting essential data. • Continue to seek performance indices to quantify the role of fatigue in railroad operations. • Continue to coordinate and strengthen transfer of our knowledge to other modes. • Welcome partnerships with other modes to fill data gaps. • Continue to strengthen our working relationships with AAR’s Work Rest Task Force and NARAP.

41. Non-Prescriptive Tools for Effective Fatigue Management End of Presentation Steven R. Hursh, Ph.D. Science Applications International Corporation, 443-402-2701 Professor, Johns Hopkins University School of Medicine Hurshs@saic.com

42. Modeling Approaches • Fatigue Audit InterDyne™ (FAID) Model: Drew Dawson (University of South Australia) - fatigue model currently being applied as a fatigue management tool in Australia and by the Union Pacific. • Sleepwake Predictor: Torbjorn Åkerstedt (Karolinska Institute) & Prof. Simon Folkard (University of Wales) • Interactive Neurobehavioral Model: Megan Jewett, (Harvard University) • System for Aircrew Fatigue Evaluation (SAFE): Spencer and Belyavin (QinetiQ, Inc, UK) • Circadian Alertness Simulator (CAS): Martin Moore-Ede(Circadian Technologies, Inc., USA) • Sleep, Activity, Fatigue and Task Effectiveness (SAFTE) Model: Steven Hursh (SAIC and Johns Hopkins University, USA).