THE CHALLENGE OF ROUTINE FLIGHT OPERATIONS Loukia D. Loukopoulos R. Key Dismukes Immanuel Barshi NASA Ames Research Ce

1. THE CHALLENGE OF ROUTINE FLIGHT OPERATIONS Loukia D. Loukopoulos R. Key Dismukes Immanuel Barshi NASA Ames Research Center Moffett Field, CA U.S.A. 22nd European Annual Conference on Human Decision Making and Control 2-4 June, 2003 Linköping University, Sweden

2. Flows Checklists  Procedures Phase of flight structure •  ? MOTIVATION • Complex operating environment (cockpit) • pilots • number of tasks • timing of tasks • INTERACTIONS among tasks and operators • Characterize nature of interaction • Characterize demand such interaction places on cognition • Errors? • Solutions?

3. METHODS • Comprehensive approach • Cockpit jumpseat observation of routine flights (~100 hours) • Boeing 737; 2+ air carriers; 1- 4 hours per leg; U.S.-wide • notes; informal interviews • Training (initial, transition/upgrade) • Written materials (manufacturer and carrier FOMs) • Incident reports (ASRS, NTSB) • Qualitative analysis applies to all phases of flight*use Preflight for illustration (Taxi Out in paper) *

4. assumed ideal • Errors pilot perspective (ASRS reports) real PERSPECTIVES - layering • Ideal manufacturer perspective (manuals) • Assumed carrier perspective (FOMs, training) • Real observer perspective (jumpseating)

5. Manufacturer’s representation? (ideal)

6. PHASE OF FLIGHT - ideal (manufacturer) • Goals: • Prepare aircraft for next phase of flight • Considerations: • Safety • Design specifications • Perspective: • Detached from operational reality

7. Preflight - ideal (manufacturer) CAPTAIN FIRST OFFICER • Ask for checklist • Ask for checklist • Begin checklist • Checklist complete • Begin checklist • Checklist complete procedure procedure checklist • CLEARANCE checklist • ENGINE START & PUSHBACK

8. Carrier’s representation? (assumed)

9. PHASE OF FLIGHT - assumed (carrier) • Goals: • Prepare aircraft for next phase of flight • Considerations: • Safety • Design specifications • Additional constraints • Priorities (fuel, time, passenger comfort) • Type of operations (short/long-haul, turn-arounds, regions/airports, fleet) • Philosophy (safety, efficiency) • New technologies (ACARS, on-board performance computer) • Coordination with agents (dispatch, maintenance, company) • Perspective: • Suggestive of operational reality

10. Preflight - assumed (carrier) Ground/ Company/ Dispatch Frequencies Interphone Cabin Attendant Gate Agent ACARs / OPC CAPTAIN FIRST OFFICER • Review paperwork • Sign flight release • Prepare/review charts • Review Load Schedule • Review FMC • Takeoff brief • Ask for checklist • Ask for checklist • Obtain ATIS • Obtain clearance • Review paperwork • Prepare/review charts • (Passenger count) • (Load Sheet) • Program FMC • Begin checklist • Checklist complete • Begin checklist • Checklist complete procedure procedure checklist • CLEARANCE checklist • ENGINE START & PUSHBACK

11. PHASE OF FLIGHT - assumed (carrier) CAPTAIN FIRST OFFICER + x monitor A B C D 1 2 3 4 - y INFORMATION if z then C’ • EVENT • Activities are: • Linear: task B always follows task A, in this sequence • Controllable: tasks initiated by pilot, independently • Predictable: information available when needed • communication possible when necessary • PILOT-driven operation

12. COGNITIVE ASPECTS - (carrier perspective) • Activities are: Linear, Controllable, Predictable • Training and repeated Practice on the line build: • Automaticity(reduced cognitive requirements) • Habit(speed, accuracy) • Reliance on triggers(memory aid) • (task 2 triggers task 3) • (Captain’s request for the checklist cues FO to begin checklist) • Are these good outcomes? • √ Standardization (between pilots/crews) • √ Accurate, efficient, “safe” performance

13. … and so it should all work as expected… … or maybe not?!

14. PREFLIGHT Captain’s ASRS Report (edited) • “At this point the scheduled flight is about 1.5 hours behind schedule. Prior to our arrival we experienced some problems with the autopilot system and had to call maintenance for approval to continue operations in accordance with the MEL. I discussed this with FO who contacted company, and called for fuel. I went inside the terminal to call Maintenance. • I headed back to the aircraft… the passengers were boarding and the FO was at the back of the aircraft. As we both got into the cockpit, the FO started the Load Manifest and I completed the write-up in the logbook. I then started the Before Start checks but was distracted by the events of a final baggage and passenger count. There were other flights on the ground and I had seen the fuel truck moving about. • With the passengers and bags onboard and the doors closed, the engines were started and a call for taxi made. The main runway has been closed for construction and we were given a shorter runway. After a look at the takeoff data, winds, and weights, we advised Tower we needed a differentrunway. When cleared for takeoff we had not completed the checklist so we advised Tower we needed some more time. Once completed, we departed with a climb in VFR conditions. After climbing through 12000 ft I was scanning the instruments and noticed the fuel quantity only showed 980 lbs. • The series of events from the first landing until takeoff was about 15 minutes. I feel there were many distractions that interrupted the flows on the preflight check and items were skipped.”

15. Skipped over checklist item - interruption - departed without lobgook Forgot to call maintenance - distractions - depart with hatch installed backwards Neglected to add MEL to flight release - multiple distractions, busy with preflight - discovered en route Forgot logbook at ramp - kept deferring to check it; distractions; busy with preflight - discovered en route Never finished review of maintenance log - interruptions - aircraft flown with “open” item in logbook Omitted review of (SID) charts - distractions - speed violation on departure Missed aircraft not refueled - checklist interrupted - return to airport after taking off Skipped over checklist item - fuel pumps deferred during preflight because refueling - engine starvation in flight Skipped over checklist item - interruptions - discover insufficient fuel after pushback Omitted flow and checklist items - interruptions; delay; change in departure runway - discover insufficient fuel at 12000 ft Missed checking main tank fuel quantity - distractions - departed with insufficient fuel Forgot to request updated PDC - distractions; misled because preflight already complete - take off with expired PDC Improper setting of pressurization during preflight flow - interruptions - cabin altitude warning light in cruise Entered wrong weight into FMS - tail strike at takeoff Failed to program new departure - interruptions during preflight - discover after takeoff Omit requesting Load sheet - defer during preflight; ACARS inoperative - takeoff without load sheet Forgot to complete preflight flow - interruption - took off with APU running Forgot to request PDC - deferred when instructed to request by voice, not ACARS - discover after takeoff Omitted check of circuit breakers - busy with preflight; rushing - unable to start engine after pushback Checklist item read but not verified - interruption - pushback with emergency door slides armed Skipped checklist item - interruption - push back with Seat Belt signs off Read but not verify checklist item - distractions - pushback with throttles open, damage to aircraft PREFLIGHT errors attributed to “distractions”(incident reports) CAPTAIN FIRST OFFICER • Review paperwork • Sign flight release • Prepare/review charts • Review Load Schedule • Review FMC • Takeoff brief • Ask for checklist • Ask for checklist • Obtain ATIS • Obtain clearance • Review paperwork • Prepare/review charts • (Passenger count) • (Load Sheet) • Program FMC • Begin checklist • Checklist complete • Begin checklist • Checklist complete • CLEARANCE • ENGINE START & PUSHBACK

16. ERRORS • Wide range of outcomes • Large number of omissions not caught by later checklist • Consequences spill into all phases WHY? • Potential for accidents • $$$ time, fuel, ramp mishaps, taxiway/runway incursions, rejected takeoffs

17. Line representation (real)

18. Preflight - real (compiled observations) Conduct exterior walk-around no time, familiarity Ramp and/or Ground? Check charts busy frequency Keep trying Double-check charts no time, familiarity Interruption Interruption Resume flow Still refueling Check fuel quantity and pumps Inoperative item Inoperative item Call maintenance Confirm Mx responded Passenger count unavailable Confirm resolution Request passenger count Confirm Mx departed Data unavailable Confirm logbook on board Defer programming FMC Flight release still not picked up Look for ops/gate agent New PDC Delay at gate Re-program FMC New flight release/PDC? Re-set MCP Time pressure FO busy Re-flow trim & other settings Ask for checklist Takeoff brief Flight plan/ Departure runway change Interruption Resume checklist Communicate with company Compute new performance #s Re-program FMS Re-brief CAPTAIN FIRST OFFICER • Review paperwork • Sign flight release • Prepare/review charts • Review Load Schedule • Review FMC • Takeoff brief • Ask for checklist • Ask for checklist • Obtain ATIS • Obtain clearance • Review paperwork • Prepare/review charts • (Passenger count) • (Load Sheet) • Program FMC • Begin checklist • Checklist complete • Begin checklist • Checklist complete Ground/ Company/ Dispatch Frequencies Interphone Cabin Attendant procedure Gate Agent ACARs / OPC procedure procedure checklist • CLEARANCE checklist • ENGINE START & PUSHBACK

19. Dynamic Inter-dependency of tasks and operators Unpredictable interactions (nature and timing) ENVIRONMENT-driven operation PHASE OF FLIGHT - real z1 z2 z3 z4 z5 CAPTAIN FIRST OFFICER x y *monitor z A B C D 1 2 3 4 z *Wait • CLEARANCE • Activities are: • Linear: • Predictable: • Controllable: PILOT-driven

20. Results • Multiple interactions among operators and tasks (within, outside cockpit) • Pervasive and unnoticed: most often dealt with successfully • Training • does not adequately reflect reality • does not adequately prepare pilots for line flying (learn on the job) • Interactions are not well characterized • Effects (challenges) are underestimated • Vulnerability to errors are not appreciated • Neither is the potential for such errors to lead to accidents

21. So what are these Interactions? • Characterize nature of interaction • Characterize demand such interaction places on cognition

22. Interactions 1 • Pilots are constantly being interrupted • Interruptions demand attention (to note and assess) • and action (to resolve) • Observations from the jumpseat • The first officer monitored the Ground frequency throughout the preflight preparations. A call with departure information interrupted her as she was setting the pressurization panel (a prescribed step in the procedure). She continued “dialing” the destination altitude before interrupting the procedure to acknowledge the call from Ground. She then copied the memorized information on the load sheet before resuming the interrupted procedure.

23. Interactions 2 • Pilots must often perform >1 tasks concurrently • Some combinations are often practiced together • Others combinations are new & must be carried out on the spot • From the jumpseat… • The captain monitored the cockpit interphone while performing his preflight duties and expecting to be contacted by the push back crew. A flight attendant notified him of a potential issue with a passenger. He relied on the cabin crew to resolve it with the help of the Gate Agent and continued with the preflight procedure. At the same time, he continued to monitor the conversation taking place right outside the cockpit door, intending to intervene if necessary.

24. Interactions 3 • Pilots do not/cannot always initiate actions • Circumstances may not be right for an action • Information/tool/agent may not be available • From the jumpseat… • CA reached the point on his preflight procedure calling for verification of sufficient fuel on board. He looked around for a fuel slip but quickly realized that it had not yet been brought to the cockpit - that implied that refueling was still underway. He formed the intention to return to the fuel quantity action item later and proceeded on with the remaining procedural steps.

25. requires • forming an intention (acknowledge call, copy information) • maintaining the intention (while continuing primary activity) • remembering what/when suspended/deferred (call from Ground) • noticing when it is time to resume/start intended task (once finished with primary activity) The first officer monitored the Ground frequency throughout the preflight preparations. A call with departure information interrupted her as she was setting the pressurization panel (a prescribed step in the procedure). She continued “dialing” the destination altitude before interrupting the procedure to acknowledge the call from Ground. She then copied the memorized information on the load sheet before resuming the interrupted procedure. Challenge 1 • In order to respond to the multiple, complex, and unpredictable interactions, a pilot must … • Suspend/Defer activities • i.e., must resume the suspended task … or • start the deferred task at a later time

26. Challenge 2 • In order to respond to the multiple, complex, and unpredictable interactions, a pilot must … • Interleave activities • i.e., combine habitual or novel activities … • in previously-encountered or new ways • requires • shift attention between activities • remembering the activities • remembering to check on the progress of each activity • deciding when/how often to check progress

27. Research Implications • Memory • esp. Prospective Memory (for intentions) • Attention • continuous sampling of environmental stimuli monitoring (essentially a continuous PM demand) Pilots are vulnerable to errors under such circumstances because… • Events in the environment continue to flow in habitual sequence and with their usual timing • “Triggers” for sequence of activities are disrupted or even lost altogether

28. Practical Implications (carriers) • Acknowledge pilot fallibility • Examine procedures and checklists, incident (ASAP) reports • Blank-slate approach • source • nature • timing of challenges in everyday operations • Design robust procedures • warning signs (red flags) • triggers for items at risk (tie critical functions to fixed reference points) • traps (strict decision criteria not to cross unless all expected activities fulfilled) • Develop (better) guidance for Monitoring

29. Cognitive Performance in Aviation Training and Operations http://human-factors.arc.nasa.gov/flightcognition/

30. Backup slides from here on...

31. Preflight B737 Ops Manual FLIGHT ALTITUDE indicator --- cruise altitude LANDING ALTITUDE indicator --- destination field elevation Pressurization mode selector --- AUTO AUTOMATIC FAIL light --- Extinguished PANEL OFF light --- Extinguished Set panel --- As desired WARNING: Do not key HF radio while airplane is being fueled. Injury to personnel or fire may result

32. TAXI - ideal (manufacturer) • Goals: • bring aircraft to departure runway • configure it for takeoff (next phase of flight) • Considerations: • safe operation • within performance limitations and design specifications

33. Taxi B737 Ops Manual

34. Taxi - ideal (manufacturer) CAPTAIN FIRST OFFICER • Receive taxi clearance • Start taxiing • Ask for checklist • Receive takeoff clearance • Ask for checklist • Line up with runway • Receive taxi clearance • Start checklist • Checklist complete • Receive takeoff clearance • Start checklist • Checklist complete • TAKEOFF

35. CLEARANCE • Activities are: • Linear: task B always follows task A, in this sequence • Predictable: information available when needed • communication possible when necessary PHASE OF FLIGHT - ideal (manufacturer) CAPTAIN FIRST OFFICER A B C D 1 2 3 4 • Controllable: tasks initiated by pilot, independently

36. TAXI - assumed (carrier) • Goals: • bring aircraft to departure runway • configure it for takeoff (next phase of flight) • Considerations: • Safe operation • Within performance limitations and design specifications • Carrier (additional) considerations • Priorities (fuel, time, passenger comfort) • Type of operations (short/long-haul, turn-arounds, regions/airports, fleet) • Philosophy (safety, efficiency) • Technologies on-board (ACARS, laptop) • Coordination with agents (Ground, Tower, dispatch, other aircraft)

37. Taxi - assumed (carrier) CAPTAIN FIRST OFFICER • Start taxiing • Ask for checklist • Receive takeoff clearance • Ask for checklist • Line up with runway • Receive taxi clearance • Start checklist • Checklist complete • Receive takeoff clearance • Start checklist • Checklist complete MONITOR Ground Company/Dispatch MONITOR Ground Company TaxiClearance MONITOR Captain taxiing Takeoff Clearance • TAKEOFF

38. TAXI-OUT -real Captain’s ASRS Report # 425357 (edited) • “The aircraft we had for this flight had an inoperative fuel gauge, and an inoperative APU… DFW was in the midst of a departure rush. Since we had to do a crossbleed start, we did not do the After Start checklist immediately... • We got clearance to taxi and requested a spot to do the start. We stopped ... as instructed and completed the start. As we were finishing, ground control was giving instructions to aircraft taxiing mentioning us in reference. We heard this, completed the after start checklist, and told ground we were ready to taxi. • We were given instructions to hold short of taxiway WJ, give way to opposite direction traffic, then proceed N to taxiway Z to taxiway HY to taxiway Y across the bridge to taxiway J to taxiway EF for takeoff on runway 17R, follow an ACR aircraft. With all the above, taxiing to a new and unfamiliar runway (for this crew) in the dark, we didn't complete the before takeoff checklist. As we were cleared for takeoff and applied power, the takeoff warning horn sounded and we immediately realized that the flaps had not been extended for takeoff. • There were many distractions leading up to this incident which should have been warning signals. • There was a time pressure element, we were running late and knew we had an airplane change in a short ground time in Denver before our next flight. • The APU and the necessity for the crossbleed start precluded the normal flow of calling for flaps and the before takeoff checklist as we taxi from the gate. • I was distracted by calling for taxi, so ground control knew we were ready to move. • Immediately after that call we were given a complicated taxi route with hold short and follow instructions and we were concentrating on finding the taxiways in the dark…”

39. Omitted call for flaps -rushed to clear ramp/gate area for arriving aircraft -aborted takeoff Started taxi without clearance - trouble-shooting problem with engine start - nearly hit ground handler Started taxi without clearance – rushed by other aircraft waiting to pull into gate; radio congestion; marshaller’s headset inoperative – query by ground controller CA taxied without having fully understood instructions - busy looking at other aircraft on taxiway and ramp - ground controller issued warning Started taxi without clearance - crew discussing taxi instructions - struck pushback tug Incorrect trim setting - checklist interrupted after item had been read but not verified - aborted takeoff Failed to start engine #-2 - distracted while discussing special operations for destination; omitted checklists - delay takeoff Neglected to set flaps -preoccupied with new departure clearance and packs-off operation -aborted takeoff FO failed to monitor CA - busy checking and correcting calculations of load data - taxi past hold short line Omitted flaps - crew discussing problem with APU, delayed flaps due to snow - aborted takeoff Flaps incorrectly set, missed noticing during checklist - crew busy with fuel problem, runway changes, programming FMC - aborted takeoff FO failed to monitor CA – busy with flow; night taxi – taxi in wrong direction Omitted checking reason for bleed air indicator light-busy with delayed engine start and checklists - takeoff without troubleshooting Confuse own position on taxiway diagram - new terminal; studying NOTAMs; runway change – taxied into ditch Fail to confirm flap position - evaluating heavy rain showers; rushed to accept takeoff clearance - aborted takeoff FO failed to monitor CA -runway change; busy reprogramming FMC -taxied past intended taxiway FO failed to monitor CA - busy with pre-takeoff preparations - aircraft crossed hold short line Omitted flaps - checklist interrupted by thrust reverser light; crew busy troubleshooting - aborted takeoff Omitted checklist - busy with delayed engine start and checklists; rushed to accept takeoff clearance - flaps not set, aborted takeoff Misunderstood Tower instruction - new FO on IOE, CA coaching FO - taxi onto runway w/o clearance Flaps incorrectly set - late paperwork ; runway change; programming FMC; short taxi; rushed to accept takeoff clearance- aborted takeoff Omit above-line Checklist - running late, checklist interrupted by Tower, unexpected clearance for takeoff - abort takeoff Omitted flaps - checklist interrupted by Tower; crew rushed to accept takeoff clearance - aborted takeoff TAXI-OUT errors attributed to “distractions” (ASRS incidents) CAPTAIN FIRST OFFICER • Start taxiing • Ask for checklist • Receive takeoff clearance • Ask for checklist • Line up with runway • Receive taxi clearance • Start checklist • Checklist complete • Receive takeoff clearance • Start checklist • Checklist complete • TAKEOFF

40. Taxi-out - real (compiled observations) Ice/Snow Defer takeoff flaps Ramp and/or Ground? Check charts Keep trying Unfamiliarity with airport Consult charts busy frequency Consult charts “Clear” ramp area Form mental picture of route Unfamiliarity with airport Acknowledge clearance Ice/Snow Busy airport/ departure rush Form mental picture of taxi route De-icing Checklist Confirm CA’s understanding MONITOR airport traffic MONITOR aircraft position on chart APU? Packs? Extended taxi delay Restart engine Delayed engine start Just-in or new load data traffic, FO busy Repeat checklists Before/After StartChecklist Defer checklist Calculate & reset Performance data New/ Additional taxi instructions FMC: program/verify Remember taxi instructions Id taxiways and turns Keep head up/ outside Interruption Inform Company (new #s, delays) Resume checklist Remember to follow aircraft Identify aircraft to follow Cross check with CA Remember to hold short Id correct place to hold short busy frequency MONITOR Tower frequency APU off for 2 min before off-loading Keep trying Change in takeoff runway Accept new runway? Acknowledge clearance Consult charts Confirm CA’s understanding Brief new runway Strobes FMC update Change in takeoff sequence Shoulder harnesses FO’s leg Landing lights Repeat Checklist Take control of aircraft while finishing checklist Radar? “Clear” runway CAPTAIN FIRST OFFICER • Start taxiing • Ask for checklist • Receive takeoff clearance • Ask for checklist • Line up with runway • Receive taxi clearance • Start checklist • Checklist complete • Receive takeoff clearance • Start checklist • Checklist complete MONITOR Ground//Dispatch MONITOR Ground/Company Taxi Clearance MONITOR Captain taxiing Takeoff Clearance • TAKEOFF

41. components nature (constraints) tools (solutions)   cognitive abilities flows checklists phase of flight structure timing   number procedures (SOP) (design - $$$ - technology)   (mfgr/engineers) interactions ?  ?  MOTIVATION: study complex operating environment as an entity • operator(s) • external agents • tasks • hardware/software • Characterize nature of interactions • Characterize their effects on the (routine flight) operation • Develop or help guide development of tools/solutions to mitigate errors