Effective Team Support for Time Critical Complex Work Environments

1. Effective Team Support forTime Critical ComplexWork Environments PIs:Bonnie E.John, Katia Sycara Key personnel:Sandy Esch, Joseph Giampapa, Rahul Singh

2. Goals of the project • Advance understanding of team performance • Provide a simulation environment to cognitive modeling / teamwork research community • Executables and source code • Operated by humans and computational cognitive models • Scenario editor • Data logging • Documentation • Data of team performance • Build cognitive models of team members • Build software agents to support team members & the team • Evaluate effectiveness of approach and resulting systems

3. Outline of the talk • Domain of interest • Simulation environment overview • Teamwork characteristics of the simulation environment • Factors effecting performance of teams • How these factors can be manipulated in the simulation environment • Dependent measures of team performance • Project status and next steps

4. Range operations • Safe and efficient operation of the launch range during launch • Several distributed range operators must collaborate to achieve successful launch within the launch window or abort the mission • Responsible for monitoring a particular area in the launch zone • Negotiate with other range operators for resource allocation • Some tasks include… • Maintaining clear airspace over the entire range during launch • Predicting plume dispersion • Efficient allocation of resources to minimize risk and cost while maximizing safety • Typically a team task with distributed responsibility • Typically a time critical task • Information-rich and communications-intensive

5. Simulation environment forcognitive modeling / teamwork research:The MORSE task

6. MORSE Architecture MORSE-Command Scenario File • Flow of the task is controlled by the MORSE Command • MORSE Command models entire mission and simulation world • MORSE Stations display focused subset of simulation world to each user • MORSE Command maintains accounting of costs incurred, coordination penalties Incursion Information Weather Queries Timing Synchronization MORSE-Station AlphaCape Canaveral MORSE-Station CharlieAscension Islands Shared Information between Stations MORSE-Station BravoAntigua

7. Team performance(Galbraith, 1977; Kraut, Fussell, Lerch, & Espinosa, 2002) • The more uncertainty in a task, the more information-processing necessary to achieve a given level of performance (Galbraith, Organization Design,1977) • The more coordination within a team, the higher level of performance achieved (Kraut, et. al., 2002) • Thus, to increase performance on a given task, take measures to • decrease uncertainty • make information-processing less costly • improve coordination • Such measures may include • Designing the task environment, including software agents • Designing the organization

8. Team performance(Galbraith, 1977; Kraut, Fussell, Lerch, & Espinosa, 2002) • The more uncertainty in a task, the more information-processing necessary to achieve a given level of performance (Galbraith, Organization Design,1977) • The more coordination within a team, the higher level of performance achieved (Kraut, et. al., 2002) • Thus, to increase performance on a given task, take measures to • decrease uncertainty • make information-processing less costly • improve coordination • Such measures may include • Designing the task environment, including software agents • Designing the organization

9. Determinants of informationand task uncertainty(adapted from Figure 3.1 Galbraith, 1977) Determined by goal diversity,level of goal performance, division of labor Uncertainty or information to be acquired and processed Amount of information required for task performance Amount of information possessed by organization

10. Factors affecting uncertainty (Galbraith 1977) instantiated in MORSE • Diversity of goals • Safe launch • No incursions or interceptors within Impact Lines • Toxic plume would not disperse to populated area • Level of goal performance • Time constraints • Limited resources: interceptors, weather balloons, participant attention & actions • Cost metrics • Division of labor • Small team, organically coordinated, or trained • Territory covered by each team member • Radar placement

11. Factors affecting uncertainty (Galbraith 1977)manipulable in MORSE • Diversity of goals • Safe launch • No incursions or interceptors within Impact Lines • Toxic plume would not disperse to populated area • Level of goal performance • Time constraints • Limited resources: interceptors, weather balloons, participant attention & actions • Cost metrics • Division of labor • Small team, organically coordinated, or trained • Territory covered by each team member • Radar placement

12. Mechanisms to economize oninformation processing capacity (Galbraith 1977) instantiated in MORSE • Rules, programs or procedures - decisions in advance of their execution • Could arise organically, or could be trained • Discretion guided by planning or professionalism • Could plan via chat (or talking outside the experiment) • Could involve activation of radar, activation/location of interceptors • Adjusting the hierarchy of authority (Narrowing span of control) • Could arise organically, or could be trained • Could assign roles, e.g., one person monitor weather, others handle incursions

13. Mechanisms to economize oninformation processing capacity (Galbraith 1977) manipulable in MORSE • Rules, programs or procedures - decisions in advance of their execution • Could arise organically, or could be trained • Discretion guided by planning or professionalism • Could plan via chat (or talking outside the experiment) • Could involve activation of radar, activation/location of interceptors • Adjusting the hierarchy of authority (Narrowing span of control) • Could arise organically, or could be trained • Could assign roles, e.g., one person monitor weather, others handle incursions

14. Reduce the need for information processing (Galbraith 1977 )manipulable in MORSE • Environmental management • Provide software agents that, in effect, change the environment • Creation of slack resources • Evaluation function, e.g., tolerate some incursions or interceptors left within the Impact Lines, change the cost of deploying weather balloons or sharing data, etc. • Creation of self-contained tasks • Could arise organically, or could be trained • E.g., handle only own radar areas, no communication

15. Increase capacity to process information (Galbraith 1977)manipulable in MORSE • Invest in vertical information systems • Provide software agents to collect and process information for the team • Creation of lateral relations • Small team, free to communicate; this mechanism does not apply to this task “The organization must adopt at least one of the five strategies when faced with greater uncertainty. If it does not consciously chose one of the five, then slack, reduced performance standards will automatically happen.” (p. 55, Galbraith 1977)

16. Coordination in teams(Kraut et. al. 2002) • Factors contributing to coordination • Communication frequency • Communication evenness • Common history • Shared cognition • Factors are beneficial individually, compensate for each other • The Morse environment can measure these factors more directly than Kraut, et. al. were able to in their study of a simulated management game

17. Objective Measures in MORSE • Communication frequency • Clicks to ask for resources • Chat messages • Communication eveness • Who asked whom for what • Who chatted with whom about what • Common history • Time on task

18. Shared cognition • Kraut, et. al., investigated shared cognition about expertise of team members • Skills will not differ between team members in MORSE • Software agents will have specialized expertise that would complement what the people are doing • Other types of shared cognition to investigate • Weather (what is it doing and what does everyone else know about it?) • Plume dispersion • Incursions on the way • Availability of other participants (how busy are they) • Measuring shared cognition in MORSE • Questionnaires after a trial (like Kraut et. al. did) • Think-aloud protocols during a trial

19. Coordination in the MORSE task • Mechanistic coordination vs. organic coordination • Mechanistic: align individual actions through structure o directive • Organic: individuals exchange information about their current states and adjust their behavior to other’s goals and actions • Could have either, or both, in MORSE • Benefits of coordination • Allocate limited or costly resources more efficiently • Improve performance (according to Kraut, et. al.) • Costs of coordination • It takes time away from other task activities

20. Direct measures of coordination in MORSE • Failures, e.g., • Two participants assign interceptors for same incursion • Two participants deploy weather balloons to same location • Denial of resource request • may be failure from one participant’s perspective • may not be a failure for the team • Using a costly interceptor when coordinating would have been less costly • Chat to plan, then plan ignored • Launch safety disagreement • Successes, e.g., • Clicks and acceptances to share/release resources • Chat to plan, then plans executed

21. Measures of performance • Both team and individual • Evaluation function • Launch safety miscalculation • Incursions or interceptors left with Impact Lines • Radar costs • Interceptor costs • Weather balloon costs

22. MORSE status • Interface • Colors accommodate color-blind population • Icons encode speed, direction, ownership • Less cluttered screen • More natural ways to provide information • Window management • Chat • Evaluation function • Drafted, not yet implemented • Far fewer bugs!

23. Next steps • Building MORSE • Redesign command sequences for less clicking • Usability of scenario editor • Usability of data logging • Documentation • Collect human performance data • Develop cognitive models • Develop software agents • Study their effectiveness

24. Demo! • In the downstairs lobby (right outside the doors)at the break and after the sessions