Day 2 Prior to today

1. Day 2Prior to today • Participated in the one day class on Cognitive Ergonomics, read all material of that class • studied the reader for today • read the paper by vdVeer & Mariani

2. After today • management plan and time schedule • interviews with potential users • video records for interaction analysis • review relevant information from web and other document sources • version 1 of Task Model 1 • collect ideas for task model 2 prepare presentations for next time: • management plan • task analysis methods and techniques used: why, how, issues • task model 1

3. 5. Task analysis of complex systemsOverview 5.1. Task analysis in the process of designing complex systems 5.2. Describing current work situations: 5.2.1. Analytic methods 5.2.2. Understanding implicit expert knowledge 5.2.3. Explicit group knowledge (video illustration- Boeing 747-400 accident) 5.2.4. Understanding implicit group knowledge (video ill. Examination system) 5.3. GTA 5.4. Envisioning the future system 5.5. Evaluating task model 2

4. 5.1. Task analysis in the process of designing complex systems Design of complex systems for users: the virtual system and the mental model Design for the mental model: 3 types of design activities • analysis • specification • evaluation “Task analysis”, in modern approaches, indicates analysis of the whole task domain: business, work, users, stakeholders, situation “Specification” focuses on details of technology

5. work organization/practice ethnography Task Model 1 validity analysis Documents/artifacts users’knowledge/behavior/needs psychologicalknowledgeacquisition/ hermeneutics problemanalysis/specification specification/negotiation usabilitymeasuring Task Model 2 Client constraints/opportunities specification feedback early evaluation Technology Functionality Scenario Simulation Dialog early evaluation maintainingconsistency UVM Prototype Representation As soon as the system is implemented Implementation

6. Task analysis • describing the current situation • task model 1 • envisioning the future • task model 2

7. Task analysis in literature Several meanings • collecting knowledge about, and describing, current task domain (task model 1) • specifying new task situation, envisioning use of novel technology (task model 2) • specifying details of technology (existing or new) (the user virtual machine) approaches always consider type of task knowledge, and relate this to model intended for design process Literature and approaches often consider only one or two, and often mix up meaning of what is being modeled

8. Important concepts and distinctions task and goal: • each task (activity) has a goal (state to reach) • a goal may be reached through several different tasks primary and secondary tasks • a primary task is generic for the task domain irrespective of technology and situation • a secondary task is related to the use of tools and technology basic and unit tasks • a users’ unit task is an atomic primary task • a basic task is the simplest task available for delegation to technology actions • an action has no goal, its meaning is derived from the task it is part of

9. 5.2. Describing current work situations: task model 1Brigitte Jordan (1996): sources of task knowledge Explicit expert knowledge • facts of task domain • heuristics • skills Explicit group knowledge • process models, rules • instructions • historic documents • anecdotes Implicit expert knowledge • “expertise” • intuition • common sense Implicit group knowledge • “culture” • COP (community of practise) • work practice

10. 5.2.1. Analytic methods: conceptual framework • set of basic concepts (task, goal, condition, action) • relations between concepts formal representations, based on conceptual framework • task hierarchies (HTA, based on Taylor) • task-action grammars (TAG, ETAG), • goals and operation rules (GOMS, CCT) planned process • steps in collecting knowledge, modeling, specification focus on the individual human and his knowledge • cognitive psychology

11. TKS (task knowledge structure) P. Johnson & H. Johnson (QMW college, London) conceptual framework • tasks, plans (task structures), strategies • procedures (object - action relations) • roles (sets of tasks assigned to certain individuals) Psychological basis is conceptual knowledge in long term memory • psychological techniques: interview, observation, card sorting • document analysis

12. TKS (continued) method: • from individual TKSs to generic task model GTM (task model 1) • from GTM to Specific task model STM (task model 2) • from STM to spec. interface model SIM (functionality at conceptual level) problems: • experts often do not know details of others’ roles • often there is no overview of all roles in an organisation • objects (things) are only analysed at secondary level

13. ATOM - Paul Walsh (STC Technology, Harlow, UK) Start of analysis is identification of “principal objects” • active objects (agents) • passive objects (things) • environment (includes things, agents, technology objects structured in class hierarchies and relations between types actions establish a causal link between object states actions are identified secondary, identified by type of verb: • process verbs (d.g. “dies”) • state verb (“waits”) • action verb (“transports to”)

14. MAD - D.Scapin, S. Sebillotte (INRIA, France) Conceptual framework • tasks, goals of people - states to be reached • functions of systems (delegated subtasks) • task hierarchy, subtasks, elementary tasks • constructors (PAR, ALT, PAR, LOOP, COND, OPT) • initial state, final state (describe change due to effect of task) • prior condition, post condition Formalism • task / goal tree (HTA)

15. MAD (continued) The Graesser method: • describe as complete and detailed as possible how you …. (name the global label of the task to be described) “... because ...” “... Therefore ...” • keep track of hierarchical structure (need a second analyst) • filling the gaps “How ?” “Why ?”

16. MAD (continued) Generic levels of task knowledge • abstract general task formulation • specific context sub-tasks (expert task language) • domain independent procedures: (“store a copy”, “send a message”) • elementary actions (“ask for signature”, “put date on record”)

17. 5.2.2. Understanding implicit expert knowledge Implicit knowledge, behavior, skills, “expertise” • individual knowledge that can not be verbalized Technique: • observation, registration Hermeneutic interpretation • “what would be my knowledge and mental process, if I would show the behavior that this expert is showing” Reliability problem • needs training and multiple interpreters

18. Explicit sources: official documented rules documented business plans documented laws, regulations memos in-official artifacts (incl. documents) stories, anecdotes (location dependent) Validity: example: rules, regulations, laws intention date of installation friction with novel developments "officially still valid" overruled 5.2.3. Explicit group knowledge

19. Example: analysing history documents(Video “Kegworth fatal error” - training video HRA) An example: the Kegworth accident Failure of a safety critical system: a Boeing 737-400 crash at East Midlands Hardware failure: left motor flutter • up-scaling problem • no operational test Video document shows original material, and actual actors, but cockpit scenes are played by actors, based on cockpit voice recorder (another historic document)

20. Analysing history documents (continued) A combination of problems merged after the original hardware failure: • functionality of the control system • representation of the system towards the users • training and information • organisation problems • human failure watch the video and try to identify problems in this task domain

21. /ONDERWIJS/PAO-NFatalErrorCDR.mov

23. 5.2.4. Understanding implicit group knowledge Ethnographic methods No a priory conceptual framework • bootstrapping - hanging around and identifying relevant concepts • informal representations - diary and journalist type records • activity theory, anthropology, sociology Goal: understanding complex settings • multiple activities, multiple actors, multiple agendas • in natural groups Focus on the situation, on work practice • knowledge and meaning are developed in “communities of practice” • knowledge is “ability to participate meaningfully” • people co-construct knowledge and skills, partly implicit or “tacit” • situated tasks, situated actions, situated understanding

24. Ethnographic methods (continued) Ethnography is fieldwork ethnographer is participant observer • part of the situation, and co-constructor of events • novice, if possible apprentice, or “legitimate peripheral participant” • accepted to be “nosy” the two viewpoints of ethnographic analysis • the view of the analyst: being surprised, recording novel phenomena • the view of the aboriginal: understanding in situation and the relevance of resources and representations • diary needed from day one

25. Ethnographic methods (continued) Ethnographic methods • hanging around - identifying basic situational phenomena, getting accepted • informal questions, triggered by activities and situation, based on growing understanding • observation of routine and expert behaviour • interaction analysis

26. Interaction analysis Focus on interactions (humans, artifacts, situation) • identify "hot spots” • documents, stories, participant observation • collect registrations - multiple instances • content log • participants, situation, artifacts, technology, time • verbal and non verbal (body) language, territory, space, messages Video review sessions: • comments of original participants • explanations, thinking aloud • ask to identify important issues • propose hypotheses

27. Interaction analysis (continued) Interpretation: • patterns in interaction, resources • verifiable observations: records of different instances • generalization: theory • supplementary interviews, archive, observations Group sessions: • interdisciplinary • replays to generate hypotheses • max 5 minutes for discussion • identify analytic categories • confront hypotheses with other instances of hot spot theory

28. Generalizing ethnographic knowledge Surprise index • events and phenomena should be as predictable for the ethnographer as they are for the original actors Systematic coverage: • space, territory, off location • make sure all relevant locations are considered (coffee lounge, corridors) • temporal (shifts, seasonal changes, off hours) • related events during commuting, during lunch hour • actors (supporting people, spouses, unofficial consultants)

29. Generalizing ethnographic knowledge (continued) Orientation - systematic viewpoints: choose one or more • person oriented record • daily activities, interactions • object oriented record • movements, change of ownership • setting oriented record • at location, during shifts, movements in the space • task / process oriented record • focus on process

30. Generalizing ethnographic knowledge (continued) • event structure • preparation, start, re-arrangements, intro of new objects • temporal patterns • day, season, technology driven rhythm • “acted” continuity (receptionist, pilot, underground train “driver”) • interaction structure • verbal, instrumental, body language • interrupted conversation (continuation triggered by availability of info or resources or time) • just in time finalization triggered by interruption

31. Generalizing ethnographic knowledge (continued) • spatial organization • meaning of positions (head of the table) • fixed places, free choice, stable or re-arrangements • artifacts and documents • location and movements • construction and revision (documents) • ownership (allowed activities related to objects) • symbolic meaning (dress code, stethoscope) • non physical objects (PIN, password, anecdotes, jokes) ownership, trajectory, availability and inspectability

32. Example ethnographic record(video “Math Examination” VU 1994) Task analysis for the design of an automated examination system (preparing questionnaires, registering for exam, taking the exam, scoring results, administering marks) An examination: setting oriented record • roles: professor, supervisors, students • objects: forms, boxes, writing equipment, calculating equipment • situation: time structure, entrance, scenes (hot spots) watch video and identify and analyse unexpected events

34. 5.3. GroupWare task analysis (GTA) Conceptual framework for collecting and modeling task knowledgehttp://www.cs.vu.nl/~gerrit/gta agents (with description of relevant attributes) • actors (individual humans, institutes and groups, machines, systems) • roles and their representation (incl. symbols) • organization (structure of actors, roles, and allocation rules) work • task / goal structure / actions (acts, communication acts) • decomposition and temporal structure • protocols and (expert) strategies - partly situated situation • objects in object structure (type relations and semantic relations) • environment • history

35. GTA representations conceptual entities • template structures • tree • flow media registrations • video • sound track • picture

36. name(string) goal(string) name(string) attribute(Name,Value)* action(Name)* Used_by Subrole Object Contains Role Is Responsible Uses Event Triggers Plays Performed_by name(string) description(string) Performed_by Agent Task name(string) skills(string) attitude(string) miscellaneous(string) name(string) goal(string) start_condition(string) stop_condition(string) initial_state(string) final_state(string) duration(integer) frequency(string) type(enum) user_actions(string) system_operations(string) Subtask Triggers Task World Ontology (Martijn van Welie)

37. EUTERPE GTA representations