1. Simulation-based tutoring systems CPI 494/598, April 16 Kurt VanLehn

2. Classification of step-based tutors User interface • Forms e.g., CTAT • Canvas e.g., Andes • Text e.g., Criterion • Dialogue e.g., AutoTutor • Simulation e.g., ??? Tutoring on • Process • During e.g., Cog Tutors • After e.g., ??? • Both e.g., Andes • Product e.g., Criterion • Both e.g., AutotTutor

3. Prototypes (ill-defined classes) of simulation-based tutoring systems • Equipment operation & maintenance • E.g., nuclear reactor operation • Vehicle driving • E.g., flight simulator • Tactical decision making • E.g., defend ship from possible aerial attack • Negotiation, leadership, other soft skills • E.g., convince an NGO’s clinic to move • Lab/field experimentation • E.g., discover cause of an epidemic

4. What’s different between simulation-based interface vs. others? • Time matters in the sim • Heavy reliance on delayed feedback • Game-based expectations of sim quality • Game-based technology e.g., game engines • Just making the sim work – huge! • Game-like expense e.g., $1M each! • Most research done outside universities • Mostly military

5. Steamer – Very early example(Hollan, Hutchins, Weitzman 1984) • Steam plant occupies 1/3 of space on Navy ships; miles of pipe; • Engineer of the watch supervises 16 to 25 people who run it • Years of experience required

6. Operator’s screen:conceptual, not physical, fidelity of controls

7. One actual operator’s consolephysical & conceptual fidelity

8. First, build the simulation • Underlying mathematical model – huge! • Built before Steamer • Objects for each gauge & control authored • Visual presentation • Hooked up to underlying math model • Augment math model: models of faults/failures • Math may not be perfect • Especially in abnormal operating conditions or failures • Validate the sim • check that standard operating procedures work • Compare real to sim behaviors; many conditions

9. Sim is not an ITS but… • Can be used for training anyway • Drill to memorize standard op procedures • Exercise in handling standard breakdowns • Instructor(s) supervises trainees • Decides on exercise • Sets sim into initial state • Watches trainees • Gives immediate feedback occasionally • Gives delayed feedback: After Action Review (AAR)

10. Problem: After training, still can’t handle non-routine situations • Need a “mental model” of the whole steam plan • Helps to diagnose what is causing trouble • Helps to hypothesize & test (mentally) repairs • This sim is a black box; need a “glass box” • Shows multiple levels of detail • Should it to be active; can manipulate & watch • Animation; real time?

11. Top level view of the whole steam plant

12. One of about 100 subsystem diagrams

13. Controlling sim via the schematics • Can click on schematics to e.g., open a valve • Can do “abstract” actions that are hard or impossible to do in real world • Replace boiler • Raise chill-water temp to 100 degrees

14. Problem: Can’t understand time courses • Neg feedback loops in steam plant control e.g., fuel flow to boiler • Need to see first and second derivatives as the sim is running • Add new “gauges” to the sim • Some were later added to the real steam plant • Need to see graphs

15. New gauges at bottom; graph at top

16. Was Steamer ever an ITS? • Only for safety issues • Wired to give student immediate negative feedback if certain safety conditions violated • E.g., starting a pump without first opening the input and output values • Doesn’t know what student is attempting to do • Can’t offer immediate or delayed feedback • Can’t offer hints • Even if it knew goals, can’t connect goals to actions • Has no representation of knowledge-to-be-learned • Except mental model of the plant itself, which is huge

17. RIDES(Munro Johnson PizziniSurmon Towne Wogulis, 1997) • 15 years later • Like Steamer, has authoring system for widgets • Has underlying language for implementing Sim • Integrates widgets & sim

18. Has panels with widgetssome are controls, some are gauges

19. Has better tools for authoring instruction

20. Instructional design procedure • Specify objectives; specify scenes • Develop sim • Graphical objects • Behavior rules for each • Debug sim • Develop lessons • Author procedures using the sim • Add texts • Debug lessons

21. Object palette: Note how mouse controls rotation

22. Authoring a procedure • Put sim in an initial state; name the procedure • Do it • End it

23. Put sim in an initial state

24. Half done with the procedure

25. Three modes for instructionProvided automatically • Demo mode • Tutor executes procedure • Student controls pace • Practice model • Student tries to execute • Gets immediate feedback before sim reacts • If authored hints exists, gets them • Test mode • No feedback & no hints • Sim does whatever

26. Practice mode

27. Curriculum editor

28. Is Rides an ITS Authoring tool? • Procedural training = CTAT = ITS • Drill on identifying parts = tutor, but not so intelligent?