Continuous Planning and Execution

1. SRI International Continuous Planning and Execution Dr. Karen Myers (PI) Mr. David Blei Mr. Thomas J. Lee Dr. Charlie Ortiz Dr. David E. Wilkins Artificial Intelligence Center SRI International 333 Ravenswood Avenue Menlo Park, CA 94025 Project URL http://www.ai.sri.com/~cpef/

2. Domain Characteristics • Tasks are complex and open-ended • Operating environments are dynamic and possibly hostile • Complete and accurate knowledge of the world can never be attained • Full automation is neither possible nor desirable • Successful operation requires a mix of • user involvement and control • continuous planning • rapid response to unexpected events • dynamic adaptation of activities

3. CPEF Foundations • leverage several mature SRI technologies • Procedural Reasoning System (PRS) • Knowledge-based Reactive Control system • SIPE-2: Hierarchical Task Network (HTN) planner • Advisable Planner (AP) • Multiagent Planning Architecture (MPA) • Functional integration: more than just interfaces ...

4. Technical Thrusts A. Flexible Process Management • provide intelligent management of planning and execution that is responsive to the dynamics of the operating environment B. Dynamic Plan Adaptation • provide situation monitoring, execution monitoring, and plan repair that enable reactive, timely adaptation of plans C. Robust Plans • generate plans that are sensitive to the execution environment, knowledge limitations D. User Guidance • enable users to direct and manage key aspects of the planning and execution processes

5. CPEF Architecture MPA Messages

6. Infrastructure: Multiagent Planning Architecture (MPA) • Builds on components from the Multiagent Planning Architecture (MPA) • Agent-based framework for addressing large-scale planning problems • distributed operations • modularity via plug-and-play paradigm • Protocols related to plans and planning activities • layered on top of KQML • Plan Server for storage/retrieval of plans and plan fragments • Extensions to MPA for plan execution • class of Executor agents • protocols for plan execution, repair, updates, advice

7. Plan Manager: Responsibilities Execution Tracking • Supervise progress through execution of plans Knowledge Management • Situation monitoring • Perform information-gathering tasks Process Management • Control generation of plans and options for outstanding tasks • Provide timely response to user requests, unexpected events • Reactive response to unexpected events EX: downed pilot in Area of Operations • Runtime adaptation of plans in response to failures, events EX: pop-up targets, change in weather

8. Plan Manager Design • PRS reactive control technology • Multi-threaded, highly responsive • Mixture of goal-directed and event-directed activity • Execution tracking via Flow Model • Await outcomes in accordance with sequencing info in plans Outcomes: success, failure, unknown, time-out • More opportunistic models would be preferable

9. PRS Reactive Control Architecture User Plan: partial order of goals Controller: manages procedure application in accordance with Plan and New Events Monitors: checking for critical events Database: dynamically maintained knowledge of the real world Procedures: • goal refinement • reactive responses Procedures Plan Controller Database Monitors World

10. Monitors Event change in the world state • action completes, new info, time passes Response several possibilities: 1. Invoke standard operating procedures 2. Perform plan transformations (e.g., plan repair, plan extension) 3. Record changed world model Monitor Classes Failure Monitors respond to failures that occur during plan execution Knowledge Monitors test for availability of info needed for decision-making Assumption Monitors respond to situation changes that violate key plan assumptions

11. Automated Monitor Generation • Create assumption monitors with a range of possible responses from formal plan representation • alerts, plan repairs, standard operating procedures • Traversal of causal links within plan derivation structure to collect conditions/assumptions that are: • Dynamic • Not established by earlier actions in the plans (ie, in initial world) • Declared as significant • Certain violations can be disregarded until entry into a critical time window (Ex: weather) PLAN = Actions + Monitors

12. Plan Repair • Perform minimal-perturbation replanning for impacted portions of current plan • wedge beneath ‘failure nodes’ • minimal changes provide plan continuity, understandability • can be computationally expensive Planning-time • Adaptation of plan in response to information updates Asynchronous Execution-time • Adaptation of active plan during execution • world continues to change, unaffected actions are executed • Plan Manager must synchronize new plan with continued progress along previous plan

13. Strategy-to-Task refinement of selected Air objectives Final plans at the level of targets, CAPS (with several thousand nodes) Key Components strategic knowledge for objective refinement hierarchical target network models threat models geographic knowledge force and equipment knowledge Assumes key intel info: COGs, threats Air Objectives Planner Targets CTEM Support Packages Planner Strikes ACP Knowledge Bases Scope of TIE-97-1 Air Campaign Planning (ACP) Knowledge Base

15. CPEF Demo: Technical Highlights • Rapid generation of alternative courses of action • Strategy-to-task refinement of Air Superiority objectives • Incremental generation of qualitatively different options by user (via advice) • Application of automatically- and user-created monitors • Realtime Execution Tracking in a simulated environment • Asynchronous adaptation of activity in response to realtime monitoring of • situation changes (Ex: Downed Pilot) • plan execution results (Ex: failure to neutralize critical targets) • Advised Plan Repair

16. Process Management Plans Air Campaigns Conclusions • CPEF Prototype demonstrates flexible process management for living plans in Air Campaigns • Full Spectrum: plan generation, execution, repair • Several Notable Technical Accomplishments • Models for Plan Management, Execution Tracking • Generalized Failure Models, Repair Methods • Promising preliminary work on Open-ended Planning • Major Contributor within the JFACC Program CPEF

17. Backup Slides

18. Execution Models Direct Execution (Do it!) • actions are dispatched directly by the system • EX: controller for a mobile robot Indirect Execution (Supervisory) • plan is executed by diverse, distributed agents • agents are pre-assigned execution tasks • status of action execution is not directly available • delays in redirecting agents that perform planned actions • time lag on receipt of information about the world

19. Generalized Failure Models • Limited scope of current models: Precondition Failure action precondition not satisfied Action Failure intended effects not achieve Maintenance Failure established condition no longer maintained • New directions --- beyond plan dependency structures Unattributable Failure no individual action has failed or assumption violated yet plan is deemed inadequate • Ex: CA indicates failure to establish sufficiently strong breach of IADS Aggregate Failure require collections of failures, possibly with key relationships among them (eg, A fails then B fails) • Ex: key subset of a target network

20. New Facts & Goals 2 (overpressurized fuel-tank) 7 (ACHIEVE (position ox-valve closed)) Act Library 1 ACT2 Cue: Act Execution (TEST (overpressurized tank.1)) Facts & Goals ACT1 Cue: (ACHIEVE (position valve.1 closed)) 6 External World 8 5 Goal2 Goal3 ACT8 ACT3 3 sleeping sleeping (ACHIEVE (position ox-valve closed)) ACT1 current Fact1 ACT2 4 normal Intention Graph PRS Control Loop Execution Cycle 1. New information arrives that updates facts and goals 2. Acts are triggered by new facts or goals 3. A triggered Act is intended 4. An intended Act is selected 5. That intention is activated 6. An action is performed 7. New facts or goals are posted 8. Intentions are updated

21. Process Management Living Plans Air Campaigns CPEF: Continuous Planning and Execution Framework • process management technology for living plans • plan creation, execution, repair • vertical slice of the JFACC system JFACC System Layered View of CPEF Workflow Management C P E F Plan Gen Specialized Components

22. Process Management: Generality and Ubiquity Process Management Process Management Process Management Info Needs Agents Plans Air Campaigns ISR SWIM (AIM) CPEF (JFACC, SUO) TRAC (CoABS)

23. Accomplishments: Technical • Process Management for plan generation, indirect execution, monitoring, repair • Automated extraction of monitors from plans • Generalized models of failure and execution monitoring • Mixed-initiative options generation and plan repair (using advice) • Preliminary models for open-ended planning • “Towards a Framework for Continuous Planning and Execution”, AAAI 1998 Fall Symposium on Distributed, Continual Planning (Special Issue of AI Magazine)