The Hybrid Deliberative/Reactive Paradigm

1. The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu Xuezhou Ma Junjun Li

2. Describe the hybrid paradigm in terms of SAP and sensing organization. Distinguish the responsibilities between the deliberative layer and reactive layer. List the basic components of a Hybrid architecture: sequencer agent, resource manager, cartographer, mission planner, performance monitoring and problem solving agent. Identify the difference between managerial, state hierarchy and model-oriented styles of Hybrid architectures. Be able to describe the use of state to define behaviors and deliberative responsibilities in state hierarchy styles of Hybrid architectures. Chapter Objectives

3. However, the robot could not… Remember the state of the robot/world Plan optimal trajectories Make maps Monitor its own performance Select the best behaviors for a task Reactivity more art then science? Should planning be reintroduced? Overview • Reactive Paradigm is the major trend by the end of the 1980’s.

4. Not all of these activities involve Planning: Make maps Monitor its own performance Select the best behaviors for a task To differentiate this from path planning, the term deliberative was coined. Deliberative Vs. Planning

5. How can slow planning be intergraded with fast reactivity? Five examples of architectures will be illustrated: AuRA, SFX, 3T, Saphira and TCA. First Opinion: The worst of both worlds! Reactive systems for unstructured worlds Hierarchical systems for knowledge-rich worlds Nowadays: The best of both worlds! Reactive functions forlow level control Deliberation for higher level tasks Hybrids

6. PLAN Reactor Deliberator ACT SENSE Hybrid Paradigm Organization: Plan, Sense-Act:

7. Cohesion (object oriented programming) Reactivity: Short time horizon (Present) No global knowledge Work with sensors and actuators Deliberation: Long time horizon (Pass, Future) Global knowledge Work with symbols Multi-tasking Deliberative functions execute in parallel with reactive functions. Motivation of Hybrids

8. World Map/ Knowledge Rep Eavesdrop Virtual sensor Feedback Behavior Behavior Behavior control only Behavior Planning only Sensor 1 Sensor 3 Sensor 2 Sensing Organization The Map (World Model) • Can have its own sensors • Can “eavesdrop” on other sensors • Can act as “virtual” sensor

9. Not purely reflexive: Reflexive (response to stimulus) Innate (virtual sensor turns behavior on or off) “If power is low, charge” Learned Retain feedback to determine best behavior sequence to instantiate next time More complex emergent behaviors: Behavior sequences Skill Vs. Behaviors

10. “Global” isn’t always truly global in Hybrids. Behavioral Management Planning which behaviors to use requires knowledge about current and future world state Performance monitoring Detecting task progress and sensor confliction require knowledge about the robot hardware and the overall goals. Connotations of Global Nonetheless "Globle" - "Deliberative" "Local" - "Reactive"

11. Sequencer Generates a sequence of behaviors Resource Manager Allocates resources to behaviors Cartographer Creates, stores, maintains, accesses map information Mission Planner Interact with human and create a plan to achieve a goal Performance Monitor/problem solver Determines whether the robot is making progress toward its goal Common Components

12. Managerial (division of responsibility as in business) AuRA SFX State Hierarchies (strictly by time scope) 3T Model-Oriented (Model serve as virtual sensors) Saphira TCA Architecture Styles

13. Styles of hybrid architectures● Managerial styles● State hierarchies styles● Model-oriented styles

14. • Managerial ArchitecturesDescription -- top agents – high level planning ↓ subordinate agents – refine plan, gather resources ↓ lowest level agents▲ AuRA Architectures ▲ SFX Architectures

15. ▲ Autonomous Robot Architecture (AuRA)It consists of five subsystems -- planner : responsible for mission and task planning -- cartographer : all map making, reading functions -- motor : motor schema -- sensor -- homeostatic control : modify the relationship between behaviors by changing the gain as a function of robot or other constraints

16. AuRA Architectural Layout

17. The table below summarizes AuRA in term of the common components and style of emergent behavior

18. ▲ Sensor Fusion Effects (SFX)description –It is an extension to AuRA. The extension was to add modules to specify how sensing and handling sensor failure.

19. Deliberative layers -- Mission planner : acts as a CEO giving a directions -- effector -- Task -- SensorAll of three of above determine the best allocation of effect, sensing resource and perceptual schema. -- Cartographer : map making, path planning

20. Recognition perception Cartographer (model/map making) Deliberative Layer Managers Cerebral Cortex-like functions Deliberative Layer Choice of behaviors, resource allocation, motivation, context Sensor Whiteboard Behavioral Whiteboard Parameters to behaviors, sensor failures, task progress Behaviors (using direct perception, fusion) Superior Colliculus-like functions Sense Reactive Layer Sense Sense Receptive Field actions Sense Sense Sense Muscle Sense Sense Muscle Sense Sense Muscle Focus of attention, recalibration Sensor Actuators SFX (Sensor Fusion Effects)

21. Reactive layersAll these layers reflect to ------- strategic behaviors and tactical behaviorsTactical behavior serves as filter on strategic commands to ensure to robot acts in a safe manner in as close accordance with the strategic intent as possiblethe interaction of strategic and tactical behaviors is still considered emergent behavior

22. Tactical Behaviors

23. The table below summarizes SFX in term of the common components and style of emergent behavior

24. • State-hierarchy Architectures(3 layers)

25. ▲ 3 – tiered (3T)Used for : planetary rovers underwater vehicles robot assistants for astronauts

26. Structure-- planner : setting goal and strategic plans-- sequencer : select a set of primetive behaviors develop a task network-- skill manager : in this layer the skills have associated events to verify explicitly that an action has had to correct effect

27. 3T Architecture

28. The table below summarizes 3T in term of the common components and style of emergent behavior

29. • Model-oriented Architecturestwo of best-known model-oriented architecture▲Saphira architecture▲Task Control Architecture

30. ▲Saphira Architecture-- PRS-Liteit is capable of taking natural language voice commands from humans and then operationalizing that into navigation tasks and perceptual recognition routines. -- virtual sensor -- navigation tasksmanage the behaviors -- LPS (Local Perceptual Space)determine the planning and execution improve the quality of the robot’s overall behavior

31. Saphira Architecture

32. The table below summarizes Saphira in term of thecommon components and style of emergent behavior

33. ▲Task Control Architecture (TCA) -- Task Scheduling (Mission Planner)determine the goal and order of execution -- Path Planning (Cartographer) -- Navigation (Sequencer)to determine what the robot should be looking for, where it is, where it has been. -- Obstacle AvoidanceTo factor in not only obstacle but how to respond with a smooth trajectory for the robot’s current velocity.

34. TCA

35. The table below summarizes TCA in term of the common components and style of emergent behavior

36. Paradigm Paradigm is both a way of looking at the world and an implied set of tools for solving problems. Sense, Plan, Act. Commonly accepted robotic primitives. Robotics have to go through these three, or at least two process to complete a mission. Local Processing and Global World Model Local: sensor data used in specific for each function. Global: all sensor data is processed to single model. Basic Important concept

37. What are the two main features? Robot operates in a top-down fashion. All sensor data tends to be gathered to one global world model. A single representation that planner can use to rout the action. SENSE PLAN ACT Hierarchical Paradigm

38. What are the two main features? Throw out planning all together. The inputs to an act are the direct output of a sensors. examine living example of intelligence. SENSE Reactive Paradigm ACT

39. Features of Hybrid Deliberative/Reactive Paradigm It is reactive planning, Planning to subtask is done at one step. Deliberative planning take a long time comparing to the time of reactive execution Sensor data go directly to each behavior but is also available to the planner for construction of task-oriented global world model. Model-based Architecture focuses on the creation and maintenance of a global world model. Hybrid Paradigm

40. The basic models of Hybrid Paradigm Sequencer: generate a set of behaviors for subtasks. Resource manger: allocate resources to behavior Cartographer: for creating, storing, maintaining map or spatial information. Mission Planner: interact with man, construct a mission plan. Performance Monitoring: monitor the process of the executing, It’s self-awareness. Hybrid Paradigm

41. Plan SENSE ACT Hybrid Paradigm

42. Hybrid Paradigm Robot Primitive output Input Information( sensed and cognitive ) PLAN Directives Actuator command BEHAVIOR Sensed data

43. DARPA UGV Demo II and Demo III. Outdoor ground vehicle control and navigation. given a map and a set of directions find enemy location. Reach in automating highway vehicles by European Community ESPRIT agency and some United States agency Autonomous planetary rovers by NASA. Mapping planetary surface, planning path. Other Hybrid Paradigm

44. Architecture is highly modular Architecture is highly modular of the deliberative with object-oriented programming. Full knowledge of environment Software agents can use agent-specific abstractions to exploit the structure of an environment in order to fulfill their particular role in deliberation. Use of Global models Global models are only for symbolic functions and Planners( sequencers) often produce partial plans. Advantages of Hybrid

45. Execution is reactive. No frame problems. In the Hybrid Paradigm almost no the frame problems resulted by the Hierarchical. Self-consciousness. Ensure robustness by monitoring the performance of the robot and self-diagnosing, this is called self-consciousness. Advantages of Hybrid

46. Example1 we don’t need to turn all sensed data to global model to use in order to accuracy, convince, reliability, and saving time. Example 2 in Hierarchical Paradigm it is unwise in a lot of practical problems to block out the sensed data to Behaviors( Actuator). Examples For Good of The Reactive

47. A/D D/A • f Sensor 1 CPU Sensor 2 A/D LED Sensor 3 Pressure Sensor

48. Gas Sensor 1 Alarm A/D D/A CPU D/A A/D

49. For navigation Deliberation: Cartographer( planner) generates a complete optimal route, decompose the route to segments-waypoints. Reactive Control: Waypoint can be accomplished by behaviors. Top-down method Deliberative layers decompose the missions to finer steps. Reactive layers accomplish the first sub-goal. Interleaving Deliberation and Reactive Control

50. Bottom-up method. Deliberative layers act as virtual sensors. The analyzed information as a sensed data input into behaviors( reactive layers)-Bottom-up Other functions of Deliberations In the deliberative layers, sequencer must know why a failure and know the need to change the behaviors and alert the human supervisor.-self-consciousness. Interleaving Deliberation and Reactive Control