Pat Langley Dongkyu Choi Computational Learning Laboratory

1. A Unified Cognitive Architecture for Physical Agents Pat Langley Dongkyu Choi Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA http://cll.stanford.edu/ Thanks to K. Cummings, N. Nejati, S. Rogers, S. Sage, and D. Shapiro for their many contributions. This talk reports research. funded by grants from DARPA IPTO, which is not responsible for its contents.

2. Psychological Ideas as Design Heuristics To develop intelligent systems, we must constrain their design, and findings about human behavior can suggest: how the system should represent and organize knowledge how the system should use that knowledge in performance how the system should acquire knowledge from experience This approach has led to many new insights, starting with Newell, Shaw, and Simon’s (1956) work on the Logic Theorist.

3. Cascaded Integration in ICARUS In this talk I will use ICARUS a unified cognitive architecture  to illustrate the value of ideas from psychology. learning problem solving skill execution conceptual inference ICARUSadopts a cascaded approach to system integration in which lower-level modules produce results for higher-level ones.

4. E.g., psychology makes claims about conceptual knowledge: Representing and Using Concepts • concepts are distinct cognitive entities that support both categorization and inference; • the majority of human concepts are grounded in perception and action (Barsalou, 1999); • many human concepts are relational in nature, describing connections among entities (Kotovsky & Gentner, 1996); • concepts are organized in a hierarchical manner, with more complex categories defined in terms of simpler ones. ICARUS adopts these assumptions about conceptual memory.

5. ICARUS Concepts for In-City Driving ((in-rightmost-lane ?self ?clane) :percepts ( (self ?self) (segment ?seg) (line ?clane segment ?seg)) :relations ((driving-well-in-segment ?self ?seg ?clane) (last-lane ?clane) (not (lane-to-right ?clane ?anylane)))) ((driving-well-in-segment ?self ?seg ?lane) :percepts ((self ?self) (segment ?seg) (line ?lane segment ?seg)) :relations ((in-segment ?self ?seg) (in-lane ?self ?lane) (aligned-with-lane-in-segment ?self ?seg ?lane) (centered-in-lane ?self ?seg ?lane) (steering-wheel-straight ?self))) ((in-lane ?self ?lane) :percepts ( (self ?self segment ?seg) (line ?lane segment ?seg dist ?dist)) :tests ( (> ?dist -10) (<= ?dist 0)))

6. Structure and Use of Conceptual Memory ICARUS organizes conceptual memory in a hierarchical manner. Conceptual inference occurs from the bottom up, starting from percepts to produce high-level beliefs about the current state.

7. Representing Short-Term Beliefs/Goals (current-street me A) (current-segment me g550) (lane-to-right g599 g601) (first-lane g599) (last-lane g599) (last-lane g601) (at-speed-for-u-turn me) (slow-for-right-turn me) (steering-wheel-not-straight me) (centered-in-lane me g550 g599) (in-lane me g599) (in-segment me g550) (on-right-side-in-segment me) (intersection-behind g550 g522) (building-on-left g288) (building-on-left g425) (building-on-left g427) (building-on-left g429) (building-on-left g431) (building-on-left g433) (building-on-right g287) (building-on-right g279) (increasing-direction me) (buildings-on-right g287 g279)

8. Psychology also makes claims about skills and their execution: Skills and Execution • the same generic skill may be applied to distinct objects that meet its application conditions; • skills support the execution of complex activities that have hierarchical organization (Rosenbaum et al., 2001); • humans can carry out open-loop sequences, but they can also operate in closed-loop reactive mode; • humans can deal with multiple goals with different priorities, which can lead to interrupted behavior. ICARUS embodies these ideas in its skill execution module.

9. ICARUS Skills for In-City Driving ((in-rightmost-lane ?self ?line) :percepts((self ?self) (line ?line)) :start ((last-lane ?line)) :subgoals ((driving-well-in-segment ?self ?seg ?line))) ((driving-well-in-segment ?self ?seg ?line) :percepts((segment ?seg) (line ?line) (self ?self)) :start ((steering-wheel-straight ?self)) :subgoals ((in-segment ?self ?seg) (centered-in-lane ?self ?seg ?line) (aligned-with-lane-in-segment ?self ?seg ?line) (steering-wheel-straight ?self))) ((in-segment ?self ?endsg) :percepts((self ?self speed ?speed) (intersection ?int cross ?cross) (segment ?endsg street ?cross angle ?angle)) :start ((in-intersection-for-right-turn ?self ?int)) :actions((steer 1)))

10. ICARUS Skills Build on Concepts ICARUS stores skills in a hierarchical manner that links to concepts. concepts Each concept is defined in terms of other concepts and/or percepts. Each skill is defined in terms of other skills, concepts, and percepts. skills

11. Skill Execution in ICARUS Skill execution occurs from the top down, starting from goals to find applicable paths through the skill hierarchy. This occurs repeatedly on each cycle to support reactive control with a bias toward persistence of initiated skills.

12. Psychology also has ideas about problem solving and learning: Ideas about Problem Solving and Learning • humans often resort to means-ends analysis to solve novel problems (Newell & Simon, 1961); • problem solving often occurs in a physical context and is interleaved with execution (Gunzelman & Anderson, 2003); • efforts to overcome impasses during problem solving leads to incremental acquisition of new skills (Anzai & Simon, 1979); • structural learning involves monotonic addition of symbolic elements to long-term memory; • learning can transform backward-chaining heuristic search into informed forward-chaining execution (Larkin et al., 1980). ICARUS reflects these ideas in its problem solving and learning.

13. ICARUS Interleaves Execution and Problem Solving Skill Hierarchy Problem Reactive Execution ? no impasse? Primitive Skills yes Executed plan Problem Solving This organization reflects the psychological distinction between automatized and controlled behavior.

14. Skill Hierarchy Problem ? ICARUS Learns Skills from Problem Solving Reactive Execution no impasse? Primitive Skills yes Executed plan Problem Solving Skill Learning

15. Learning Skills for In-City Driving We have trained ICARUS to drive in a simulated in-city environment. We provide the system with tasks of increasing complexity. Learning transforms the problem-solving traces into hierarchical skills. The agent uses these skills to change lanes, turn, and park using only reactive control.

16. ICARUS has much in common with other cognitive architectures like Soar (Laird et al., 1987) and ACT-R (Anderson, 1993): Similarities to Previous Architectures Short-term memories are distinct from long-term stores Memories contain modular elements cast as symbolic structures Long-term structures are accessed through pattern matching Cognition occurs in retrieval/selection/action cycles Learning involves monotonic addition of elements to memory Learning is incremental and interleaved with performance These ideas all have their origin in theories of human memory, problem solving, and skill acquisition.

17. However, ICARUSalso makes assumptions that distinguish it from these architectures: Distinctive Features of ICARUS Cognition is grounded in perception and action Categories and skills are separate cognitive entities Short-term elements are instances of long-term structures Inference and execution are more basic than problem solving Skill/concept hierarchies are learned in a cumulative manner Some of these assumptions appear in Bonasso et al.’s (2003) 3T, Freed’s APEX, and Sun et al.’s (2001) CLARION architectures. These ideas have their roots in cognitive psychology, but they are also effective in building integrated intelligent agents.

18. Directions for Future Research Future work on ICARUS should incorporate other ideas about: progressive deepening in forward-chaining search graded nature of categories and category learning model-based character of human reasoning persistent but limited nature of short-term memories creating perceptual chunks to reduce these limitations storing and retrieving episodic memory traces These additions will increase further ICARUS’ debt to psychology. For more details, see:http://cll.stanford.edu/research/ongoing/icarus/

19. End of Presentation