Biological Foundations of the Reactive Paradigm

1. Biological Foundations of the Reactive Paradigm • Describe the three levels in a Computational Theory. • Explain in one or two sentences each of the following terms: reflexes, taxes, fixed-action patterns, schema, affordance. • Be able to write pseudo-code of an animal’s behaviors in terms of innate releasing mechanisms, identifying the releasers for the behavior. • Given a description of an animal's sensing abilities, its task, and environment, identify an affordance for each behavior. • Given a description of an animal's sensing abilities, its task, and environment, define a set of behaviors using schema theory to accomplish the task. Review Why? -comp. theory IRM Perception -Summary Chapter 3: Biological Foundations

2. Robots In the Hierarchical Paradigm Chapter 3: Biological Foundations

3. Timeline of Influences Braitenberg’s Vehicles Arkin’s Schemas Neisser Arbib’s Schemas J.J. Gibson Payton Brook’s Insects Middlestat Marr’s Computational Theory Tinbergen & Lorenz & von Frisch 1970 1980 1990 Chapter 3: Biological Foundations

4. Level 1: What is the phenomena we’re trying to represent? Level 2: How it be represented as a process with inputs/outputs? for (i=nCol.. Level 3: How is it implemented? Marr’s Computational Theory Chapter 3: Biological Foundations

5. Level 1: What is the phenomena we’re trying to represent? Level 1: Existence Proof Goal: how to make line drawings of objects? people can do this by age 10, computers should Chapter 3: Biological Foundations

6. Level 2: How it be represented as a process with inputs/outputs? for (i=nCol.. Level 2: Inputs, Outputs, Transforms drawing light lines (edges) drawing light retina (gradient) Chapter 3: Biological Foundations

7. -1 0 +1 0 -2 +2 +1 +2 +1 -1 0 +1 0 0 0 -1 -2 -1 Sobel Edge Detector in computer vision Level 3: How is it implemented? Level 3: Implementation - - - - - + - - - Center Surround Cell in retinal ganglion Chapter 3: Biological Foundations

8. Class Discussion • Give three examples of how biology has informed modern technology? • ex. Wright Brothers- control flaps on airplane wings Chapter 3: Biological Foundations

9. Behavior Definition (graphical) BEHAVIOR Pattern of Motor Actions Sensory Input Chapter 3: Biological Foundations

10. Types of Behaviors • Reflexive • stimulus-response, often abbreviated S-R • Reactive • learned or “muscle memory” • Conscious • deliberately stringing together WARNING Overloaded terms: Roboticists often use “reactive behavior” to mean purely reflexive, And refer to reactive behaviors as “skills” Chapter 3: Biological Foundations

11. Ethology: Coordination and Control of Behaviors Nobel 1973 in physiology or medicine • von Frisch • Lorenz • Tinbergen INNATE RELEASING MECHANISMS www.nobel.se Chapter 3: Biological Foundations

12. Arctic terns live in Arctic (black, white, gray environment, some grass) but adults have a red spot on beak When hungry, baby pecks at parent’s beak, who regurgitates food for baby to eat How does it know its parent? It doesn’t, it just goes for the largest red spot in its field of view (e.g., ethology grad student with construction paper) Only red thing should be an adult tern Closer = large red Arctic Terns Chapter 3: Biological Foundations

13. behavior template BEHAVIOR Pattern of Motor Actions Sensory Input Chapter 3: Biological Foundations

14. “the feeding behavior” Feeding BEHAVIOR RED PECK AT RED Pattern of Motor Actions Sensory Input Chapter 3: Biological Foundations

15. Releaser present? N /dev/null Y the releaser template Sensory input and/or internal state Chapter 3: Biological Foundations

16. “the feeding releaser” RED & HUNGRY sensory input Releaser internal state present? N /dev/null Y Feeding BEHAVIOR RED PECK AT RED Chapter 3: Biological Foundations

17. Releaser present? N /dev/null Y Innate Releasing Mechanisms Sensory input and/or internal state BEHAVIOR Pattern of Motor Actions Sensory Input Chapter 3: Biological Foundations

18. Example: Hide Behavior • Programmed in C++, << 100 LOC • shows • taxis (oriented relative to light, wall, niche) • fixed action pattern (persisted after light was off) • reflexive (stimulus, response) • impliciting sequencing • use of internal state Chapter 3: Biological Foundations

19. Example: Cockroach Hide • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out • even if the lights are turned back off earlier Chapter 3: Biological Foundations

20. Reflexive Behaviors S-R • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out • even if the lights are turned back off earlier Chapter 3: Biological Foundations

21. Fixed Pattern Actions • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out • even if the lights are turned back off earlier Chapter 3: Biological Foundations

22. Exhibits Taxis • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out • even if the lights are turned back off earlier to light to wall to niche Chapter 3: Biological Foundations

23. Class Exercise • Draw flowchart of how this works • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out • even if the lights are turned back off earlier Chapter 3: Biological Foundations

24. Break into Behaviors Flee • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out Follow- wall hide Chapter 3: Biological Foundations

25. LIGHT LIGHT present? present? N N Find Releasers Y Flee • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out Ooops, need internal state: Scared Follow- wall SCARED & SURROUNDED present? N hide Chapter 3: Biological Foundations

26. LIGHT present? N Internal State Set Y Flee • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out SCARED BLOCKED & SCARED present? N Follow- wall SCARED & SURROUNDED present? N hide Chapter 3: Biological Foundations

27. LIGHT present? N Action steer 360, drive forward Y Flee • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out SCARED BLOCKED & SCARED present? N steer =F(dist to wall) drive forward const. Follow- wall SCARED & SURROUNDED present? steer =F(dist to wall) drive forward const. stop N hide Chapter 3: Biological Foundations

28. LIGHT present? N IR Sensory Input steer 360, drive forward Y encoders Flee • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out SCARED BLOCKED & SCARED IR present? N steer =F(dist to wall) drive forward const. IR Follow- wall SCARED & SURROUNDED IR present? steer =F(dist to wall) drive forward const. stop N IR hide Chapter 3: Biological Foundations

29. LIGHT present? N IR How Do You Link Them? steer 360, drive forward Y encoders Flee • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out SCARED BLOCKED & SCARED IR present? N steer =F(dist to wall) drive forward const. IR Follow- wall SCARED & SURROUNDED IR present? steer =F(dist to wall) drive forward const. stop N IR hide Chapter 3: Biological Foundations

30. Analogy:IRMs work on THREADS,not sequential processing! • Very simple modules • Nice building blocks since not directly linked • If one module (part of brain) fails, what happens? Chapter 3: Biological Foundations

31. What happens when there’s a conflict from concurrent behaviors? • Equilbrium • Feeding squirrels-feed, flee: hesitate in-between • Dominance • Sleepy, hungry: either sleep or eat • Cancellation • Sticklebacks defend, attack: build a nest ? Chapter 3: Biological Foundations

32. World Acts & Modifies World Samples, Finds Potential Actions Directs what to look for Cognitive Activity Perception of Environment Perception • Two uses of perception (can be the same percept) • Release a behavior • Guide a behavior • Action-oriented perception (Neisser) • Planning is not needed to act • Perception is selective Chapter 3: Biological Foundations

33. Gibson’s Ecological Approach • Acting and sensing co-evolved as agent survived in a particular environment. The environment affords the agent what it needs to survive. • The perception needed to release or guide the “right action” is directly in the environment, not inferred or memorized • Ex. Red on Artic Terns== food • Ex. Sound of filling container==full • Percepts are called affordances or said to be obtained through direct perception Chapter 3: Biological Foundations

34. Gibsonian Affordances • How do you know you’re going fast in a car? Or in a space movie? • How do animals know when to mate? • How do mosquitoes know to bite in the most tender areas? • What should you do when you think you’re being stalked by a mountain lion? • What’s your favorite fishing lure? Chapter 3: Biological Foundations

35. Sittability Chapter 3: Biological Foundations

36. But does this really hold for everything? • “my car” versus “your car”? • Difference is • where I parked it (memory) • Semantic meaning (cars aren’t generic like nuts to a squirrel) • Neisser’s Two Systems • Direct Perception: older, behavioral • Recognition: evolved later, deliberative Chapter 3: Biological Foundations

37. Review Questions • What are the levels of a computational theory? • Existence proof, inputs-outputs-transformations, implementation • What is a behavior? • A behavior is a mapping of sensory inputs to a pattern of motor actions • Is sequencing normally implicit or explicit in IRM? • implicit • What is an affordance? • A potentiality in the environment for an action Chapter 3: Biological Foundations

38. Schema Theory schema- is used in cognitive science and ethology to refer to a particular organized way of perceiving cognitively and responding to a complex situation or set of stimuli • is generic, equivalent to an object in OOP • schema specific knowledge (local data) • procedural knowledge (methods) • schema instantation is specific to a situation, equivalent to an instance in OOP • a behavior is a schema, consists of • perceptual schema • motor schema Chapter 3: Biological Foundations

39. Behavioral Schema alternative PS, MS sequencing logic for reactive skills (judgment value function) action, intensity percept, gain Perceptual Schema (PS) Motor Schema (MS) Reflexive behaviors usually just have “methods”, not data Chapter 3: Biological Foundations

40. Ex. Fly Snapping Behavior IRM Releaser: small moving dark blob present? N /dev/null Y snap, 100% x,y,z, 100% track(blob) snap(blob) Chapter 3: Biological Foundations

41. Schema Instantiation (SI) Releaser: small moving dark blob present? N /dev/null Y snap, 100% x,y,z, 100% track(blob) snap(blob) Chapter 3: Biological Foundations

42. releaser Perceptual Schema Library Motor Schema Library 1. track(blob) snap(blob) present? 2. track(blob) snap(blob) N track(blob) snap(blob) behavior schema 3. x,y,z, 100% track(blob) snap(blob) Schema/Schema Instantiation Chapter 3: Biological Foundations

43. Advantages • modular • can assemble new behaviors from existing schemas • learning by experimentation • can substitute alternatives • reroute nerves Chapter 3: Biological Foundations

44. Snap at vector sum (middle) Instantiation for each eye Releaser: small moving dark blob present? Left eye N /dev/null Y snap, 100% x,y,z, 100% track(blob) snap(blob) Releaser: small moving dark blob present? N /dev/null Right eye Y snap, 100% x,y,z, 100% track(blob) snap(blob) Chapter 3: Biological Foundations

45. LIGHT present? N IR Where’s the MS and PS? steer 180, drive forward Y encoders Flee • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out SCARED BLOCKED & SCARED IR present? N steer =F(dist to wall) drive forward const. IR Follow- wall SCARED & SURROUNDED IR present? steer =F(dist to wall) drive forward const. stop N IR hide Chapter 3: Biological Foundations

46. General Principles • All animals possess a set of behaviors • Releasers for these behaviors rely on both internal state and external stimulus • Perception is filtered; perceive what is relevant to the task • Some behaviors and associated perception do not require explicit knowledge representation (e.g., rely on affordances) Chapter 3: Biological Foundations

47. Silicon v. Carbon • Individual robots must survive, not species • detection of non-productive behaviors • graceful degradation • Must be able to predict emergent behaviors • Not clear how to learn quickly • Robots need more alternative perceptual schemas since poorer understanding of the environment Chapter 3: Biological Foundations

48. Unresolved Issues • How to resolve conflicts? • behavioral arbitration/combination • When is explicit representations, memory needed? • How to set up or learn new sequences of behaviors • What are the affordances for a particular ecology? Chapter 3: Biological Foundations

49. Take Home Thoughts… • Ideas bubbling up for robotics • Maybe programming in terms of behaviors is better than STRIPS or trying to set up a complex hierarchy • Intelligence has something to do with agent’s ecological niche: its abilities, its tasks (survival), and environment • Perception is going to be critical because it releases and guides actions • IRMs, Schemas are nice ways to start thinking about the computational structure of programming a robot Chapter 3: Biological Foundations

50. Review Questions • Think about the robots at the WTC. What are affordances of victims? • color, motion, sound, heat • Can schema theory represent behaviors in both biological and computational systems? • yes • A behavior schema is composed of at least the following: • motor schema and a perceptual schema • What is an example of behavior-specific knowledge? • sequencing in a skill, alternative PS or MS Chapter 3: Biological Foundations