Bayesian goal inference in action observation by co-operating agents

1. Bayesian goal inference in action observation by co-operating agents Raymond H. Cuijpers Project: Joint-Action Science and Technology (JAST) Nijmegen Institute of Cognition and Information Radboud University Nijmegen The Netherlands EU-IST-FP6 Proj. nr. 003747

2. Outline • About Joint Action • The problem of action observation • Simulation theory • Goal inference • Ingredients functional model • Functional model of goal inference • Scenario • Architecture • Simulation results • Conclusions

3. About Joint Action Joint action • Multiple levels of co-ordination • Kinetic: force, timing • Kinematic: speed, trajectory • Action level: what to do? • Goal level: for what purpose? • Reasoning: how to achieve destination? • Actions of co-actors typically differ • Action observation • Anticipation of behaviour of co-actor • Common (ultimate) goal • Action sequences • (immediate) action goal inference

4. The problem of action observation

5. The problem of action observation How can we infer the observed action? Simulation theory: • use own motor system to simulate actions of other Examples: • Motor control theory: • Forward modelling: Predict consequences of actions • Action observation: predict observed action from action repertoire • Robotics: • Direct mapping of observed joint angles on those of own action repertoire Problem: • Requires similar effectors and kinematics • Perception depends on viewpoint

6. The problem of action observation Evidence for simulation theory Ideomotor compatibility Brass, Bekkering, Wohlschlaeger, & Prinz (2000). Lift finger indicated by symbol Response is faster when performing congruent actions Mirror Neurons Rizzolatti, Fadiga, Gallese, & Fogassi (1996). Mirror neurons fire both during observation and execution of similar actions • Shared resources for performing and observing actions • Action system is used in action observation

7. The problem of action observation How can we recognise a dog catching a frisbee? • Different body • Observed effector differs from own effector (mouth vs. hand) • Different kinematics • Direct mapping of joint angles is impossible • Forward modelling is impossible Inference must occur at more abstract level: goal inference

8. The problem of action observation Evidence for goal inference Imitation of 14-months infants Gerely, Bekkering, & Kiraly (2002). Nature. Infants imitate with hands when the actor’s hands were occupied Hands occupied Hands free Imitate using hands Imitate using head • Imitate action goals rather than the effector

9. The problem of action observation Evidence for goal inference Mirror neurons Fogassi, Ferrari, Gesierich, Rozzi, Chersi & Rizzolatti (2005). Science 308:662-667 Firing rate during grasping dependson subsequent movement Activity is selectively tuned to the action goal (=destination of food)

10. The problem of action observation Ingredients for functional model • Viewpoint invariance • Use viewpoint independent measures (distance, colour) • Infer action goals (=intended state change of the world) • make decision at goal level • consistent with final goal state of a sequence of acts • Use your own action system for observation • Use own action repertoire • Use own preferences • Use own task knowledge assume common

11. Functional model of goal inference during action observation Cuijpers RH, Van Schie HT, Koppen M, Erlhagen W and Bekkering H (2006) Goals and means in action observation: a computational approach. Neural Networks 19:311-322.

12. Model of action goal inference Two agents co-operatively build a model from Baufix building blocks • Sequence of primitive motor acts (screw nut, put bolt through hole) • Observable current state and final goal state (final construction) • Shared task knowledge (action repertoire, action goals) • Not shared: Action sequence, viewpoint and personal preferences ? Initial state Final goal state

13. Model of action goal inference Model of action goal inference (Cuijpers et al., 2006) Belief that goal is red-bolt-screwed-in-green-nut Decision marginalisation rule Actor Observer Belief that action is to screw red bolt in green nut marginalisation rule Belief that hand moves to red bolt Bayes rule Likelihood that hand moves to red bolt Observation

14. Turn evidence into beliefs (Bayes’ rule) Belief propagation (marginalisation rule) Model of action goal inference Two fundamental processes Pr( red bolt | observ. ) ~ Pr( observ.| if target is red bolt ) x Pr( red bolt ) • Posterior belief Evidence Personal preference Pr(screw red bolt in green nut) = Pr( screw red bolt in green nut | if target is n ) x Pr( target is n ) S n Action level Knowledge own action repertoire Component level

15. Model of action goal inference Viewpoint invariance • Observations depend on viewpoint invariant measures • Distance between effector and target • Rate of distance change

16. Model of action goal inference Use your own action system • Belief propagation uses task knowledge • Components required for each action alternative p(cn|Ak) • Action goal associated to each action alternative p(ij|Ak) • Use personal preferences (priors) • component preferences p(cn) • Action preferences p(Ak) • Action goal preferences for a given final goal state p(ij|f) • Execution and observation share resources • Task knowledge • Personal preferences

17. Model of action goal inference Infer action goals rather than means • Infer action goal beliefs p(ij|ot,f) • Consistent with final goal state f • Make decision at goal level • Belief in action goal p(ij|ot, f) > threshold

18. Simulation results

19. c1 c2 c3 c4 c5 Simulation results Scenario • Joint Task: Actor: • Action Goal: bolt through slat • Action Alternative: c1+c5 • First target: c1 Observer: infer goal

20. c1 c2 c3 c4 c5 Belief that goal is red-bolt-screwed-in-green-nut Belief that action is to screw red bolt in green nut Belief that hand moves to red bolt Likelihood that hand moves to red bolt Simulation results Belief component cn is the target p(cn|ot) • Nearby targets are more likely unless movement speed is high • Beliefs are biased by personal preferences • c1 correctly identified after 40% of movement time (MT)

21. c1 c2 c3 c4 c5 Belief that goal is red-bolt-screwed-in-green-nut Belief that action is to screw red bolt in green nut Belief that hand moves to red bolt Likelihood that hand moves to red bolt Simulation results Belief in action alternatives p(Ak|ot,f) • Only possible actions (task knowledge) • Only actions consistent with goal state f (task knowledge) • Action alternatives with nearby targets are more likely Impossible! Inconsistent

22. c1 c2 c3 c4 c5 Belief that goal is red-bolt-screwed-in-green-nut Belief that action is to screw red bolt in green nut Belief that hand moves to red bolt Likelihood that hand moves to red bolt Simulation results Belief in action goals p(ij|ot,f) • Inconsistent action goals are suppressed (task knowlegde) • Correct action goal is inferred after 23% of MT • The correct action goal is inferred before the action or the target component

23. Conclusion • We made a functional model that captures behavioural and neurophysiological findings on action observation • Missing knowledge about the co-actor is replaced by task knowledge from the observer’s own action repertoire • To inference process is driven by the likelihood of observed movements and is biased by personal preferences • Action planning is driven by the intended goal and by personal preferences • As a consequence imitation need not involve the same effector (imitation) • Actions are not directly mapped onto the observer’s repertoire. • Consequently, complementary actions can be as fast as imitative actions in a joint action context

24. Thank you for your attention!

25. p(cn|ot) ~ p(ot|cn) p(cn) Component belief  likelihood, preference Action belief  action knowledge, component belief Goal belief  goal knowledge, action belief p(Ak|ot) = Sn p(Ak|cn) p(cn|ot) p(ij|ot,f) = Sk p(i  j|Ak,f) p(Ak|ot) p(i  j|Ak,f) = p(Ak|i  j) p(i  j|f)/ p(Ak|f) p(Ak|cn) = p(cn|Ak)p(Ak)/p(cn)