Learning in Artificial Sensorimotor Systems

1. Learning in ArtificialSensorimotor Systems Daniel D. Lee

2. Synthetic intelligence • Hollywood versus reality HAL Robot Governor Data

3. Biological inspiration • Biological motivation for the Wright brothers in designing the airplane

4. Robot dogs • Platforms for testing sensorimotor machine learning algorithms Custom built version Sony Aibos

5. Robot hardware • Wide variety of sensors and actuators.

6. Robocup • Grand challenge for robotics community Humanoid league Legged league Simulation league Small size league Middle size league

7. Legged league • Each team consists of 4 Sony Aibo robot dogs (one is a designated goalie), with WiFi communications. • Field is 3 by 5 meters, with orange ball and specially colored markers. • Game played in two halves, each 10 minutes in duration. Teams change uniform color at half-time. • Human referees govern kick-off formations, holding, penalty area violations, goalie charging, etc. • Penalty kick shootout in case of ties in elimination round. • Recently implemented larger field and wireless communications among robots.

8. Upennalizers in action • GOOAAALLL! 2nd place in 2003.

9. Robot software architecture Perception(Sensors) Cognition (Plan) Action (Actuators) • Sense-Plan-Act cycle.

10. Perception View from Penn’s omnidirectional camera

11. Robot vision Color segmentation: estimate P(Y,Cb,Cr | ORANGE) from training images Region formation: run length encoding, union find algorithm Distance calibration: bounding box size and elevation angle Camera geometry: transformation from camera to body centered coordinates • Tracking objects in structured environment at 25 fps

12. Image reduction Deterministic position: Probabilistic model (Kalman): • 144 176 3 RGB image to 2 position coordinates

13. Image manifolds Pixel vector • Variation in pose and illumination give rise to low dimensional manifold structure

14. Learning nonlinear manifolds Kernel PCA, Isomap, LLE, Laplacian Eigenmaps, etc. • Many recent algorithms for nonlinear manifolds.

15. Locally linear embedding • LLE solves two quadratic optimizations using eigenvector methods (Roweis & Saul).

16. Translational invariance • Application of LLE for translational invariance.

17. Action M. Raibert’s hopping robot (1983)

18. Inverse kinematics • Degenerate solutions with many articulators.

19. Gaits • 4-legged animal gaits

20. Walking • Parameters tuned by optimization techniques Inverse kinematics to calculate joint angles in shoulder and knee

21. Behavior SRI’s Shakey (1970)

22. Probabilistic localization x,y q Particle filter Kalman filter • Kalman and particle filters used to represent pose

23. Finite state machine Search for ball See ball Goto ball position Close to ball Kick ball • Event driven state machine.

24. Potential Fields • Charged particle dynamics to guide motion Potential fields for ball (attractive), field positions (attractive), robots (repulsive), penalty area (repulsive)

25. Learning behaviors Potential field parametersState selection Role switching Adaptive strategies • Reinforcement learning for control parameters

26. Reactive behaviors • “Behavior-based” robotics maps sensors to actuators without complex reasoning

27. Stimulus-response mapping Stimulus space Response space • Construct low dimensional representations for mapping stimulus to response

28. Image correspondences Object 1 Object 2 • Correspondences between images of objects at same pose

29. Data from the web... • http://www.bushorchimp.com

30. ? D1N1 D1n ? D2n D2N2 Learning from examples Given Data (X1,X2): N1 examples of object 1 (D1 dimensions) N2 examples of object 2 (D2 dimensions) n labeled correspondences X1 X2 • Matrix formulation (n << N1, N2 )

31. ? D1N1 D1n ? D2n D2N2 Supervised learning Fill in the blanks: Training Data (D1 +D2) ´ n labeled data D1´ D2 parameters • Problem overfitting with small amount of labeled data

32. Supervised backprop network Original: Reconstruction: Original: Reconstruction: • 15 hidden units, tanh nonlinearity

33. ? D1N1 D1n ? D2n D2N2 Missing data D1+D2 EM algorithm: Iteratively fills in missing data statistics, reestimates parameters for PCA, factor analysis • Treat as missing data problem using EM algorithm

34. EM algorithm X1 X X2 Y1 so that Y Y2 • Alternating minimization of least squares objective function.

35. PCA with correspondences Original: Original: • 15 dimensional subspace, 200 images of each object, 10 in correspondence

36. Factor analysis Original: Original: • 15 dimensional subspace with diagonal noise covariance

37. Common embedding space Two input spaces Common low dimensional space

38. LLE with correspondences Correspondences: • Quadratic optimization with constraints is solved with spectral decomposition

39. LLE with correspondences Original: Original: • 8 nearest neighbors, 2 dimensional nonlinear manifold

40. Quantitative comparison Normalized error Correspondence fraction • Incorporating manifold structure improves reconstruction error.

41. Summary • Adaptation and learning in biological systems for sensorimotor processing. • Many sensory and motor activations are described by an underlying manifold structure. • Development of learning algorithms that can incorporate this low dimensional manifold structure. • Still much room for improvement…