1 / 43

Learning object affordances based on structural object representation

Learning object affordances based on structural object representation. Kadir F. Uyanik Asil Kaan Bozcuoglu EE 583 Pattern Recognition Jan 4, 2011. Content. Goal Inspirations Potential Difficulties Problem Definition Proposed Method References Appendix. Goal. Goal. Goal. Goal.

alexis
Télécharger la présentation

Learning object affordances based on structural object representation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Learning object affordances based on structural object representation Kadir F. Uyanik Asil Kaan Bozcuoglu EE 583 Pattern Recognition Jan 4, 2011

  2. Content • Goal • Inspirations • Potential Difficulties • Problem Definition • Proposed Method • References • Appendix

  3. Goal

  4. Goal

  5. Goal

  6. Goal

  7. Inspirations Ecological Psychologist James Jerome Gibson 1904 -1979 Cognitive Psychologist Irving Biederman 1939 -

  8. Inspirations:Affordances[1] “… an affordance is neither an objective property nor a subjective property; or both if you like. An affordance cuts across the dichotomy of subjective-objective and helps us to understand its inadequacy. It is equally a fact of the environment and a fact of behavior. It is both physical and psychical, yet neither. An affordance points both ways, to the environment and to the observer.” [1] J. J. Gibson (1977), The Theory of Affordances. In Perceiving, Acting, and Knowing, Eds. Robert Shaw and John Bransford, ISBN 0-470-99014-7. [2] E. Sahin, M. Cakmak, M.R.Dogar, E. Ugur , G. Ucoluk, To Afford or Not to Afford: A New Formalization of Affordances Toward Affordance-Based Robot Control, Adaptive Behavior , 2007 pp: 447-472

  9. Inspirations:Affordances[1] Throw-able “… an affordance is neither an objective property nor a subjective property; or both if you like. An affordance cuts across the dichotomy of subjective-objective and helps us to understand its inadequacy. It is equally a fact of the environment and a fact of behavior. It is both physical and psychical, yet neither. An affordance points both ways, to the environment and to the observer.” Push-able [1] J. J. Gibson (1977), The Theory of Affordances. In Perceiving, Acting, and Knowing, Eds. Robert Shaw and John Bransford, ISBN 0-470-99014-7. [2] E. Sahin, M. Cakmak, M.R.Dogar, E. Ugur , G. Ucoluk, To Afford or Not to Afford: A New Formalization of Affordances Toward Affordance-Based Robot Control, Adaptive Behavior , 2007 pp: 447-472

  10. Inspirations:Affordances[1] Throw-able “… an affordance is neither an objective property nor a subjective property; or both if you like. An affordance cuts across the dichotomy of subjective-objective and helps us to understand its inadequacy. It is equally a fact of the environment and a fact of behavior. It is both physical and psychical, yet neither. An affordance points both ways, to the environment and to the observer.” Push-able (<effect>, <(entity, behavior)>) Revised Definition: An affordance is an acquired relation between a <(entity, behavior)> tuple of an agent such that the application of the <behavior> on the <entity> generates a certain <effect>[2]. environment agent <entity> <behavior> <effect> [1] J. J. Gibson (1977), The Theory of Affordances. In Perceiving, Acting, and Knowing, Eds. Robert Shaw and John Bransford, ISBN 0-470-99014-7. [2] E. Sahin, M. Cakmak, M.R.Dogar, E. Ugur , G. Ucoluk, To Afford or Not to Afford: A New Formalization of Affordances Toward Affordance-Based Robot Control, Adaptive Behavior , 2007 pp: 447-472

  11. Inspirations:Human Image Understanding[3] “There are small number of geometric components that constitute the primitive elements of the object recognition system (like letters to form words)” [3] Recognition-by-components: A theory of Human Image Understanding, Psychological Review, Vol. 94 (1987), pp. 115-148

  12. Inspirations:Human Image Understanding[3] “There are small number of geometric components that constitute the primitive elements of the object recognition system (like letters to form words)” [3] Recognition-by-components: A theory of Human Image Understanding, Psychological Review, Vol. 94 (1987), pp. 115-148

  13. Potential Difficulties[4] • Structural description not enough, also need metric info [4] M. A. Arbib CS564 – Brain Theory and Artificial Intelligence, USC, Fall 2001, Lecture 7: Object Recognition

  14. Potential Difficulties[4] • Structural description not enough, also need metric info • Difficult to extract geons from real images [4] M. A. Arbib CS564 – Brain Theory and Artificial Intelligence, USC, Fall 2001, Lecture 7: Object Recognition

  15. Potential Difficulties[4] • Structural description not enough, also need metric info • Difficult to extract geons from real images • Ambiguity in the structural description: most often we have several candidates [4] M. A. Arbib CS564 – Brain Theory and Artificial Intelligence, USC, Fall 2001, Lecture 7: Object Recognition

  16. Potential Difficulties[4] • Structural description not enough, also need metric info • Difficult to extract geons from real images • Ambiguity in the structural description: most often we have several candidates • For some objects, deriving a structural representation can be difficult [4] M. A. Arbib CS564 – Brain Theory and Artificial Intelligence, USC, Fall 2001, Lecture 7: Object Recognition

  17. Problem Definition HOW TO • decompose objects into parts/components ? • find relations between components ? • find a generic graph representation of an <action-entity-effect> three tuple ?

  18. Object DecompositionProposed Algorithm

  19. Object DecompositionProposed Algorithm

  20. Object DecompositionProposed Algorithm

  21. Object DecompositionProposed Algorithm

  22. Object DecompositionProposed Algorithm

  23. Object DecompositionProposed Algorithm

  24. Object DecompositionProposed Algorithm

  25. Object DecompositionProposed Algorithm

  26. Object DecompositionProposed Algorithm

  27. Object DecompositionProposed Algorithm

  28. Object DecompositionProposed Algorithm

  29. Object Decomposition What is missing? • Use/try different clustering algorithms • Triangulate 3D surfaces, Delaunay • Compute gaussian curvature on each vertex • Detect region boundaries, curvature thresholding • Perform iterative region growing, flood fill

  30. Graphical Representation • We represent each objects in non-directed graphs as follows: • Each node has the info of geometric shape of the part • Each edge has the information of direction of edge for three axises, i.e from node1 to node2, x axis increases.

  31. Graphical RepresentationSimilarity Checking • [isIsomorphic, label_list]= check_Isomorphism(G1, G2) • If isIsomorphic • Check geometric shapes of same labeled nodes in two graphs • Check direction of equivalent edges in both graphs • If both are matched, return true • Else return false • Else return false

  32. Graphical RepresentationSimilarity Checking Isomorphism check: Two candidates: - n1 = n6, n2 = n4, n3 = n5 (Attributes matched!) - n1 = n4, n2 = n6, n3 = n5 (Attributes isn’t matched)

  33. Current System • 80% is successful • Assumes no occlusion. • For the cup case, handles should always be visible • Needs metric info to distinguish bigger objects from small ones

  34. One way to go… • Learning a generic graph for each affordance type. • Checking the maximal- cliques of the match graph while comparing graph of an object and a generic graph. • Mahalanobis distance metric for generic graphs and use MLE

  35. Tools

  36. References [1] J. J. Gibson (1977), The Theory of Affordances. In Perceiving, Acting, and Knowing, Eds. Robert Shaw and John Bransford, ISBN 0-470-99014-7. [2] E. Sahin, M. Cakmak, M.R.Dogar, E. Ugur , G. Ucoluk, To Afford or Not to Afford: A New Formalization of Affordances Toward Affordance-Based Robot Control, Adaptive Behavior , 2007 pp: 447-472 [3] Recognition-by-components: A theory of Human Image Understanding, Psychological Review, Vol. 94 (1987), pp. 115-148 [4] M. A. Arbib CS564 – Brain Theory and Artificial Intelligence, USC, Fall 2001, Lecture 7: Object Recognition

  37. Thanks for listening

  38. Appendix

  39. Human Image Understanding • Hypothesis: small number of geometric components that constitute the primitive elements of the object recognition system (like letters to form words) • Geons are directly recognized from edges, based on their nonaccidental properties (i.e., 3D features that are usually preserved by the projective imaging process). • edges are straight or curved • pairs of edges are parallel or non-parallel • vertices will always appear to be vertices • Non-accidental properties allows geons to be recognized from any perspective. • The information in the geons are redundant so that they can be recognized even when partially occluded.

  40. AppendixThe Importance of spatial arrangement

  41. AppendixThe Principal of non-accidentalness • Examples: • Colinearity • Smoothness • Symmetry • Parallelism • Cotermination

  42. AppendixSome non-accidental differences

More Related