Grasp Types Recognition

1. Grasp Types Recognition A CyberGlove Project Supervisor: Dr. Norm Badler Peggy Choi

2. Agenda • Project Goals • Resources • Literature References • Methodology • Challenges Encountered • Statistical Testing • Future development

3. Agenda • Project Goals • Literature References • Methodology • Algorithm • Challenges Encountered • Statistical Testing • Future Development

4. Project Goals • To develop a program that can recognize grasp types using only inputs from CyberGlove • Input: Joint angles from CyberGlove • Output: Grasp Types

5. Agenda • Project Goals • Literature References • Methodology • Algorithm • Challenges Encountered • Statistical Testing • Future Development

6. Literature Reference • Douville, B.; Levison, L.; and Badler, N. I. 1996.Task-level object grasping for simulated agents. Presence: Teleoperators and Virtual Environments 5(4):416--430 • Cutkosky, M. R.; and Howe, R. D. 1990.Human grasp choice and robotic grasp analysis. In T. Iberall & S.T. Venkataraman (Eds.), Dextrous robot hands (p.5-31). New York: Spring-Verlag • Li, Y.; and Pollard N.S.2005.A Shape Matching Algorithm for Synthesizing Humanlike Enveloping Grasps, IEEE-RAS International Conference on Humanoid Robots (Humanoids 2005), Tsukuba, Japan, 2005. • Ekvall, S. and Kragic D. 2005.Grasp Recognition for Programming by Demonstration, In IEEE/RSJ International Conference on Robotics and Automation. • G.A. Bekey, H. Liu, R. Tomovic and W.J. Karplus. 1993.Knowledge-Based Control of Grasping in Robot Hands Using Heuristics from Human Motor Skills', the IEEE Transactions on Robotics and Automation, Vol.9, No.6, Dec. 1993, pp 709-722 • Kang S.; and Ikeuchi K. 1991.A Framework for Recognizing Grasps, tech. report CMU-RI-TR-91-24, Robotics Institute, Carnegie Mellon University, November, 1991.

7. Agenda • Project Goals • Literature References • Methodology • Algorithm • Challenges Encountered • Statistical Testing • Future Development

8. Wrap (Small) Wrap (Large) Wrap (Small) with Adducted Thumb Wrap (Medium) MethodologyThe formal way to categorize grasps Power Precision Tripod / 2 Fingers (Other Fingers Closed) Platform Push Lateral Pinch Pad Pinch Circular Disk Spherical Spherical (Large) Spherical (Small) Circular Disk 1 Finger (Other Fingers Closed) 1 Finger (Other Fingers Open) 2 Fingers (Other Fingers Closed) 2 Fingers (Other Fingers Open) 4 Fingers (Large) 4 Fingers (Small) 4 Fingers (Small) 3 Fingers

9. Methodology • Alternatives Explored • Distribution of joint angles for different grasp types • Joint angles of fingers that exert opposite forces in order to grasp the object stably • Joint angles relation of all fingers • Time series of joint angles – grasping involves parallel and serial aspects, e.g. while preshaping the hand, the palm is oriented and transported toward the target. The fingers close in parallel with the thumb and further refinements to the position of the palm

10. Methodology • The BEST way to do it … • Since we only have joint angles input, many methods cannot accurately describe grasp types as they also depend on information about object sizes and contact points between the object and the hand • Power/Precision categorization requires knowledge of contact points • Therefore, I Investigated 1278 sets of joint angles • Developed a rule-based algorithm which recognizes grasp types by looking at joint angles relationship

11. Agenda • Project Goals • Literature References • Methodology • Algorithm • Challenges Encountered • Statistical Testing • Future Development

12. The Program • 2 versions • Recognize grasp types in response to user request • Check joint angles every 0.5 seconds, and recognize grasp types when the joint angles are stable, i.e. handshape is not changing • 7 main groups of grasps • Output • Console and Data file

13. Tripod / 2 Fingers (Other Fingers Closed) Platform Push Lateral Pinch Pad Pinch Circular Disk Spherical Spherical (Large) Spherical (Small) Circular Disk 2 Fingers (Other Fingers Closed) Wrap (Small) AlgorithmRuled-based algorithm First, check if the grasp falls under the following 4 groups Lateral pinch satisfies two different sets of criteria 1 2 Lateral Pinch Wrap (Small) with Adducted Thumb 3 Similar enough to be classified as a group 4 Similar enough to be classified as a group

14. Spherical Spherical (Small) 1 Finger (Other Fingers Closed) 1 Finger (Other Fingers Open) 2 Fingers (Other Fingers Open) Wrap (Large) 4 Fingers (Large) 4 Fingers (Small) 3 Fingers AlgorithmRuled-based algorithm If grasp does not belong to any of the previous 4 groups, then check if it corresponds to group 5 5 If grasp also does not belong to group 5, then check if it corresponds to group 6 or 7. Within group 7, check if it belongs to group 7a or 7b 7a 6 7b Similar enough to be classified as a group

15. Agenda • Project Goals • Literature References • Methodology • Algorithm • Challenges Encountered • Statistical Testing • Future Development

16. Challenges Encoutnered • Lack of information, e.g. object sizes, contact points • Rule-based algorithm • Some grasp types are very similar, e.g. 2-finger grasp and 3-finger grasp • Joint angles depend on object sizes and contact points • Difficult to differentiate with joint angle inputs • Recognition for these grasp types is less accurate • Original idea was to keep track of joint angles from the device constantly, and recognize grasp types once the joint angles stop changing. However, the device pushes data too rapidly • So we only pull data from device every 0.5 seconds • Comparing only every 0.5 seconds is more effective

17. Agenda • Project Goals • Literature References • Methodology • Algorithm • Challenges Encountered • Statistical Testing • Future Development

18. Statistical Testing • First test: 178 out of 200 grasp types • Second test: 321 out of 350 grasp types

19. Agenda • Project Goals • Literature References • Methodology • Algorithm • Challenges Encountered • Statistical Testing • Future Development

20. Future Development • Test codes in Jack environment • If collision detection can be used in the virtual environment, the power/precision model can be employed and it should be more effective

21. What I learned how to do that I couldn't do 4 years ago • Programming • Look at problems analytically • Independent research/project