1 / 79

Virtual Hand

A HCI Testbed for Computer Vision Researches on Human Hand by Beifang Yi August, 2003. Virtual Hand. A thesis submitted in partial fulfillment of the requirments for the degree of Master of Science with a major in Computer Engineering. Virtual Hand. Virtual Hand. Objective:

annaarnold
Télécharger la présentation

Virtual Hand

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A HCI Testbed for Computer Vision Researches on Human Hand by Beifang Yi August, 2003 Virtual Hand

  2. A thesis submitted in partial fulfillment of the requirments for the degree of Master of Science with a major in Computer Engineering Virtual Hand

  3. Virtual Hand • Objective: • To provide a virtual environment for experiments on human hand in computer vision domain: • A dynamic hand model • A graphical user interface

  4. Virtual Hand • Presentation outline: • Background • Hand model • Interface design • Demonstration

  5. Virtual Hand: Background • Human-Computer Interaction (HCI) • A discipline concerned with the design, evaluation and implementation of interactive computing systems for human use and with the surrounding environment. • People • Computing system • Usability • Interaction

  6. Virtual Hand: Background • Human-Computer Interaction (HCI) • On human side: human cognitive features—attention, motivation... • On machine side: input/output devices.

  7. Virtual Hand: Background • Human-Computer Interaction (HCI) • “Almost any natural communication among humans involves multiple, concurrent modes of communications.” • Therefore, “people prefer tointeract multimodally with computers.”

  8. Virtual Hand: Background • Human-Computer Interaction (HCI) • Traditional computer has made full use of its “touch” faculty: keyboard, mouse, touchpad... • Now, HCI is trying to employ other means that human boasts, especially, that of sight (vision) sense.

  9. Virtual Hand: Background • Vision-based HCI: • By vision, we refer to the use of camera, • And a set of visual or graphical techniques for presenting and processing information. • This visual sensing endows a vigorous input mode to HCI. • Examles: face/gaze/fatigue/on-road-car detection...

  10. Virtual Hand: Background • Hand-related Vision-based HCI: • Hand gestures, a kind of non-verbal interactions among humans, • But have a linguistic role.

  11. Virtual Hand: Background • The dexterity of human hand: • Humans have five senses, but animals enjoy far more acute, perfect sensory faculties with exception of the sense of touch. • “We ought to define the hand as belonging exclusively to man,” Sir Charles Bell acknowledged in 1885, because its sensibility and ingenuity “converts the being who is the weakest in natural defense, to the ruler over animate and inanimate nature.”

  12. Virtual Hand: Background • The dexterity of human hand: • Our ancestors long ago did understand this. • Left: a red hand silhouette painted on a stone slab, about 10,000 years ago.

  13. Virtual Hand: Background • The dexterity of human hand: • Do you know Chinese Gongfu (Taiji)? • With exertion on your hand a little force (only of several Newtons), you can fend off offender's strike of thousands of Newtons, which will bounce back upon the offender.

  14. Virtual Hand: Background • The dexterity of human hand: • At present, we have been intensively taking advantage of our hand.

  15. Virtual Hand: Background • The dexterity of human hand: • From now on, we will be depending on our hand gestures to give commands to computing systems.

  16. Virtual Hand: Background • Functions of the hand: • “The hand at rest is beautiful in its tranquility, but it is infinitely more appealing in the flow of action.” • The functions of the hand can be described through: • Opposition • Prehensile movement • Non-prehensile movement

  17. Virtual Hand: Background • Function of the hand—opposition: • The pulp surface of the thumb is placed squarely in contact with or opposite to the terminal pads of any rmaining digits.

  18. Virtual Hand: Background • Function of the hand—prehensile: • Gripping or pinching an object between the digits and palm.

  19. Virtual Hand: Background • Function of the hand—non-prehensile: • Pushing, lifting, tapping, typewriting...

  20. Virtual Hand: Background • Hand gestures: • “If language was given to men to conceal their thoughts, then gesture's purpose was to disclose them.” • Classification of hand gestures varies from their functionality, linguistic implication, application, and etc.

  21. Virtual Hand: Background • Anatomy of the hand: • Main elements: bones, muscles, tendons. • Tendon, the rough cord of dense white fibrous tissue, unites muscles with bones and transmits the force which the muscles exert.

  22. Virtual Hand: Background • Anatomy of the hand: • The transmission of force produces hand movement at a joint between bones. • We can see this movement as a result of the hand bones' positions and orientations.

  23. Virtual Hand: Background • Hand as a machine: • There are reasons: • Motor: muscle, two-stroke engine: pull-push. • Transmission: tendon. • Application: skin and pulp tissue.

  24. Virtual Hand: Background • Hand joints: • Fingers: • DIP • PIP • MCP • Thumb: • IP • MCP • CMC

  25. Virtual Hand: Background • Hand joints: • For simplicity, use left diagram. • Hand is considered as a manipulator with 27 degrees of freedom (DOFs).

  26. Virtual Hand: Background • Hand joint movement: • Flexion and extension, • Abduction and adduction, • Rotation.

  27. Virtual Hand: Background • Biomechanics of the hand: • At a joint, two coordinate systems: • Proximal • Distal • Their axes' representations.

  28. Virtual Hand: Background • Biomechanics of the hand: • Thus we have:

  29. Virtual Hand: Background • Biomechanics of the hand: • The finger joint motion (flexion-extension) ranges.

  30. Virtual Hand: Background • Biomechanics of the hand: • The thumb joint motion (flexion-extension and abduction-adduction) ranges.

  31. Virtual Hand: Background • Hand constraints: • Usually, we study natural hand movement. • Hand can not make any arbitrary gestures.

  32. Virtual Hand: Background • Some of the hand constraints: • The four fingers are planar manipulators with the exception of the MCP joints. • There exists a relationship between DIP and PIP flexions. • The MCP joint in middle finger displays little abduction and adduction.

  33. Virtual Hand: Background • Some of the hand constraints: • There exist “finger planes”: for each finger, the five points(tip, DIP, PIP, MCP, wrist joint) are coplanar. • There exist a “thumb plane”: four thumb points are coplanar. • There exists a “palm plane”: all the finger MCP joints are in a plane, which is perpendicular to “finger plane”.

  34. Virtual Hand: Background • Hand modeling: • Physical models: hand deformation under internal and external forces applied to hand. • Statistical models: deformation “learning” through a set of training examples. • Geometrical models: cubes, cylinders, 3D polygons, spline-based surfaces, and etc.

  35. Virtual Hand: Background • Hand modeling (example): • Multi-layered (physical) modeling: complicated.

  36. Virtual Hand: Background • Hand modeling (examples in computer vision): • Geometrical models: simple.

  37. Virtual Hand: Background • Hand modeling (examples in computer vision): • Geometrical models: a little sophisticate.

  38. Virtual Hand: Background • Hand modeling (examples in computer vision): • Geometrical models.

  39. Virtual Hand: Modeling • Hand modeling: • Static hand model • Dynamic hand model • Implemented with Open (Coin) Inventor

  40. Virtual Hand: Modeling • Static hand model: • Raw hand data acquisition: • Simple: cyliners, transformed cubes and spheres. • Sophisticate: a set of 3D points from 3D scanner..., the size varies. • Can be found on Internet or purchased from company.

  41. Virtual Hand: Modeling • Static hand model: • Once raw data obtained, it's easy to reconstruct a static hand model through the points' connections: • Normals are calculated for all points: vector addition, multiplication(cross), normalization. • Displays rendered in different styles.

  42. Virtual Hand: Modeling • Static hand model: • About 3,000 polygons, • Forearm included, comfortable to look at.

  43. Virtual Hand: Modeling • Static hand model (displayed in different styles):

  44. Virtual Hand: Modeling • Dynamic hand modeling: • Partition the static hand model into parts according to hand joints, • Exert motions at joints, • Integrate the hand parts as a whole hand.

  45. Virtual Hand: Modeling • Dynamic hand modeling: • Partitioning hand (Inventor's symbols used):

  46. Virtual Hand: Modeling • Dynamic hand modeling: • Partitioning hand to 17 hand parts with points numbered: find and record the edge points between adjacent hand parts. Following is the thumb base.

  47. Virtual Hand: Modeling • Dynamic hand modeling: • Hand part movement: find (calculate) rotation axes and points for different movements.

  48. Virtual Hand: Modeling • Dynamic hand modeling: • Now we exert motions at hand joints and end up with “holes”.

  49. Virtual Hand: Modeling • Dynamic hand modeling: • Hole of a hand part: produced by the trajectories of the edge points of the hand part after it rotates to a new place. • How to fill (patch) up the hole? To sample the trajectories to obtain some surface points of the hole.

  50. Virtual Hand: Modeling • Dynamic hand modeling: • Now a hand part at motion is made of two parts: itself and the patch that is used to fill up the hole.

More Related