Optical Motion Capture

1. Optical Motion Capture Bobby Bruckart Ben Heipp James Martin Molly Shelestak

2. Outline • History • Types of Motion Capture • Development of Motion Capture • State-of-the-Art • Hardware / Software • Data Collection Algorithms • Pros & Cons • Future • Possible solutions to current problems • Possible future paths

3. History Types of Motion Capture Development of Motion Capture State-of-the Art Future Mechanical Motion Capture Optical Motion Capture Electromagnetic (magnetic) Motion Capture Types of Motion Capture

4. Mechanical Motion Capture • performer wears a human-shaped set of straight metal pieces • other types: gloves, mechanical arms, articulated models • pro: • no interference from light or magnetic fields • con: • the technology has no awareness of ground • equipment must be calibrated often. • does not know which way the performer's body is pointing • absolute positions are not known but are calculated from the rotations

5. Optical Motion Capture • performer wears reflective dots that are followed by several cameras and the information is triangulated • markers are either reflective, or infra-red emitting • developed primarily for biomedical applications • pro: • performer feels free to move due to no cables connecting body to the equipment • larger volumes possible • more performers are possible • very clean, detailed data • con: • it is prone to light interference • occlusion • rotations of body parts must be solved for and are not absolute • performer must wear a suit with dots and balls • information has to be post-processed or 'tracked' before viewing • comparatively high cost

6. Electromagnetic Motion Capture • performer wears an array of magnetic receivers which track location with respect to a static magnetic transmitter • First use: military • often layered • pro: • positions are absolute, rotations are measure absolutely • can be real-time • relatively cheaper than optical • con: • magnetic distortion • data prone to noise and interference • performers wear cables connecting them to a computer • relatively low sampling speed

7. History Types of Motion Capture Development of Motion Capture State-of-the Art Future Development

8. Early 1980s – Beginning of optical systems • MIT Architecture Machine Group & New York Institute of Technology Computer Graphics Lab experimented with optical tracking of the human body • few cameras and markers with limited success. • created virtual stick-figure marionette

9. 1985-1988 Jim Henson Productions • ability to control the position and mouth movements of a low resolution character in real-time • Waldo C. Graphic

10. 1988: deGraf/Wahrman “Mike the Talking Head” • shows off the real-time capabilities • driven by a specially built controller • The Silicon Graphics hardware provided real-time interpolation between facial expressions and head geometry as controlled by the performer

11. 1989 Kleiser-Walczak • Dozo • used optically-based solution from Motion Analysis (multiple cameras to triangulate the images of small pieces of reflective tape placed on the body) • resulting output is the 3-D trajectory of each reflector in the space

12. 1992 - SimGraphics • facial tracking system called a "face waldo.“ • could track the most important motions of the face and map them in real-time onto computer puppets • Important: one actor could manipulate all the facial expressions of a character by just miming the facial expression himself

13. SIGGRAPH 1993: Acclaim • realistic and complex two-character animation done entirely with motion capture • developed a high-performance optical motion tracking system able to track up to a 100 points simultaneously in real-time • used the system to generate character motion sequences for video games

14. History State-of-the Art Hardware / Software Data Collection Algorithms Pros & Cons Future Data Acquisition Optical Motion Capture

15. History State-of-the Art Hardware / Software Data Collection Algorithms Pros & Cons Future Data Translation Optical Motion Capture

16. History State-of-the Art Hardware / Software Data Collection Algorithms Pros & Cons Future Data Implementation Optical Motion Capture

17. History State-of-the Art Hardware / Software Data Collection Algorithms Pros & Cons Future Cameras placed around center of room Recording at 320x240 @ 15fps Cameras calibrated into common coordinate system Voxel-based volume rendered through a space-carving technique Also uses color-consistency to enhance quality Data Collection Algorithms

18. History State-of-the Art Hardware / Software Data Collection Algorithms Pros & Cons Future Run a quick test to determine if the point lies on, in, or outside the shape Moment analysis is then used to determine other information about the point The fitting error is then found A split and merge technique is used to recreate the volume Data Collection Algorithms

19. History State-of-the Art Hardware / Software Data Collection Algorithms Pros & Cons Future The whole voxel volume is approximated as one ellipsoid It is subdivided into two ellipsoids so D is less than some threshold value A new voxel set is created from a pairing of voxel sets and neighboring sets A novel ellipsoid is paired with this new set D is computed, the smallest D value is used to replace the two ellipsoids its replacing This is performed only at the first time step to reduce error in merging Data Collection Algorithms

20. History State-of-the Art Hardware / Software Data Collection Algorithms Pros & Cons Future Problems arise when there are differing numbers of ellipsoids from one time step to the next ellipsoid sets are merged Data Collection Algorithms

21. History State-of-the Art Hardware / Software Data Collection Algorithms Pros & Cons Future To identify rigid body structures, it must be known that the mutual Euclidean distance between two points will not change if they are in the same rigid body The paths through the time steps are compared in an iterative fashion until all ellipsoids are handled Data Collection Algorithms

22. History State-of-the Art Hardware / Software Data Collection Algorithms Pros & Cons Future Two adjacent rigid bodies found in the previous step are used to sample points along any of the three major axes which lie on this ellipse The number of point samples increases along all three axes simultaneously until the average 3D location of these points lies within the adjacent ellipse (or vice versa) Data Collection Algorithms

23. History State-of-the Art Hardware / Software Data Collection Algorithms Pros & Cons Future Pros Pros of Optical Motion Capture

24. History State-of-the Art Hardware / Software Data Collection Algorithms Pros & Cons Future Cons Cons of Optical Motion Capture

25. History State-of-the Art Future Possible solutions to current problems Possible future paths Geometric Dissimilarity Make human-shaped data work on one of these characters, without introducing effects of mismatched proportions Current Issues

26. History State-of-the Art Future Possible solutions to current problems Possible future paths Different movement qualities in performers and cartoon characters Ability to adjust the speeds of capture to simulate more cartoon-like movements when capturing natural human movement Current Issues

27. History State-of-the Art Future Possible solutions to current problems Possible future paths Increasing data collection ability Make motion capture easier to process with increased technology in the current software Current Issues

28. History State-of-the Art Future Possible solutions to current problems Possible future paths Increased number of performers which can be captured simultaneously Ability to capture groups of performers with out reducing the image quality Current Issues

29. History State-of-the Art Future Possible solutions to current problems Possible future paths ·enhancement of performance conditions through lack of tethering and simplification of performance apparel ·increased speed of the technology ·increased 'volume' or area in which performances can be captured ·lower cost, so that consumers and independent artists can have access & experiment / expand the technology Future Paths

30. History State-of-the Art Future Possible solutions to current problems Possible future paths ·increased accuracy of the results, including improved physical abilities, so that characters can touch each other and feet meet solidly on the ground ·greater ability to capture data from multiple characters ·combination of virtual reality and existing motion capture technologies could aid in technological development Future Paths

31. Discussion Questions • Should using motion capture for animation be considered art? • Could motion capture be used to create realistic human motion in robotics? • What would be some other ways to capture motion? Audio? • Four