1. Tracking

2. Tracking • Tracking is following the state of an entity over time • What is state? • Sensors collect data = F(H(state))+ noise • H is how the measurement is related to state • F is the measurement function itself • Most often “sampled” (i.e. discrete, k, k+1, k+2…) at some rate and resolution • Trackers approximate true state from sensor data • Calibration to determine unknown parameters of F, H • Data integration to determine state • Filter to reduce noise (e.g. Kalman Filter) • Sensing and Tracking take finite non-zero time to perform

3. Tracking example • Following the position of a falling object with an accelerometer • Acceleration sensor sends a value • Value must be some function of our state + some error • Data = F(H(state))+noise • What is H? • How to calibrate? (Determine F) • How to find position? (Determine state) • How to remove noise?

4. Tracking Systems • What we are usually concerned with as VR Scientists/Engineers • Choice based on application • “no silver bullet” • Application Requirements • Application Constraints

5. Tracking for VR Applications • Enable the user to influence the virtual space • Locomotion • Interaction • Control • Tracking system influences • Usability • Presence • Task performance • The single biggest “problem” in VR/AR/MR • “Pose” is the most commonly tracked state

6. Y Z X Pose Tracking • Track position and orientation of a rigid object with respect to another coordinate system. • Why rigid? Y Z X Object Coordinates Object Pose Tracker Coordinate System

7. Degrees of freedom • The amount of pose information returned by the tracker • Position (3 degrees) • Orientation (3 degrees) • There are trackers that can do: • only position • only orientation • both position and orientation

8. Question • Given that I want to track your head, I attach a new tracker from NewTracker Corp. it returns 6 degrees of freedom (6 floats). What questions should you have? • What are some evaluation points for a tracking system?

9. Tracking Performance • Accuracy • Difference between an object’s pose and the reported pose • Resolution • Granularity that the tracking system can distinguish individual points or orientations • Jitter • Change in reported position of a stationary object (Gaussian noise) • Drift • Steady increase in error with time

10. Tracking Performance • t0– time when object is at point p • t1– time when sensor reports p • Lag or Latency – t1 -t0 • What causes latency? • Sensor acquisition time • Sensor transmission time • Tracker data processing time • Tracker transmission time • Filtering

11. Tracking Performance • Update Rate • Discrete sampling of time-continuous state into k measurements / second, or • Hz, KHz, MHz, GHz • Result of sensor, bandwidth limitations • Higher is better • Why?

12. Environmental Factors • Interference - external phenomenon that degrades system’s performance • Occlusion • Tracking space distortions • Echos

13. Mass, Inertia and Encumbrance • VR Users already burdened by displays • Trackers often add more weight, inertia, and wires, and other attachments

14. Working Volume • What is the shape? • Frustum • Hemisphere • Performance may change with object state • E.g. distance from sensor often decreases performance exponentially

15. Multiple objects • Number of potentially tracked points • Unique • Simultaneous • Difficulties • Sensing interference • Multiplexing (lower update rate) • More processing/transmission time

16. Cost • Monetary • You generally get what you pay for. ($30-$100k+) • Consumer gaming market lowering price/performance ratio • Space • For same working volume, some tracking systems require more physical space. • Setup • How long does calibration take? How long does it last? Can it be performed by an end-user? Does it require special equipment? How difficult is it?

17. Performance Accuracy Resolution (precision) Jitter (zero mean) Drift (non-zero mean) Lag Update Rate Environment Interference Mass, Inertia and Encumbrance (wires) Space (Range) Number of tracked entities Cost Monetary Setup Space Evaluation Criteria Which of these are most important?