1 / 19

Programming with the Kinect for Windows SDK

This overview delves into the Kinect for Windows SDK, covering essential components such as Kinect sensors, depth data processing, and skeletal tracking. Learn how to utilize RGB and depth cameras, manage audio input for speech recognition, and view player data through extensive demos. We will explore depth calculations for pixel distances, joint tracking for up to two players, and techniques for noise reduction in skeletal data. Additionally, discover resources and sample grammars for enriching interactive experiences with audio input.

hisa
Télécharger la présentation

Programming with the Kinect for Windows SDK

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Programming with the Kinect for Windows SDK

  2. What we’ll cover • Kinect Sensor • Using Cameras • Understanding Depth Data • Skeletal Tracking • Audio

  3. 3D DEPTH SENSORS RGB CAMERA MOTORIZED TILT MULTI-ARRAY MIC

  4. Demos Displaying RGB & Depth Camera Data

  5. Depth Data • The distance from the Kinect for every pixel • Optionally includes Player information

  6. Understanding Depth Data • ImageFrame.Image.Bits • Array of bytes - publicbyte[] Bits; • Array • Starts at top left of image • Moves left to right, then top to bottom • Represents distance for pixel

  7. Calculating Distance • 2 bytes per pixel (16 bits) • Depth – Distance per pixel • Bitshiftsecond byte by 8 • Distance (0,0) = (int)(Bits[0] | Bits[1] << 8); • VB (int)(CInt(Bits(0)) Or CInt(Bits(1)) << 8); • DepthAndPlayerIndex – Includes Player index • Bitshift by 3 first byte (player index), 5 second byte • Distance (0,0) =(int)(Bits[0] >> 3 | Bits[1] << 5); • VB:(int)(CInt(Bits(0)) >> 3 Or CInt(Bits(1)) << 5);

  8. Depth Reference • Distance Range: 850 mm to 4000 mm range • Depth value 0 means unknown • Shadows, low reflectivity, and high reflectivity among the few reasons • Player Index • 0 – No player • 1 – Skeleton 0 • 2 – Skeleton 1 • …

  9. Demos Working with Depth Data

  10. Skeleton Data

  11. Maximum two players tracked at once Six player proposals Each player with set of <x, y, z> joints in meters Each joint has associated state Tracked, Not tracked, or Inferred Inferred - Occluded, clipped, or low confidence joints Not Tracked - Rare, but your code must check for this state Joints

  12. Joint Smoothing • Use to remove joint “noise” • Small, high frequency jitter • Temporary Spikes • nui.SkeletonEngine.TransformSmooth = true; • Fine tune using TransformSmoothParameters • Correction, JitterRadius, MaxDeviationRadius, Prediction, Smoothing

  13. Demos Skeletal Tracking

  14. Audio • Kinect as a microphone • Kinect for Speech Recognition

  15. Engagement model Mic indicator for speech-enabled menus Keyword to engage Feedback and confirmation(both passive and active) Multi-modal Feedback

  16. Speech Recognition • Kinect Grammar available to download • Grammar – What we are listening for • Code – GrammarBuilder, Choices • Speech Recognition Grammar Specification (SRGS) • C:\Program Files (x86)\Microsoft Speech Platform SDK\Samples\Sample Grammars\

  17. Demos Audio

  18. Resources • Download • http://research.microsoft.com/kinectsdk • Kinect Programming Walkthroughs • http://research.microsoft.com/kinectsdk/ • Coding4Fun Kinect Toolkit • http://c4fkinect.codeplex.com • Kinect SDK Quickstarts • http://channel9.msdn.com/series/KinectSDKQuickstarts

More Related