1. KINECT REHABILITATION Stroke Therapy Research Kathryn LaBelle

2. RESEARCH TOPIC Can the Kinect’s joint-tracking capability be used in clinical and in-home stroke rehabilitation tools?

3. OUTLINE • Background • Stroke Therapy • Kinect • Potential of Kinect in Rehabilitation • Research Questions • Software • Data Gathering • Data Analysis • Conclusions

4. STROKE THERAPY • Stroke survivors can experience: • restricted movement • loss of sense of balance • decreased strength • Regained through physical therapy • balance exercises • range of motion activities • coordination practice

5. MICROSOFT KINECT • Developed for the Xbox 360 gaming console • Tracks your movements: you are the controller • Sensors • Depth Camera and Sensors • RGB Camera • Microphone array • Motorized base

6. DEPTH IMAGING • Infra-red projector shines grid of light on the scene, encoded with data. • Light bounces off objects in the scene. • Kinect light sensors receive reflected light. • By analyzing time of flight and distoritions in the encoded data, the Kinect makes a depth map of the scene.

7. JOINT TRACKING ALGORITHM • Input: depth map • Machine learning algorithm • Collected recordings of people using the Kinect • Joint positions marked by hand • Algorithm was fed this “training” data and learned how to correctly identify joints from a depth image • Output: x, y, z joint positions

8. JOINT TRACKING AND STROKE REHABILITATION • Clinical applications: • assess patients’ performance • track patients’ progress • pinpoint areas for improvement • At-home exercise aids: • provides constructive feedback to patients • give encourgement and motivation • generate summary reports for doctors

9. RESEARCH QUESTIONS • What SDKs and drivers are available for use with a PC? • What type of information can be obtained? • What is the quality of the joint data obtained from the Kinect? • Sampling rates • Consistency • How resilient is the Kinect’s joint data and performance to variation in testing conditions? • What functionality could be provided in a stroke therapy application that uses the Kinect?

10. SDK COMPARISON

11. SOFTWARE DEVELOPED • Display depth video and skeleton • Joint positions and instantaneous frames per second written to file • Balance board integration • Record depth stream to file • Obtain joint positions from recording

12. DATA GATHERING

13. DATA ANALYSIS • Sampling rates of joint position data • Identifying phases of movement from joint positions • Consistency and stability of joint positions

14. SAMPLING RATE

15. IDENTIFYING PHASES OF MOVEMENT

16. DATA STABILITY Standard Deviation of Joint Positions while Subject is Motionless

17. DATA STABILITY: Assisted Tests • Clinical therapy often involves an assistant supporting a patient while he performs exercises • Test procedure: • subject begins by sitting alone • assistant joins, putting hands on subject’s shoulders • subject stands up

18. DATA STABILITY: Assisted Tests

19. DATA STABILITY: Assisted Tests

20. DATA STABILITY: Assisted Tests

21. CONCLUSIONS • OpenNI Framework and Microsoft SDK for Windows are best tools to use • Can provide significant functionality in a joint-tracking application • track and record joint positions in three dimensions • display image of tracked joints in real time • integrate Kinect with the Wii balance board • Sampling rate exceeds acceptable level • Phases of movement are easily identifiable from graphs of joint positions • Joint position stability is more than adequate with one subject in view • Skeleton merging could pose a problem for clinical use of Kinect

22. FUTURE WORK • Deeper investigation into assisted exercises • Different types of exercises • Position the assistant differently • Determine conditions causing skeleton merging • Further development of software • Investigate applications in other fields of physical therapy

23. Questions?