1. Classification and Clustering of Brain Pathologies from Motion Data of Patients in a Virtual Reality Environment Via Machine Learning Uri Feintuch, Hadassah- Hebrew University Medical Center Larry Manevitz, University of Haifa, Natan Silnitsky, University of Haifa Data from: Assaf Dvorkin, Northwestern University Neuro computation laboratory day, December 2011

2. Outline • Rehabilitation of Patients of Brain Pathologies • Virtual Reality (VR) in Rehab • Research Goals • Techniques Used • Experiments • Architecture and Training • Results • Future directions

3. Brain pathologies • CVA - CerebroVascular Accident (Stroke) • Hemispatial neglect • TBI - Traumatic Brain Injury

4. Rehabilitation • Diagnosis • Differential Diagnosis (e.g., Neglect vs. Hemianopsia) • Evaluation • Severity of deficit • Progress during intervention

5. Example - Neglect

6. Traditional tools • Star Cancellation

7. and their shortcomings… • HD applied for and received back his driver’s license, having shown intact visual fields at Perimetry and no signs of neglect. • HD scored 143/146 on his BIT test. • Since obtaining the license, however, he was involved in 9 car accidents, all concerning the left side of his car. (Deouell, Sacher & Soroker, 2005)

8. VR in Rehab (1) • Virtual Mall The way the patient views himself within the virtual environment A camera which films the patient and a monitor which displays her

10. VR in Rehab • Replaces traditional methods • Ecological validity • Safety • Absolute control of stimuli

11. VR in Rehab (2) • Assaf Dvorkin, Rehabilitation Institute of Chicago, Northwestern Uni. A target in the field of view The VRROOM haptics/graphic system

12. VR in Rehab - Challenges • Human behavior is very complicated • Vast amount of information • geometry or physics formula (?) • Simplistic analysis (e.g., RT, % Errors)

13. Proposed solution • Apply Machine Learning tools. Such tools may detect patterns of behavior performed within the Virtual Environment. • In this work we used Artificial Neural Networks (NN) Classifiers ,SVM, SOM and k-means.

14. Research Goals • Identify and differentiate between meaningful clinical conditions • Scarce data • Perhaps noisy • Broad spectrum conditions like neglect • Mild, severe • Use unsupervised learning approach

15. Machine Learning Techniques Used • Supervised Learning • Backpropogation NN • SVM • Unsupervised Learning • Kohonen • K-means

16. 2D Experiment • Population: 54 HA, 11 CVA (without neglect), 9 TBI, 25 HC. • Data Encoding: Vector of hand movement (dx,dy,dt)

17. NN Architecture for 2D Output Layer Hidden Layer …(Full connectivity)… dt dx dy dt dt dt dt dx dy dx dy dx dy dx dy Data point (t+1) Data point (t+2) Data point (t+3) Data point (t+4) Data point (t) Input Layer

18. NN Architecture for 2D (TBI vs. CVA) dy dx dy dx dy dx dy dy dx dx dt dt dt dt dt Output Layer …(Full connectivity)… …(Full connectivity)… Data point (t+1) Data point (t+2) Data point (t) Data point (t+3) Data point (t+4) Input Layer

19. Training for 2D • Levenberg-Marquardt • resilient back-propagation • 300 epochs • Cross-validation

20. 2D Results – 2 class

21. 3D Experiment • Population: 9 H, 9 N, 10 S • Data Encoding: Vector of movement (x,y,z,t) • Only trials where movement occurred at all • Phases • Long vector: Entire trial from appearance of stimulus (includes pre-movement data) • Movement: Vector only from commencement of movement • Initial/Final segment – beginning/end of movement

22. NN Architecture for 3D • 1400 elements for a long vector (1400-5-1) • 1000 elements for a movement vector (1000-5-1) • 130 elements for initial/final vectors (130-5-1)

23. 3D Data Set • Population of Healthy, Neglect, Stroke • Movement Vectors (x,y,z,t) of different lengths • Also tested on “snippets” for cross platform • Resilient back-propagation • 50 to 300 epochs

24. 3D Results – 2 class

25. 3D Results – 2 class

26. Clustering for 3D • Kohonen Self Organizational Map (SOM) • Reproduced with K-means

27. Clustering for 3D

28. Clustering for 3D • 2 Neurons • 7 Neurons • 200 Neurons

29. 3D Results – 0 class • Movement Vector, Neglect, (7 clusters) • Movement, Healthy/Neglect, (7)

30. 3D Results – 0 class • Movement Vector, Neglect/CVA , (7 clusters)

31. Clustering for 2D Kohonen Self Organizational Map (SOM) Reproduced with K-means

32. 2D Results – 0 class • Healthy/CVA, (7 clusters) • -> …

34. 3D expriment - 1 class • Architecture • Movement vectors – 1000-200-1000 • initial/final vectors - 130-26-130

35. 3D expriment - 1 class • Architecture • Movement vectors – 1000-200-1000 • initial/final vectors - 130-26-130

36. 3D expriment - 1 class • Threshold choice

37. 1 class results for 3D

38. 1 class results for 3D • Neglect classifier for "Left targets trials only" - 62% • Non-Mild Neglect classifier for "Left targets trials only" - 83%

39. Combined Platforms • Merging small samples from different platforms. But…

40. Combined Platforms – 2 class • (x,y,z) -> (x,y,0) • "snippets" • Experiments different data amounts

41. Combined Platforms – 2 class

42. Summary of results • 2D experiment – Differential Diagnosis: • Healthy vs. Patients • TBI vs. CVA • 3D experiment – DD + Evaluation • Neglect vs CVA • Clusters by severity • 1 class classifiers (Severe Neglect)

43. Future Work • Merging data across platforms • Automatic Prognosis and Individualized Treatment Protocols • Construct models of patients with their movement restrictions • Run potential rehab protocols on the model • Prognosis: via best results on model • Apply best protocol to the patient

44. Acknowledgment • Assaf Dvorkin • Jim Patton • Eugene Mednikov • Debbie Rand • Rachel Kizony • Neta Erez • Meir Shahar • Patrice L. Weiss • The Caesarea Rothschild Institute