1 / 53

Using Virtual Intelligent Environments to Understand and Enhance Human Motor Learning

Explore the use of Virtual Intelligent Environments to enhance human motor learning. Learn about virtual visual and haptic environments, adaptation, and the advantages and limitations of VIE. Discover a new haptic device for task-specific whole-body movements and incorporating biological parameters in VIE.

zsmith
Télécharger la présentation

Using Virtual Intelligent Environments to Understand and Enhance Human Motor Learning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Virtual Intelligent Environments to Understand and Enhance Human Motor Learning Yoky Matsuoka Division of Engineering and Applied Sciences Harvard University

  2. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES What is “Virtual Intelligent Environment”? • Virtual “Visual” Environment • Virtual “Haptic” Environment • Virtual “Adaptive” Environment

  3. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Virtual Intelligent Environments Haptic environment (sensory input) positional sensors force sensors Haptic environment (motor output) actuators joints I n t e r f a c e Computation control adaptation Visual environment computer generated objects computer generated body

  4. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Ultimate Goal Use virtual intelligent environments to enhance human motor learning ability • Beyond natural capability • Faster

  5. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Examples of Learning Enhancement • Robotic rehabilitation after injuries (stroke) • Movement enhancement (Parkinson’s, CP, etc.) • Electrical stimulation during athletic training • Training surgeons • Virtual training to prevent injuries Courtesy of Krebs and Hogan, MIT Newman Laboratory for Biomechanics and Human Rehabilitation

  6. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Advantages of VIE Integration in Learning Enhancement • Recording Ability • exact execution recorded to be analyzed • No Biological Limitation • no neural delays • Portability • remote training. • Use it to Understand Biological Systems

  7. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Issues with Virtual Intelligent Environments • Limitations in the haptic devices • Not adaptable to the changes in interacting neuromuscular systems

  8. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Haptic Environment Limitations • Currently available haptic device • Workspace is too small for whole-body movements • active and not safe • Difficult to apply appropriate forces during task-specific movements • machine constraints not matching human constraints

  9. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES A New Haptic Device • 6 DOF (3 actuated) • yaw, pitch, linear • Large workspace • 1.1m radius half sphere • Completely passive • Actuated by magnetic particle brakes • Cable driven = no backlash

  10. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES A New Haptic Device • Designed specifically to allow task-specific whole body movement • Software controlled (adaptable to individual’s need) • Designed to be completely scalable.

  11. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES New Haptic Device Controller Current Amplifier DSP Motion Controller Current Motor command Mechanical Device Brakes Desired force/trajectory Encoders Computer Monitor Positional information Visual information

  12. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES New Haptic Device Real-Time Calculations • All interface is in Cartesian coordinates • Jacobians • Cartesian coordinate variables, a, v, x. • Force applied

  13. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Incorporate the Biological Parameters in Virtual Intelligent Environments • Making virtual intelligent environment adaptable to the neuromuscular changes • How does haptic input (force) affect adaptation • for muscles • for central nervous system

  14. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Adaptation to External Force: CNS • Representation of “learned information” • How is it represented? • How does it change over time? • What are the characteristics?

  15. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Internal Model of External Force Field Internal model of force field Fext Plant Baseline conditions of arm control +

  16. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Hypotheses • Modularity in generalization • the learned information transfer to other movements (Shadmehr and Mussa-Ivaldi, 1994; Wolpert et al., 1995). • the generalization is partitioned in the workspace (Gandolfo, et al. 1996). • What domains do generalization occur? • What are the limitations?

  17. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Experimental Environment: Test with Simple Movements • Haptic environment • 2 link planar robot arm that applied force perturbation (F = Bv). • Visual environment • hand cursor and targets displayed on the computer screen. • Processing • recorded the hand location at 100 Hz. Haptic Device Computer Human

  18. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Training Conditions • Subjects were asked to make one continuous movement from one target to another. • Force perturbation is applied perpendicular to the movement. • Distort normal movement with force perturbation = motivates the CNS to produce counterbalancing forces • 200 movements are executed under force perturbation. • Forces are removed to observe the effect learned (aftereffect).

  19. Spatial Generalization Test Early Trials Later Trials Aftereffects

  20. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Directional Generalization Test

  21. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Motor Primitive Position Velocity

  22. Learning Capacity and Size of Primitives Position&velocity Position&velocity Small primitives can represent high frequency fields Large primitives can only represent low frequency fields

  23. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Small Primitives Large Primitives Model aftereffects Actual aftereffects

  24. Large Primitives Overlap 7.5 degrees 45 degrees Middle Trajectory Training Location Interfering Trajectory Training Location Interfering Trajectory Training Location Middle Trajectory Training Location

  25. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Increase in Frequency of Force Field Function Band-limited frequency (1/cm) 285 120 67 Human

  26. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Low Frequency Field Learning Learning Curve for Various Fields High Frequency Field Learning Initial velocity profiles Aftereffects profiles # of peaks in LFF: 2 # of peaks in LFF: 2 # of peaks in HFF: 7 # of peaks in HFF: 1

  27. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Sequential Study: Difference in Aftereffects for High Frequency Field • HFF HFF HFF • LFF MFF HFF • LFF MFF null • LFF MFF UHFF

  28. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Summary of Results • Understanding how the CNS learns to interact with a virtual environment • modular representations in space and direction of the movement • effects sum and negative interference occurs when motor primitives overlap • primitives are large • high frequency components are not learnable but can help retain lower frequency components

  29. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Adaptation to External Force: Muscles • Can we identify the muscle impedance while interacting with a haptic device? • Can we capture the change during learning? Neural Inputs Kp B Ks

  30. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Identify Impedance Learning Strategy in Human • What is the initial strategy used to cope with unknown/unstructured environments? • How does impedance change over time? • After learning, what does the biology pick as the good solution for impedance for a given environment?

  31. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Comparison Between Analytical and Biological Solutions • We can mathematically derive optimal impedance for a linear world. • Biological system converges to the analytical solution. --- great! • Biological system converges to a different solution. --- what and why: put the biological solution back in the equations and reverse engineer. • What about a nonlinear varying world where it is difficult to derive the optimal impedance? • What does the biological system do? Can it be modeled?

  32. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Example: Linear World --- Catching a Ball ball • Goal: Find the “best” impedance. • For this case, find best Khand. • Uncertainty in the world • mball, kball, ball(0), and khand mball kball mball xball kball hand mhand mhand xhand khand khand

  33. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Example: Linear World --- Catching a Ball • Cases: 1. Hand stiffnes (khand) is too high • hand< 0 bounces up 2. Hand stiffness (khand) is too low • xhand > Threshold bottoms out 3. Hand stiffness (khand) is just right • xball xhand until switch is pressed mball xball kball mhand xhand khand 2 3 1 khand 0 infinite

  34. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Example: Linear World --- Catching a Ball • Solve for xhand(t) and xball(t) • initial condition • ball(0) > 0 • xball(0) = 0 • hand(0)= 0 • xhand(0) = 0 mball xball kball mhand xhand khand

  35. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Analytical Linear World to Biological Motor Control • The example relates task performance to limb impedance and optimal solution. • Now measure human strategy…. • “System identification” • Need a new technique

  36. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Existing System Identification Techniques • Time invariant systems --- easy • assume constant m, b, and k over time. • apply external impulse perturbation force. • repeat the same condition and average.

  37. Existing System Identification Techniques • Time varying systems • Cannot apply impulses close to each other. • Need multiple impulses to solve for multiple unknowns. • PRBS (Lacquaniti, et al. 1993)

  38. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES New System Identification Technique to Observe Learning Setup Monitor Robot Processor Force Sensor Handle Data Acquisition System Human Subject Accelerometer

  39. New System Identification Technique to Observe Learning • Use short duration before reflexes • Clean data from force/acceleration sensors • Least square fit or window analysis F k*x b*v m*a m=F/a b= (F-ma)/v k= (F-ma-bv)/x

  40. Testing the New Technique • Phantom robot is used as the perturbation/measurement tool. • Task: balance the moving ball on paddle. • ball moves at constant speed • dies when the ball falls off the paddle • perturbation applied every second

  41. Impedance Change with Learning m change over time k change over time b change over time

  42. Contact Interaction Task • Observe the impedance change within one catch • Observe the impedance change between catches k b ** under development --- pilot studies underway

  43. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Virtual Intelligent Environments Brain Haptic environment (sensory input) positional sensors force sensors Haptic environment (motor output) actuators joints Biological Parameters Internal Model Impedance etc. Computation control neural networks (adaptation) Visual environment computer generated objects computer generated body

  44. Injury prevention for manual material handling workers (recently initiated) YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Example of Learning Enhancement: Application • Typical material handling work: • pick up, carry, place • pick up, toss

  45. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Application: Injury Prevention • Almost 100% of workers complain about some lower back pain after 5 years. • Severe injury occurs and chronic pain starts when a sudden load change occurs. • Nothing • Some verbal instructions on the right posture Current Solutions

  46. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Application: Questions • What is the correct posture that reduces the chance of injuries • Is there a movement execution strategy that is robust under unexpected perturbation? • Can a computer teach these robust movements?

  47. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Application: Training Robust Performance • Training with unexpected perturbations. • pick up and toss the device handle while it applies weight and perturbation. • perturbation: under development • Preventing injuries during training. • let the haptic device act as an assistive device • gravity compensated • if velocity is too high, F= - k(x-xo)

  48. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Application: Measurement of the Outcome • Pickup and throw a sack to a force plate under unexpected perturbation • Performance accuracy (landing location) • Performance strength (landing force) • Lower back load (5-sagittal plane model) • Movement consistency (model)

  49. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Summary • Goal: understand and enhance learning with virtual intelligent environments • Built a large passive haptic device • Investigated the change in the neuromuscular system • internal model • impedance • Applications: injury prevention training

More Related