Trajectory formation in timed rhythmic movements: Patterns, clocks and control

1. Trajectory formation in timed rhythmic movements: Patterns, clocks and control Ramesh Balasubramaniam Sensory Motor Neuroscience Behavioural Brain Sciences Centre University of Birmingham, England. Acknowledgements: Alan Wing Andreas Daffertshofer Andras Semjen.

2. Timing • Ensemble – Neuromuscular apparatus. • Conductor – Clocking mechanism? • Sections – Subsystems. • Metric – Score, perceptual information. • Performance – Prediction, correction • Orchestra

3. How do we synchronize repetitive finger movements to an external beat? Discrete approaches Continuous or Dynamical approaches Synchronization errors Movement trajectories

4. Error/Event based approaches • Mismatches with the external beat: Aschersleben & Prinz, 1995. • Sequential relationships: Wing & Kristofferson, 1973. • Error correction: Vorberg & Wing, 1996; Semjen, Schulze, Pressing. • Statistics of error: short and long term (Chen, Ding & Kelso, 2000; Madison, 2004).

5. Dynamical approaches • No explicit representation of time. • Variability in trajectory over cycles. • Autonomous limit cycle oscillator models: Kay et al (1987, 1991); for review see Schöner (2002). • Jirsa, Foo (2000) oscillators, anchoring. • Timing is an emergent property (Turvey, 1977; Yu, Sternad et al, 2003).

6. “Inbetween” approaches • Implicit vs. Explicit timing (Ivry, Zelaznik & colleagues). • Is drawing circles different from tapping to a beat? • Clock or Emergent behavior? • Can we really have both? • Timing deficits and related inferences.

7. Overview • Interval timing: Knowns and unknowns • Trajectories: What can we learn from discrete movements? • Models of timing • Brain mechanisms • Can we have a grand unified theory of timing?

8. Time time series Paced; 250 - 2000 ms 30 - 50 free responses Event times kymograph Time Index number

9. Response interval fluctuations • Short term • “the hand (or perhaps the will during the interval) cannot be accurately true” • Long term • “rhythmic variation of the standard carried in the mind” Stevens 1886

10. perception production Timer model • Count based • Variance increases with mean Gibbon et al 1984

11. Motor timing Timekeeper • Inter-response interval, I • Two stages • Timekeeper interval, C • Motor implementation delay, D • Independent errors • variances Var(C) Motor system Var(D) Observable response Wing and Kristofferson 1973 Var(I)

12. W-K Model operation Hierarchical open loop C(j) Timer pulses D(j-1) D(j) Overt responses Interresponse intervals I(j) = C(j)+D(j)-D(j-1)

13. Acorr[I(1)] I(j)=C(j)+D(j)-D(j-1) • Interresponse interval • Variance = var(C)+2*var(D) • Lag 1 autocovar = -var(D) • Lag 1 autocorrel = acovar/var = -1/[2+var(C)/var(D)] Wing and Kristofferson 1973 Wing 1980

14. Velocity Position Variability Dynamical systems approach Cannot reproduce negative lag one effect, with additive or multiplicative noise.

15. Problem for modelling • Daffertshofer (1998): minimal autonomous limit-cycle model to reproduce Wing-Kristofferson results would require extremely unrealistic stiffness parameters. • Or two oscillators (forcing functions) • Or two noise sources (Wing-Kristofferson model). • Back to the laboratory.

16. Questions: • What kind of trajectories do we need to produce for timing accuracy? • On what principle are these movements (statistics of) organized and optimized? • How to develop a model (preferably limit cycle) to accommodate what we know about event based responses? • If timing is emergent, what is the informational basis for timed action?

17. Modes of coordination • External metronome. Synchronization: flex on beat Time Syncopation: flex off beat Flex off beat may be achieved through extend on beat

18. Pattern stability: • Extending on the beat (eON) is more stable than flexing off the beat (fOFF) but less stable than flexing on the beat (fON). • Kelso et al, 1998, Nature found that cortical dynamics differentiate synchronization from syncopation promoting case for motor equivalence. • Hence definition of coordination with respect to an external metronomic event should include both task goals (such as synchronize or syncopate) but also motor goals (flexion and extension); For review see Carson (2002).

19. Trajectory symmetry? • Repeated cyclic movement is approximately sinusoidal in form and hence symmetric in the out and back phases (similar movement times and velocities)– inherent assumption in all limit cycle models. • Symmetry in form is found even though the muscle activation required in each phase may be quite different due to dynamic factors such as the effects of gravity and mechanical differences between flexors and extensors. • Independent of finger orientation and oscillation frequency. • Departure from symmetry in timed movements.

20. v flex vext text tflex Paced vs unpaced repetitive movements • Unpaced movements: no metronome. flexion text = t flex & vext = vflex

21. Test of symmetry. • Eight musically trained subjects, 30s trials. • Instructed to flex on, flex off and extend on the beat. (Auditory metronome 1KHz beep) in addition to performing equivalent unpaced repetitive movements. • Finger trajectories sampled and recorded at 200Hz using Qualisys Pro-reflex system. • Intervals of 500ms (2Hz), 750ms (1.33Hz) and 1000ms (1Hz).

22. 2cm/s unpaced velocity -4cm/s position fON fOFF anchoring 0 500ms eON 6cm -3cm 3cm Time velocity position Balasubramaniam, Wing & Daffertshofer, 2004.

23. extension extension .8 .8 flexion flexion flexion .7 .7 .6 .6 .5 .5 Time (s) .4 .4 .3 .3 .2 .2 .1 .1 0 0 1Hz 1Hz 1.33Hz 1.33Hz 2Hz 2Hz 1Hz 1Hz 1.33Hz 1.33Hz 2Hz 2Hz 1Hz 1Hz 1.33Hz 1.33Hz 2Hz 2Hz fON fOFF eON Movement Times In general for fON and fOFF: text >> t flex; and in eON text < t flex Asymmetry reduces with increasing movement frequency.

24. 600 flexion extension 500 400 Velocity (mm/s) 300 200 100 0 synchronize syncopate extonbeat Velocity asymmetry: In general for fON and fOFF: vext < vflex; and in eON vext > vflex

25. Some recent evidence:Spencer, Ivry, Zelaznik, 2003 Science. DT- discrete tapping DC- discontinuous circles CC- continuous circles TT- discontinuous CS- continuous slow IM- intermittent tapping CF- continuous fast

26. CLAIM: Differences between implicit and explicit timing using data from patients with cerebellar damage. Cerebellum (usually implicated in representing timing goals) is less involved in implicit timing tasks such as circle drawing and continuous finger movements. Ivry & colleagues suggest that implicit timing might indeed be an emergent property  is organized by controlling joint stiffness, jerk, spatial noise.

27. 2.5 2.25 2 1.75 Mean squared jerk (mm2s-6) 1.5 1.25 1 .75 .5 .25 1Hz 1.33Hz 2Hz unpaced Movement optimization ? Why make movements that are jerky and more costly to produce?

28. Timing accuracy and movement asymmetry 4 14 3.5 12 3 10 500ms Asymmetry index Count 750ms 2.5 8 1000ms 2 6 1.5 4 1 2 -50 -45 -40 -35 -30 -25 -20 -15 -10 Interval asynchrony Interval asynchrony 4 0 4 -50 -45 -40 -35 -30 -25 -20 -15 3.5 3.5 Asymmetry index Asymmetry index 3 3 2.5 2.5 2 2 1.5 1.5 Count 1 1 -50 -45 -40 -35 -30 -25 -20 -15 0 2 4 6 8 10 12 14 16 18 20 Interval asynchrony (ms) The more asymmetric the trajectory the lower the timing error. Correlation= - 0.69, p< 0.001

29. The correction mechanisms .08 .07 .06 Rel. req. .05 .04 Correlation = -0.63, p< 0.001 .03 .02 RelAsynchrony .01 .4 .4 0 -20 -15 -10 -5 0 5 10 15 20 RelExtension .2 .2 Slow phase (t+1) 0 0 -.2 -.2 Count -.4 -.4 -.6 -.6 -.8 -.8 -20 -15 -10 -5 0 5 10 15 20 0 50 100 150 200 250 300 350 400 450 500 Relative Asynchrony (t) Balasubramaniam, Wing & Daffertshofer, 2003.

30. What does this mean for timing models? • Existence of > -0.5 correlation suggests closed loop control or feedback mechanisms between cycles (Pressing, 1998, Psych Rev) • High velocity movements towards the target provide perceptual information relevant to phasing and slower return phase accommodates period adjustment. Jerk is useful for timing. • More of this when we discuss timing in force fields. • Movement trajectories contribute to movement timing.

31. What else can asymmetry do? Changes in anticipation of step length change Balasubramaniam, Wing & Semjen, 2004.

33. Asymmetry index Interval

34. Predictive timing? • Can trajectories change in anticipation of change in step length? • How is this voluntary transition made? • Is this evidence for co-articulation? • What governs prediction in timing? • How to test this?

35. What about the cerebellum? • Case study. • SH is a left-handed man who was 35 at the time of testing. Three years previously he had suffered a unilateral left cerebellar hemisphere haemorrhage from an arteriovenous malformation.

36. impaired unimpaired unimpaired hand :Movement asymmetry is exhibited & correction mechanisms between cycles close to controls. Impaired hand: Trajectory is unstable and symmetric No correction mechanisms present (as revealed by correlations) .

37. impaired unimpaired

38. SH appears fine with circle drawing. • No correlational effects were seen between cycles in circle drawing • Is the Ivry argument partially correct? • However the minimisation of jerk based criterion is most probably not true. • No statistical differences between control subjects and SH on mean squared jerk for circle drawing.

39. Phantom Robot Arm (Sensable Technologies, Cambridge MA) Timing in force fields • Changes in trajectory due to loading • How does it affect timing? • Can error correction still be performed reliably? Balasubramaniam, Wing & Barbieri-Hesse, 2003.

40. Force fields: • Virtual limb-wrist pendulum (Kugler & Turvey, 1986; Yu & Sternad, 2002) effector system or early unloading experiments by Feldman (1980). • Grip force adjustment (Flanagan & Wing). • Reaching (Shadmehr et al). • Speech production (Ostry et al, 2003). • Spring (elastic –position based), Viscous (velocity based), Inertial (acceleration based): resistive and compliant forces can change the dynamic environment thus upward and downward velocities in finger oscillations. • What about time dependent fields ?(Mussa-Ivaldi, Karniel et al)

41. Position dependent force field

42. Position- spring effects • Trajectory is not smooth (even after several training trials). Possibly due to mechanical characteristics of the system. • But no timing effects. Suggesting that the spring force load did not challenge the system’s ability to predict and correct. • Increase in stiffness does not contribute to timing error.

43. Velocity dependent (viscous) force field.

44. Velocity • Subjects found it very difficult to achieve timing accuracy. • Intervals became unstable (errors accumulate over time) • In continuation trials, no negative autocorrelations were found. • Trajectory asymmetry also compromised.

45. Possible limit cycle model? ………….

48. Problems: Too many free parameters. Might still require a forcing function. Fminsearch is futile.

49. Model produces bad to modest fits……….

50. Interval timing: more unknowns than knowns • Trajectories: What have we learned from discrete movements? Same principles might not be effective. • Can we have a grand unified theory of timing?