MotionComposer - the search for coherence in movement and sound

1. MotionComposer- the search for coherence in movement and sound Robert Wechsler Workshop/Roundtable McGill University, Feb.29-Mar.2, 2012 www.motioncomposer.de this power point may be downloaded from: www.palindrome.de/ppt

2. Überblick Mercredi 1 – 6 workshop Jeudi 1 – 6 workshop Vendredi 2 – 3 presentation 3 – 5 roundtable www.motioncomposer.de this power point may be downloaded from: www.palindrome.de/ppt

3. Überblick - Workshop (both days) introductions demonstration 3 talks - 40 min ea. discussion after each 20 – 40 min. project design and execution project presentation, de-briefing www.motioncomposer.de this power point may be downloaded from: www.palindrome.de/ppt

4. Überblick – 3 talks over 2 days. who am i? interactive performance motion tracking mapping and synaesthesia the motioncomposer: an intuitive expression device for persons with disabilities (Bauhaus Universität) www.motioncomposer.de this power point may be downloaded from: www.palindrome.de/ppt

5. me • dancer, choreographer, developer • New York, Nürnberg, Stuttgart, Weimar • Palindrome Dance Company • interactive since 1975 • developed EyeCon software with Frieder Weiss 1995-2005 • teach motion tracking • MotionComposer project director, Bauhaus University www.palindrome.de/ppt Part 1: me

6. influences New York City in the 1980's www.palindrome.de/ppt Part 1: me

7. John Cage Merce Cunningham

9. Interactive what activity /response activity /response activity response www.palindrome.de/ppt Part 2: Interactive Performance

10. interactive performance is nothing new Cave drawings, Spain ca. 4000 BC

11. mechanical technology was used to empower performers Dionysis Thr. Athens - 300 BC

12. Louis XIV - sun king and megalomanic

13. Louis loved mechanical devices, toys, remote acting devices (automation)

14. He also loved cannons, the ultimate amplifier of gesture.

15. Interactive devices were used by New York performance artists in the 60's and 70's Robert Rauschenberg - "Pelican" 1963 Laurie Anderson - "Walking Solo for Tape Bow Violin" 1979

16. Interactive Performance today Interaction Design Gesture Sonification Motion Capture Virtual Environments Immersive Environments Haptic responders Diversified Performance (using internet) and and and www.palindrome.de/ppt Part 2: Interactive Performance

17. Part 3: Motion Tracking

18. motion tracking vs. motion capture motion capture • highly accurate, expensive (ca. 1m €) • generally not realtime • used for data collection (research), making realistic human or animal animations (films, games, etc.) motion tracking • less data, less equipment, less cost (ca. 1k-2k €) • concerned with motion qualities like dynamic, direction of motion • real time www.palindrome.de/ppt Part 3: Motion Tracking

19. motion capture • tracks location of fixed positions on body with reflective markers • 12-24 cameras, each lens is ringed with infrared lights www.palindrome.de/ppt Part 3: Motion Tracking

20. motion capture • Viconis a leading company in motion capture

21. motion capture

22. motion capture

23. motion capture

24. real-time mo-capture • performances with interactive animated characters: "No DNA" (http://www.nodna.com) • dance pieces on stage • Motione (2003-5) Arizona State U. (http://ame.asu.edu/motione) • (Trisha Brown, Kaiser, et.al.) • "22" (Bill T. Jones, Kaiser , et.al.) • Luc Vanier et al. at the interactive performance facility U of Wisconsin at Milwaukee (http://www.isl.uiuc.edu/Publications/final%20dance1.pdf) www.palindrome.de/ppt Part 3: Motion Tracking

25. motion tracking sensors • body-oriented • environment-oriented www.palindrome.de/ppt Part 3: Motion Tracking

26. motion tracking sensors • body-oriented • body-worn electrodes • EEG • ECG • EMG • Touching • joint bend-sensors • accelerometers, other body-worn devices • environment-oriented • video camera-based (eg. eyecon) • infrared camera-based • floor sensors, wall sensors, etc www.palindrome.de/ppt Part 3: Motion Tracking

27. Part 4: Mapping and Synaesthesia

28. Synaesthesia Dostoyovsky Kandinsky www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

29. Dance + Music = ? www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

30. sound movement Where , how do they connect? How can we build up this connection? What is Mapping? www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

31. sound movement What possibilities does the technology allow? www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

32. e.g. sound e.g. movement sensor device a-to-d conversion d-to-a conversion output device processing analogue Digital analogue

33. What is Mapping? makes sound movement www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

34. What is Mapping? makes music dance www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

35. What is Mapping? dance music www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

36. What is Mapping? dance music www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

37. What is Mapping? dance music correlation?

38. What is Mapping? correlation dance music turns on turns off toggle iterate raise lower complex continuous variables

39. What is Mapping? dance correlation music turns on turns off toggle iterate raise lower complex continuous variables

40. What is Mapping? dance correlation music

41. What is Mapping? Number of mappings (M) = number of Input parameters (I) x number of Relationships (R) x number of Output parameters (O) M = I x R x O www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

42. There are thousands of possibilities. Most are intelligible. few are palpable. www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

43. Coherent Mapping -- movement parameters

44. Coherent Mapping -- movement parameters spatial / temporal

45. Coherent Mapping -- movement parameters / spatial

46. Coherent Mapping -- movement parameters / spatial

47. Coherent Mapping -- movement parameters / spatial

48. Coherent Mapping -- music parameters / time and timing 1 discrete events

49. Coherent Mapping -- music parameters / time and timing 1 discrete events Accents Stops Starts Other instaneous changes www.palindrome.de/ppt Part 4: Mapping and Synaesthesia

50. Coherent Mapping -- music parameters / time and timing 2 Tags www.palindrome.de/ppt Part 4: Mapping and Synaesthesia