Subjectivity and Whole Body Interaction

1. Subjectivity and Whole Body Interaction

2. Subjectivity and Whole Body Interaction Peter Wright Art and Design Research Centre C3RI Sheffield Hallam University

3. Acknowledgements • EPSRC Leonardo Network • Research at the intersection of art and HCI • EPSRC LTR Project • Theory and method in experience-centred design • Design 21 My Exhibition Project • Affective communication in product and exhibition design • Sheffield Health Trust • 2nd Life for the Third Age • Hester Reeve, • Ben Heller, Jon Wheat, • John McCarthy

4. Overview • The interesting case of Installation Art • Is it happening? • Theories of Embodiment and experience • Paul Dourish: McCarthy and Wright • Second Life for the Third Age • agency and coupling in virtual space • My Exhibition • Personalisation and ambient interaction • Towards a critical framework for whole body interaction

5. Disclaimer!!

6. The interesting case of Claire Bishop on the History of Installation Art

7. The subject in the art work • “Installation differs from traditional media (sculpture, painting, photography and media) in that it addresses the viewer directly as a literal presence in the space. • Rather than imagining the viewer as pair of disembodied eyes that survey the work from a distance, installation art presupposes an embodied viewer whose senses of touch, smell and sound are as heightened as their sense of vision.

8. But more than this… • The viewer is not just ‘in the art work’ but they complete it • “The spectator is in some way regarded as integral to the completion of the work.” Reiss • They are then at once experiencing the art work and part of it. • The spectator is so integral that “…without having the experience of being in the piece, analysis … is difficult.” Reiss

9. The experiencing subject • If installation art is at pains to offer 1st-hand experience what kind of experience does it offer? • Four modalities of experience that installation art structures for the viewer, each with a different model of the subjective self: • The Freudian self • The phenomenology of perception • Lacan and Barthes beyond the pleasure principle • The activated/politicised subject

10. In general HCI has been slower than Installation Art to escape the gravitational pull of Cartesianism • As a consequence it has under theorised experience and subjectivity • The disembodied mind • The dominance of the eye • and hand • Representation rather than participation • But things are beginning to change

11. Embodiment and Embodied Interaction • Paul Dourish

12. Dourish: Embodied interaction Embodiment is both a physical and a social phenomenon Meaning is constructed in practical acts of engagement with social and physical worlds Objects and people in the world can be understood as affording different forms of engagement Meaning is maintained by a coupling between the agent and the world “Embodied phenomena are those which by their very definition occur in real time and real space” “Embodiment is the property of our engagement with the world that allows us to make in meaningful.” “Embodied interaction is the creation, manipulation and sharing of meaning through engaged interaction with artifacts”

13. Dourish: Space and place • Space is often characterised in Cartesian terms as a container for our action (an immutable backdrop) • But in fact space is constructed in response to human concerns • Place is characterised in terms of human habitation, adaptation, culture but also personal histories and emotional meanings • See also Fitzpatrick on Locales

14. Dourish Key Points • the key things to take from Dourish then are: • embodiment as both a physical and social • participation rather than representation • meaning as constructed through physical/social interaction in place • coupling between the embodied person and the environment • The theory orients towards the situated nature of our interactions with technology but under-theorises the ‘felt life’ of these interactions

15. But more than this… • With his concern for the social, Dourish under-theorises, the subject • While he acknowledges both the physical and the social aspects of embodiment, he does not explore how these inter-relate • We have tried to foreground felt life and the subjectivity of our experience with technology The spectator is so integral that “…without having the experience of being in the piece, analysis … is difficult.” Reiss What kinds of experience of experience does technology structure?

16. Just three themes • A holistic approach to “Felt Life” • where embodied, narrative, emotional and sensual aspects of felt life are treated as equal, interdependent constituents of experience • Continuous engagement and sense making • Wherein the self is the centre of experience, is already engaged in experience and bring to a situation a history of personal, social and cultural meanings, and anticipated futures that structure experience through acts of sense making • A dialogical approach to self • Wherein self, other, object and setting are actively constructed as multiple centres of value (multiple voices) and are always an unfinalised project

17. Sensual our sensory engagement with a concrete situation- palpable visceral Emotional the evaluative relations that unite needs and desires to the particular “that was satisfying, I’m proud of that” Compositional the narrative structure of a situation, before during and beyond “what’s this about, what has happened, what should I do?” Spatial & temporal The quality of space and time, Place, pace, control, A holistic approach The threads in experience

18. anticipating expectation as a continuous process in experience connecting immediate pre-linguistic sense of a situation interpreting understanding the events in terms and how they fit with expectations reflecting making judgements about the experience appropriating making something our own recounting to self and others shapes our understanding and fuels anticipation Interpreting Connecting Reflecting Anticipating Appropriating Recounting Sense making Making sense of the threads

19. Dourish, McCarthy and Wright: Summary • Our embodied interactions are shot through with sensations, emotions, values, ideals, and feelings that are subjectively felt and can be intersubjectively shared • We experience our interactions from a unique position… • …yet our sense of self is constituted in response to the voices (values) of others • Experiences do not come ready made they are actively finished by the subject

20. Second life for the Third Age • Or • “Nelly and the avatars”

21. Nelly and the Avatars • The people • Fall Pre-hab class, 6 people aged 86-91, • No previous experience of computers, WIIs, Playstations or X- Boxes • The technology • Weight sensors on weighing scales controlling a Line Dancing avatar created though 3-D motion capture • The aim • to explore the feasibility of using whole body interaction with such an extreme user group • To assess whether the technology would be experienced positively by both client and therapist

22. Video Excerpt

23. Nelly and the Avatars • Initial impressions • Fall Pre-hab class, 6 people aged 86-91, • no previous experience of computers, WIIs, Playstations or X Boxes • The technology • Weight sensors on weighing scales controlling a Line Dancing avatar created though 3-D motion capture • The aim • to explore the feasibility of using whole body interaction with such an extreme user group • To assess whether the technology would be experienced positively by both client and therapist • The vision • to have a group of people some of who cannot walk, some of whom cannot talk, some of whom cannot move their upper torso dancing together in virtual space.

24. Nelly and the Avatars: Initial Impressions • The emotional response • What a remarkable bunch of elderly people! • They loved it! • They want to do it again • They talked about dancing • Coupling issues • The mapping between user and avatar movements didn’t appear to be self-evident • Remembering the required movements was an issue • The language of movement was confusing • Multiple couplings between feet, weight and avatar movement were confusing

25. My Exhibition

26. Background • Project aims • How can the visceral qualities of interactive images, sounds, lighting and other sensory factors, can be used to help people personalise experiences of exhibition content? • We are designing interactive "zones" in the exhibition which will help people attend to narratives and forms of engagement that will suit them. • Zones will recognise individual visitors, offer choices, record their decisions and actions to build up a record of the visit that will inform the next waypoint about that visitor's interests and guide them onward in their visit. The "ambient" communication, and unencumbered interaction

27. Background • The Froissart Chronicles • The exhibition is centred on a collection of illuminated manuscripts chronicling The Hundred Years War • Each written by the same author - John Froissart, but for different Patrons- French and English.

28. Background • Our Corridor • Our installation will be in the approach corridor and will comprise three zones • Each Zone will present a story • Sir John Chandos • Sieging the Castle • The writing of the Chronicles • When our part of the exhibition is open it will be the visitors first experience of the content

29. Background • Our Corridor • A physical mock-up of part of the corridor has been built in our interaction lab

30. Background • Some themes • The stories offer a wealth of personal and historical perspectives which we have provisionally organised around themes: • Passion, History, Politics, Forensics, The corpse, The chronicler, The curator… • These themes offer a way of organising and potentially personalising content • The also offer the possibility of experiencing multiple points view and multiple voices that characterise the chronicles

31. Interaction Design • We are in the middle of this, we have • A digital infrastructure for controlling lights, video and sound • A database to record personal details, develop personalisation trajectories and to track movement through the space and interaction in it. • We have a interoperable toolkit for rapidly prototyping interactions including RFID, and gesture-based interaction • The system knows who a person is where they are where they have been and what they have seen. • The aim is personalisation of based on who a person is where they have been and what they have already seen

32. Towards a Critical Framework

33. So what would I like to see in a critical framework? • A framework that: • Centres on the experience of interaction not the technology for interaction • Takes as its starting point the richness of the embodied subject • Focuses on the design space but connects to use • Encompasses work, leisure, educational and cultural applications • Can be made meaningful to user researchers, artists, performers, designers and engineers

34. A possible starting point: Hornecker & Buur 2006 • Many one-dimensional viewpoints already exist: • Data-centred viewpoint • Focus on body as a source of data and coupling between body and digital representations • Expressive-Movement Centred viewpoint • Focus on “sensory richness” of expressive movement • Space-Centred view • Focus on the installation and the interaction space (whole body multiple people

35. A possible starting point: Hornecker & Buur 2006 • The framework themes: • Tangible Manipulation (sensual interaction, objects, tactile qualities) • Can users grab, feel and manipulate the important elements • How easy is it to grasp action-effect mappings? • Expressive Representation (what is represented/manipulated,its legibility) • Are representations meaningful and relevant? • Are the representations self-evident in terms of use? • Spatial Interaction (relations of people and objects in space) • Do people and objects meet in meaningful ways? • Is movement meaningful to system and others? • Embodied Facilitation (relations in social space) • Does the physical arrangement encourage/force collaboration? • Does ensemble of representations build on users’ experiences?

36. Looking around • Paul Dourish • Where the action is (MIT Press 2001) • McCarthy and Wright • Technology as Experience (MIT Press 2004) • Hornecker and Buur • Getting a grip on tangible interaction (CHI 2006 Proceedings)