1 / 115

Alignment in human-human and human-computer interactions

Alignment in human-human and human-computer interactions. Holly Branigan Jamie Pearson University of Edinburgh. Collaborators. University of Edinburgh Martin Pickering Stanford University Clifford Nass John Hu. Outline. Our approach Features of Human-Human interaction (HHI)

benjamin-bird
Télécharger la présentation

Alignment in human-human and human-computer interactions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Alignment in human-human and human-computer interactions Holly Branigan Jamie Pearson University of Edinburgh

  2. Collaborators University of Edinburgh • Martin Pickering Stanford University • Clifford Nass • John Hu

  3. Outline • Our approach • Features of Human-Human interaction (HHI) • Audience design • Alignment • Patterns of alignment • Explanations for alignment • Relevance to HCI • Methodology for studying HCI • Data from experiments investigating lexical/syntactic alignment in HCI

  4. Our approach • Psycholinguistic accounts of HHI: • Characterising linguistic behaviour • Identifying underlying psychological mechanisms • Focus on lexical and syntactic processing • Applying to HCI: • Predicting human (user) behaviour • Simulating human behaviour • Modifying human behaviour

  5. Features of HHI • Linguistic behaviour in HHI is highly flexible and adaptive: • An individual speaker’s behaviour is partially contextually-determined. • Linguistic choices (semantic, lexical, syntactic, phonological..) choices may vary according to context. • Relevance to HCI: • Features of HCI communicative context may influence linguistic behaviour.

  6. Determinants of Behaviour • Influence of addressee on speaker: • Indirectly – speaker’s beliefs about addressee’s state of knowledge, interests etc • Audience Design • Directly – via addressee’s own linguistic behaviour • Alignment • Both influences may be active simultaneously

  7. Audience Design • Tailoring utterance to fit addressee: • Their knowledge, interests, beliefs etc • May be based on speaker’s a priori beliefs or assumptions about addressee • Or may be based on direct evidence from addressee • E.g., addressee’s feedback

  8. Audience Design – lexical choice • Fussell & Krauss (1992): • speakers adjust their choice of lexical terms to fit the assumed knowledge of their addressees: • Proper names when addressee likely to know referent: Clint Eastwood • More detailed descriptions when addressee less likely to know referent: Media mogul, was married to Jane Fonda, owns CNN, has a moustache and grey hair • Assessment of addressee knowledge is based on a priori judgements about social distribution of knowledge • movie stars > industrialists

  9. Audience Design – lexical choice • Isaacs & Clark (1987): • A priori assessment of addressee knowledge can be rapidly adjusted on basis of direct evidence from feedback • Lexical choices then reflect this adjusted assessment • ‘experts’ in a domain (New York landmarks) quickly adjust their choice of lexical terms to fit the apparent expertise of their addressees: • Proper names when addressee is apparently fellow expert Chrysler building • More detailed descriptions when addressee is apparently non-expert Building with a tall pointy roof and a spike on top

  10. Audience Design - syntax • Audience design may influence syntactic choices: • Speakers may choose syntactic structures that are most easily understood by addressees. • Ambiguity-avoidance: Put the penguin in the cup on the star vs Put the penguin that’s in the cup on the star When two penguins (and an empty cup), temporary ambiguity at penguin in the cup…

  11. Audience Design - syntax • Audience design may influence syntactic choices: • Speakers may choose syntactic structures that are most easily understood by addressees. • Ambiguity-avoidance: Put the penguin in the cup on the star vs Put the penguin that’s in the cup on the star When two penguins (and an empty cup), temporary ambiguity at penguin in the cup… Speakers include that’s more often (removing ambiguity) when there are two penguins than where is only one penguin. Haywood, Pickering & Branigan (2005)

  12. Audience Design - summary • Speakers in HHI may engage in strategic planning of aspects of utterances: • Calculations of addressee knowledge, beliefs etc • Based on a priori assumptions or direct evidence • A priori assumptions are quickly updated in light of direct evidence; audience design reflects this • Strategic planning may affect lexical choice, syntax, semantic choices • E.g., choice of reference frame (Schober, 1993)

  13. Alignment • Speakers are affected directly by addressee’s linguistic behaviour: • Tendency for speakers in a dialogue to converge linguistic behaviour. • After hearing an interlocutor use particular linguistic behaviour, speakers tend to repeat that behaviour. • This tendency towards alignment is robust and highly pervasive. • Rhetorical structure, semantic structure, syntactic structure, lexical choice, accent, speech rate, ….

  14. Alignment • Alignment can be implicit: • Almost always arises without explicit negotiation • Speakers often align on different expression than negotiated expression • Speakers usually unaware of effects • Can sometimes (but often cannot) report awareness of meaning-related alignment • Usually unaware of alignment of form

  15. Alignment – semantic choices • Alignment occurs for aspects of language associated with meaning: • e.g. reference frames (Watson, Pickering & Branigan, 2005) The dot is left of the cameravs The dot is below the camera

  16. Alignment – semantic choices • Alignment occurs for aspects of language associated with meaning: • e.g. reference frames (Watson, Pickering & Branigan, 2005) The dot is left of the cameravs The dot is below the camera • e.g., description schemas(Garrod & Anderson, 1987) I’m at B5 vsI’m two along and two up

  17. Alignment – lexical choice • Also alignment of lexical choice: • Use same words in same ways • boxvs node • square = single nodevsconfiguration of nodes

  18. Alignment – lexical choice • May align on unusual/rare lexical choice: • e.g., if first person to name object is a non-native speaker

  19. Alignment – lexical choice • May align on unusual/rare lexical choice: • e.g., if first person to name object is a non-native speaker Rocking chair

  20. Alignment – lexical choice • May align on unusual/rare lexical choice: • e.g., if first person to name object is a non-native speaker Rocking chair The chair that can go back and forth

  21. Alignment – lexical choice • May align on unusual/rare lexical choice: • e.g., if first person to name object is a non-native speaker Rocking chair The chair that can go back and forth You can shake your body Bortfeld & Brennan, 1997

  22. Alignment • Alignment at semantic and lexical levels may be linked to different types of meaning: • Alignment of perspectives on a situation (ways of thinking about the world) • E.g., rainbow troutvscoloured fish • But other alignment seems to be unrelated to convergence on types of meaning • e.g., speech rate • e.g, syntax (alternatives express same meaning)

  23. Alignment – syntactic choices

  24. Alignment – syntactic choices The artist is selling the dancer the gun(Double Object) vs The artist is selling the gun to the dancer (Prepositional Object)

  25. Alignment – syntactic choices The artist is selling the dancer the gun(Double Object) vs The artist is selling the gun to the dancer (Prepositional Object) • A: The chef handing the book to the teacher • B: The artist selling the gun to the dancer

  26. Branigan et al. (2000) • Picture-description/-matching game • Stooge and naïve subject alternately describe pictures • Stooge produces scripted descriptions (50% PO, 50% DO) • Stooge describes source picture to subject: • The nunshowing the monk the banana • Subject chooses matching picture: • Yup

  27. Branigan et al. (2000) • Picture-description/-matching game • Stooge and naïve subject alternately describe pictures • Stooge produces scripted descriptions (50% PO, 50% DO) • Stooge describes source picture to subject: • The nunshowing the monk the banana • Subject chooses matching picture: • Yup • Subject describes target picture to matcher: • The artistselling the dancer the gun • Matcher chooses matching picture: • Uh huh

  28. Branigan et al. (2000) • Picture-description/-matching game • Stooge and naïve subject alternately describe pictures • Stooge produces scripted descriptions (50% PO, 50% DO) • Stooge describes source picture to subject: • The nunshowing the monk the banana • Subject chooses matching picture: • Yup • Subject describes target picture to matcher: • The artistselling the dancer the gun • Matcher chooses matching picture: • Uh huh

  29. Confederate-scripting paradigm Naïve participant Confederate Blah Blah Blah Confederate’s script Picture cards to be matched Branigan et al (2000)

  30. Alignment - syntax • Strong tendency to align syntax with interlocutor: • More likely to produce a Prep Obj description after hearing a Prep Obj description than after a Double Obj description, and vice versa. • 77% aligned descriptions when verb repeated • 63% aligned descriptions when verb not repeated • Chance = 50% • NB: speakers align sentence form: both alternatives have same denotational meaning • (cf. semantic/lexical alignment) • Speakers align dynamically (produce both PO and DO)

  31. Alignment - syntax • Similar effects found: • for other structures: • E.g., NP structure: a red catvs a cat that’s red (Cleland & Pickering, 2003) • NB: Relative clause = strongly dispreferred in null context • in multi-party dialogues: • Speakers syntactically align, whether directly addressed or not. A: The waitress is giving the monk the banana C:Yup

  32. Alignment - syntax • Similar effects found: • for other structures: • E.g., NP structure: a red catvs a cat that’s red (Cleland & Pickering, 2003) • NB: Relative clause = strongly dispreferred in null context • in multi-party dialogues: • Speakers syntactically align, whether directly addressed or not. A: The waitress is giving the monk the banana C:Yup B:The teacher is handing the cowboy the jug C:Uh huh Branigan et al. (in press)

  33. Alignment - syntax • Similar effects found: • in different communicative contexts: • e.g., during walkie-talkie communication • in ‘special’ populations such as bilinguals, L2 learners, children…

  34. Alignment - syntax • Similar effects found: • in different communicative contexts: • e.g., during walkie-talkie communication • in ‘special’ populations such as bilinguals, L2 learners, children… El cerdo llora El pingüino baila (SV)

  35. Alignment - syntax • Similar effects found: • in different communicative contexts: • e.g., during walkie-talkie communication • in ‘special’ populations such as bilinguals, L2 learners, children… El cerdo llora El pingüino baila (SV) Llora el cerdoBaila el pingüino (VS) Flett, Branigan & Pickering, in prep

  36. Alignment - summary • Alignment with addressee’s linguistic behaviour: • is robust • can be very strong determinant of behaviour • occurs for many (all?) levels of linguistic structure • occurs for aspects of structure concerned with form, not just meaning

  37. Alignment and audience design • Alignment and audience design may co-occur. Put the penguin in the cup…vs the penguinthat’s in the cup…

  38. Alignment and audience design • Alignment and audience design may co-occur. Put the penguin in the cup…vs that’s in the cup… • Priming: • tendency to say The penguin that’s in the cupafter hearing The sheep that’s on the plate • Audience design: • tendency to say The penguin that’s in the cupwhen there are two penguins Haywood, Pickering & Branigan, 2005

  39. Explanations of alignment • Social psychological approaches to alignment: • Identification with a particular social group (inc. addressee) • Reciprocity effects (linked to politeness norms?) • Such approaches explain why alignment of linguistic form occurs (in absence of meaning differences) • Perceived social identity of addressee is important • e.g., politeness is only relevant for addressees that are perceived as social agents

  40. Explanations of alignment • Alignment as audience design: • Alignment may reflect strategic effects (choosing to adopt other person’s perspective to enhance communication) • Choosing e.g., the same description schema or referential expression maximises the chances of effective communication • Such approaches do not really explain why alignment of linguistic form occurs (in absence of meaning differences) • Perceived identity of addressee is important • e.g., depends on addressee being intentional agent • inanimate entities do not have perspectives or beliefs

  41. Explanations of alignment • Alignment as an automatic behaviour: • Primitive, default behaviour • children: • align significantly more than adults (e.g., NP structure: 75% vs 30%) • tend to align form, even when this leads to misunderstanding: • e.g., using same term with different reference (e.g., square) • must learn to suppress tendency towards alignment where appropriate (Garrod & Clark, 1994).

  42. Explanations of alignment • Alignment may be based on automatic priming mechanisms: • activation-based account (Branigan et al., 2000) • comprehension of word or structure activates associated linguistic representations • VP  V NP PP • representations retain residual activation • residual activation facilitates subsequent selection

  43. Explanations of alignment • Alignment may be fundamental to efficient communication (Pickering & Garrod, 2004) • Efficient communication arises when interlocutors come to have the same understanding of relevant aspects of the world. • This arises from alignment of their situation models(e.g., Zwaan & Radvansky, 1998). • Alignment of situation models arises from alignment of other aspects of language (e.g., syntax, lexical choice). • Alignment at one level promotes alignment at others Lexical alignment  syntactic alignment  semantic alignment

  44. Explanations of alignment • Explanations are not mutually exclusive: multiple factors may underline alignment. • Most likely that at least some implicit element • Participants generally report lack of awareness of alignment • But other factors may also contribute to overall effect: • Basic (automatic) alignment effect may be enhanced by e.g. social factors (e.g., social status) • Alignment at some levels of structure may be differentially susceptible to additional influences

  45. Implications for HCI • Does alignment occur in HCI? • Pickering & Garrod (2004): • alignment is fundamental to effective communication • So evidence that alignment is absent in HCI would imply that HCI is necessarily ineffective/inefficient. • May highlight inherent difficulties in HCI • Conversely, evidence that alignment is present in HCI suggests possibility of effective communication in HCI

  46. Alignment in HCI? • Why might we expect alignment in HCI? • Alignment seems to be a default linguistic behaviour: • May be consequence of architecture of human language processor • Might therefore expect to find it in any communicative context • Existing evidence that people treat computers as social actors Reeves & Nass (1996) • They interact with them and evaluate them as they do other people • People should align with computers

  47. Leveraging alignment in HCI • Alignment may be useful behaviour: • Facilitates predictions of user behaviour • Reduces decision space in e.g., speech recognition • Allows simulation of human behaviour • Users may feel more comfortable with systems that display human-like behaviour • Users might feel more positively towards systems that align than those that do not • May allow [implicit] modification of human behaviour • Users may modify their linguistic behaviour to align with system • May allow modelling of desired input

  48. Patterns of alignment in HCI • How might alignment in HCI pattern? • If alignment is purely automatic priming: • Alignment would occur whenever a linguistic structure is encountered • But in HHI, speakers’ (lexical and syntactic) choices are also influenced by: • a priori beliefs about addressee knowledge, capability etc • direct evidence about addressee knowledge, capability etc • Hence beliefs about the knowledge, capability etc of a system might influence the extent to which speakers align with it.

  49. Relevant factors in HCI alignment • A priori beliefs about social identity of system: • If systems are treated as social agents just like humans, then alignment should occur just as with humans. • But which group of humans would count as comparable social agents? • Limited capability? • If systems are not treated as social agents just like to humans, then alignment may differ: • Less alignment if alignment has substantial component related to e.g. politeness

  50. Relevant factors in HCI alignment • Beliefs about system capability: • Users may assume computers to be less capable (generally and/or specifically linguistically). • May increase likelihood of aligning, relative to HHI • Possibly different degrees of alignment with different systems • May increase likelihood of aligning with unusual structure/expression • Effects of feedback: • A priori beliefs may be updated in light of feedback • Positive feedback of understanding may lead to comparable alignment as in HHI

More Related