Listeners’ Uses of Um and Uh in Speech Comprehension

Listeners’ Uses of Um and Uh in Speech Comprehension Jean E. Fox Tree INTRODUCTION Presented by: Jennifer Ping

Outline • Introduction of main topic • Why people use um and uh • What listeners do when they hear ums and uhs • Effects of ums and uhs on on-line speech comprehension • No effect • Disruptive effect • Beneficial effect • What present study is investigating • Criticism

Introduction • Real world applicability: • Ums and Uhsin communication are very common and used by people worldwide • See it occurring in situations ranging from people merely talking to one another to formal speeches and presentations • What do people think of its use? • Considered undesirable and unnecessary

Why do people use um and uh? • Signal explanation • Serve as signals of upcoming delay • Um signals long delay • Uh signals short delay • Other proposed functions: • Indicate speaker’s speech production trouble, including the need for more time to plan upcoming speech • A person’s desire to maintain control of the floor

What do listeners do when they hear ums and uhs? • “Uh” • Focus listener’s attention on immediately upcoming information • “Um” • Since long delay follows “um”, not beneficial by listener to focus attention • If expecting a long delay after an “um”, heightened attention from a listener may not follow due to: • Listeners may not be able to maintain their heightened attention for that long. • The major delay after “um” might arise from problems that are not benefitted by a listener’s heightened attention • The anticipated major delay may shift the focus from the listeners’ anticipation of upcoming speech to something different • Listeners may play active roles in helping speakers complete their ideas

Why might umsand uhshave no effect on on-line speech comprehension? • On-line speech comprehension: the process of ongoing understanding while processing spoken language • Word integration:the assimilation of a word into the syntactic and semantic representation of a sentence • 2 predicted reasons: • May be filtered out of the speech dialogue before word integration begins • May function at different level of comprehension where instead of filtering them out, they may be processed and noted without their influencing on-line word integration

Why might ums and uhs disrupt online speech comprehension? • 2 predicted reasons: • Disruptive filter explanation: Ums and uhs are filtered out but the filtering process is disruptive. • The identification of ums and uhs in the speech stream, the flagging of them as non-propositional information and the elimination of their incorporation into the final product can take time. • Stumbling block explanation: Failed attempts of incorporation into speech comprehension • Ums and uhs would pose a stumbling block to speech comprehension models b/c they cannot be combined with surrounding words to form syntactic constructions.

Why might ums and uhs have a beneficial effect on on-line speech comprehension? • If ums and uhs serve as signals for upcoming delays, then they help on-line speech comprehension by prompting listeners to pay more attention to upcoming speech.

Present Study • Tested how ums and uhs affect on-line speech processing • Compared: • How long it took listeners to detect a word in a speech stream after a spontaneously produced um or uh with how long it took them to detect the same word when the um or uh had been digitally excised. • Hypotheses: • If ums and uhs are disruptive to on-line speech comprehension, word monitoring should be slower after an um or uh than when the um or uh has been excised. • However, if ums and uhs are beneficial for speech processing, word monitoring should be faster after an um or uh than when the um or uh has been excised. • If they are neither detrimental nor beneficial, then there should be no effect.

Criticism • No explanation or suggestions for why there is a short delay after an “uh” and why there is a long delay after an “um” before upcoming speech. • Any Suggestions??? Buckle, Ashley-Lynn; Fudge, Miranda C.; Winsor, Nicki-Lee J.

Experiment 1English Ums & Uhs Method By: Jessica Park

Participants ~ Thirty-four native English Speakers from the University of California participated in this experiment in exchange for course credit

Materials ~The materials were taken from spontaneous speech of students telling face to face stories to each other ~The speech for this study was collected by Herbert Clark at Stanford University, and consisted of: • A stretch of speech from one speaker that contained at least one spontaneously occurring um or uh • Followed immediately by a word that had not occurred earlier in the stimulus (i.e., the post um or uh word). • Each stimulus began at the beginning of an idea and finished with a completed thought.

Stimuli ~Eighty-eight stimuli were selected in total • 40 critical • 40 filler stimuli • 8 practice stimuli

Stimuli (con’t) ~All stimuli were similar in length & content ~The post um or uh wordwas the target word in the critical stimuli ~A target word was chosen that was similar semantically, and if possible, phonologically to the post um or uh word, but that did not occur in the stimulus as the filler stimuli. ~Target words were one to four syllable long and chosen from a variety of form classes.

Stimuli (con’t) ~The filler stimuli helped insure that the participants did not adapt a strategy of immediately responding after hearing an um or uh. ~This was accomplished due to the fact that the stimuli contained 24 ums and 17 uhs in addition to the 20 ums and uhs of interest that each participant heard.

Stimuli (con’t) ~ Eight of the critical ums and eight of the critical uhswere matched on the local syntactic constituent structure immediately surrounding the fillers and targets. This accounted for 40% of the critical stimuli. ~ The example given in the text stated, • Um: And he said why sure well what kind of um price range are you looking for? • Uh: Then he also toi-sold her on uh a couple of uh furniture items for the ant.

Stimuli (con’t) ~Matching was done to test the role of syntactic location in causing any effects found. ~An example of this was found in Dutch research by Swets (1998). It was found that uhs usually occur within clauses and ums between clauses. ~This led to the conclusion that uhs might be more noticeable than ums and therefore have a stronger effect. ~Ums anduhsoccurred both within and between clauses in the present experiment.

Stimuli (con’t) ~For each of the critical stimuli and for half the filler stimuli, second versions were created in which the critical um or uh was digitally excised. ~In the excised version the um or uh was not replaced by a pause, although any pauses preceding or following the um or uh were retained. ~This was done in order to keep the number of variables that might otherwise have influenced comparison between speech with or without ums oruhsconstant. ~These other variable could include things such as pronunciation, syntactic construction, frequency of the target word.

Stimuli (con’t) ~Therefore, The only difference across conditions was whether or not there was an um or uh present, as well as the processing time associated with the length of the um or uh excised.

Detectability of Editing ~ A follow-up study tested whether the listeners were able to detect editing in the stimuli. ~The 80 critical stimuli were divided into two lists, each containing: • 10 unedited um items • 10 edited um items • 10 unedited uh items • 10 edited uh items.

Detectability of Editing ~Twenty people who were not participants in the main experiment were asked to listened to each list. ~Ten people where assigned to each list. ~The participants were asked to indicate where they could “spot a splice” where the material was digitally edited. ~The participants were allowed two practice trails to help them understand the instructions prior to executing the experiment.

Detectability of Editing ~It was found that participants performed no better than chance at identifying the edited speech. ~This inability of the listeners to detect editing was consistent with similar studies conducted by Fox Tree (1995) and Fox Tree and Schrock (1999).

Design ~The design was a within-subject & within-item counterbalance design. ~Of the two lists were created: ~List 1 contained the practice stimuli • 10 unedited critical um stimuli • 10 unedited critical uh stimuli • 10 edited critical um stimuli • 10 edited critical uh stimuli • 40 filler stimuli

Design ~List two contained the same practice and filler stimuli. ~However, the edited version of the list 1 unedited critical stimuli and the unedited versions of the list 1 edited critical stimuli. ~The order of presentation was randomized the same across both lists. ~Each participant listened to only one list and therefore heard only one version of each stimulus.

Procedure The following structure was used for each trial: • 500-msec tone rang indicating that the participants should focus their attention on the computer screen. • The tone was followed by 500-msec silence. • A word then appeared on the computer screen for 1,000 msec. Followed by a 1,000-msec silence. • Immediately after the sound file was played. The participants were asked to hold the word in memory while they listened to the sound file and immediately press a button in front of them if they heard the word.

Procedure • 500-msec 500-msec silence word (1,000 msec) 1,000-msec silence sound file continues…

Procedure • Therefore critical stimuli elicited button presses, filler stimuli did not. • Reaction times were measured from the onset of the target word in msec. • Response times timed out after 1,500-msec. • All stimuli played to the end, regardless of whether a button was pressed. • There was a 1,500-msec silence between trials. The exp. Lasted roughly half an hour.

Listeners’ Use of Um and Uh in Speech ComprehensionFox Tree, Jean E. Result Section

Overview • Sig. difference in recognizing target words when uhs were present then when they were not. • Beneficial to speech • No sig. difference for ums. • Neither beneficial or detrimental to speech • Attempt to rule out alternative explainations.

Removed From Results: They eliminated both false alarms and misses. There were no differences in error rates across treatment conditions. Three test items were removed. Two because more than 20% didn’t respond to them. The other one was removed because of experimental error.

Main Finding There was a significant difference found between the Ps speed at recognizing the target words following uhs and recognizing those same words when the uhs were removed. No significant difference for the ums.

Implications of Main Finding • Uhs are beneficial to speech. • The Ps took significantly longer to recognize the target word without the uh (601 msec) then with the uh (554 msec). • Ums are neither beneficial or detrimental. • No sig. effect of these. The Ps took 561 msec to recognize the target word when the ums were present and 548 when they were not present.

Factors Involved In The Difference Between um and uh. He found that there were many other possible reasons that there could be a difference between um and uh in language. He attempted to rule out these possibilities.

Ruling Out Factors in Differences Between um and uh 1. Length of um and uhs. If uhs were longer than ums, then this could account for the difference. This is not the case, however, as the ums were actually longer on average than the uhs 2. Amount of pausing before and after ums and uhs. If uhs had longer pauses they might be more noticed. This is not relevant here because there was not much difference between pausing associated with um and those associated with uhs.

Ruling Out Factors in Differences Between um and uh Where the ums and uhs were located. If ums were in the same syntactic location as the uhs, those ums should show an effect. However, there was no significant interaction between the presence or absence of um or uh, and whether the stimuli were matched or not. The different proportion of typical and atypical ums and uhs could not account for the results here.

Ruling Out Factors in Differences Between um and uh The target words following ums and uhs. The was no difference between the reaction times for um and uh targets. So, the targets for each can be considered similar enough to have not caused the effect. The pauses in the edited conditions. The original pauses before and after the ums and uhs were retained in the edited version. There was no correlation found between the differences in reaction times across conditions and the amount of pausing that remained in the edited conditions.

Alternative Explanation: The differences could be due to coarticulation affects. This says that the sound of the phoneme is impacted by the sounds of surrounding phonemes. If targets after uh are more likely to be coarticulated with it then um targets are coarticulated with um, then removing the uh could also remove important word recognition information Ums and uhs are never attached to another word, so coarticulation effects would have been very surprising here.

Critical Analysis • The difference between the um and uh in the edited version was large (63 msec), and no significant effect. • Reasons for ruling out some of the possibilities were vague. • i.e. length of ums and uhs, & location of ums and uhs.

Exp. 2: Method Second verse (Exp. 2), same as the first (Exp. 1) Except in Dutch

Method : Exp. 2 • Ps were 32 Dutch speakers paid for participating. • Materials • Taken from spontaneous speech of students describing abstract figures to a silent listener. • Materials were the same as Exp. 1 except 10 more filler stimuli • Targets ranged from 1 to 6 syllables • 50% of critical stimuli were syntactically matched

Detectability of Editing • Ps listened to each stimulus and marked on a written transcript where they thought digital manipulation (editing) had occurred. • Ps greatly overestimated editing • 90% of stimuli were judged to have been edited (50% were edited) • No difference in the detectablility of editing between um and uh stimuli

Design: Similar to Exp. 1 • Each P heard all 5 types of stimuli • 40 critical stimuli (speech contained the target word) • 10 unedited um stimuli (um present) • 10 edited um stimuli (no um present) • 10 unedited uh stimuli (uh present) • 10 edited uh stimuli (no uh present) • 50 filler stimuli in which speech did not contain the target word. • 40 filler stimuli in Exp. 1

Procedure: Similar to Exp. 1 • 500 ms 1500 ms silence 715 ms • Warning tone ----------------Target Word-------------- Sound file • Press button if target word heard.

Critical Analysis (All) • Did the experiments test speech perception and not comprehension? • Could test comprehension with a “does the target word fit in the sentence” task • Small reaction time differences (< 50 ms., but significant) • Usually like to see a 50 ms. difference or more. • If ums signal a long delay (and attention wanders), why was there no difference when directly comparing uhs and ums in Exp. 1 • There was such a difference in Exp. 2 • Other criticisms?

Results Experiment 2 Dutch Ums & Uhs Presented by: Jessica Park

Results ~ Response times more than two standard deviations from the mean were treated as outliers. ~This accounted for 7% of the data and was removed from the analysis. ~This ensured elimination for both false alarms and misses.

Results ~There were no differences in error rates across conditions. Unedited um, 5.9% ~F1 (1, 31)= o.66, n.s Edited um, 7.4% ~F2 (1, 19)= o.63, n.s.

Results Unedited uh, 6.6% ~ F1 (1, 31)= 0.62%, n.s. Edited uh, 8.2% ~F2 (1, 19)= 1.11, n.s

Results ~These results replicated those of experiment one. ~There was a significance found in the listeners’ speed at recognizing the target words following uhs compared with their speed at recognizing the same words when the uhs were excised. ~F1 (1, 31)= 7.5, p= .01 ~F2 (1, 19)- 4.28, p= .05

Listeners’ Uses of Um and Uh in Speech Comprehension