Auditory Scene Analysis (ASA)

1. Auditory Scene Analysis (ASA)

2. Auditory Demonstrations • Albert S. Bregman / Pierre A. Ahad “Demonstration of Auditory Scene Analysis, The perceptual Organisation of Sound” ( CD in library) • For a comprehensive view of Auditory Scene Analysis: • Bregman, A. S. (1990) Auditory scene analysis: the perceptual organisation of sound. Cambridge, Mass.: The MIT Press (in library) • Other books on auditory perception also give descriptions of ASA (check in library)

3. Some terms • Source – the physical entity that gives rise to the sound pressure waves e.g. a violin being played • Stream – the percept of a group of successive and/or simultaneous sounds as a coherent whole appearing to come from a single source • The sounds we hear at any one time usually come from a number of different sources. • In most cases we can hear and identify each of the different sound sources as having its own pitch, timbre, loudness and location.

4. Auditory Scene Analysis • The inner ear separates sound into its frequency components • At some point in the auditory system these components need to be assigned to the appropriate sound source • Often called “perceptual grouping”, or “auditory scene analysis” • Two aspects: simultaneous grouping and sequential grouping

5. Auditory Scene Analysis • Simultaneous grouping – the grouping together of the simultaneous frequency components that come from a single source • Sequential grouping – the connecting over time of the changing frequencies that a single source produces from one moment to the next • First we will consider some of the physical cues used to achieve simultaneous grouping • These cues are usually studied in isolation

6. Simultaneous grouping • Some cues: • Fundamental Frequency and Spectral Regularity • Onset Timing • Correlated changes in Amplitude or Frequency • Sound Location • Important: A single cue may not be effective all the time – these cues work together for perceptual organisation of the input sound

7. Fundamental Frequency and Spectral Regularity • Consider two musical instruments each playing a note simultaneously • It is easier to hear each note and each instrument if they are playing different notes (have different fundamental frequencies) • Simultaneous sounds are more likely to fuse if they have the same fundamental frequency ** • Has been shown that a pair of simultaneously presented vowels are easier to identify if their fundamental frequencies differ

8. Fundamental Frequency and Spectral Regularity • Perceptual fusion of the frequency components from a harmonic sound – harmonicity – heard as a single sound • If a frequency component does not form part of the harmonic series it tends to be heard out separately – as if part of a different source • Demonstration 18: Isolation of a frequency component based on mistuning • 3rd harmonic is mistuned – number of steps to hear it out - % mistuning for perceptual segregation

9. Onset disparities • Perceptual separation on tones enhanced by onset asynchrony. • A frequency component that stops or starts at a different time from the complex sound is less likely to be heard as part of it than if it is simultaneous with it • Rasch, R. A. (1978). The perception of simultaneous notes such as in polyphonic music, Acustica 40, 21-33

10. Onset disparities • We can hear each of two ‘simultaneously’ played notes easier if there is a small onset difference between them • These onset asynchronies are up to 30ms – so the percept is still of the notes sounding together • The auditory system can exploit these onset differences even though we are not consciously aware of them • Ensemble playing – completely synchronised?

11. Onset disparities • Demonstration 21: effect of rate of onset on segregation • Cluster of four overlapping pure tones • The order of the onsets is either M H L M or M L H M (Medium High Low in frequency) • The rise times are made shorter with each presentation – the asynchrony of the onsets is left unchanged • affects the clarity with which the order of the tones can be heard

12. Onset disparities • Shorter rise times – easier to hear the order of the tones • Generally, sounds with abrupt onsets (shorter rise time) stand out better from a background of other sounds than do slow-rising sounds • Shorter rise times – aids the perceptual segregation of sounds – to tell them apart • Rapid onset sounds – e.g. notes from plucked or struck instruments

13. Correlated Changes in Amplitude or Frequency • A sound may be perceptually segregated from an unchanging background if its components are modulated in amplitude or frequency • Demonstration 19: Fusion based on common frequency modulation • Hear harmonic complex tone • Harmonics 1, 3, 5, 6, 7 remain steady • Harmonics 2, 4, and 8 rise and fall in frequency four times • Hear the two sets as separate sounds

14. Correlated Changes in Amplitude or Frequency • Demonstration 20: Fusion by common frequency modulation • All harmonics are frequency modulated • First all harmonics change in frequency at the same rate – one sound is heard • Then the harmonics are divided: harmonics 3, 4, and 5 undergo slow changes in frequency and the first harmonic undergoes fast changes in frequency • Hear two sounds

15. Sound Location • Sounds coming from different locations in space are generally assumed to be from different sources

16. Gestalt Principles of Organisation • Gestalt psychology founded in the early 20th century • A group of psychologists: Max Wertheimer (1880-1943), Wolfgang Köhler (1887-1967) and Kurt Koffka (1886-1941) formed the Gestalt school • Gestalt theory – the perceptual whole is more than the sum of its parts • Put forward a set of Gestalt grouping rules that describe which elements in an image belong together to form an object – mostly described in relation to vision

17. Gestalt Principles of Organisation • These principles can also apply to hearing / auditory perception • Application of these principles generally results in a grouping of the parts of the input sound that come from the same source - segregating those that don’t • Discuss each of the principles separately • Important: These principles work together – to arrive at a correct interpretation of the input sound – no single rule will always work

18. Gestalt Principles of Organisation • Similarity • Good Continuation • Common Fate • Disjoint Allocation • Closure • The Figure-Ground Phenomenon and Attention

19. Similarity • Sounds are grouped into a single perceptual stream if they are similar in pitch, timbre, loudness or subjective location • Demonstration 17: Failure of crossing trajectories to cross perceptually • Falling and rising sequence are interleaved – tones from the rising and falling sequence are alternated in time. • How easy is it to hear out each of the four standards • Grouping by timbre and frequency region