Audio theory and practice for language documentation

1. Audio theory and practicefor language documentation LingDy 12 Feb 2013 TUFS, Tokyo David Nathan Endangered Languages Archive Hans Rausing Endangered Languages Project SOAS, University of London

2. Questions • have you recorded audio? • have you published audio? • what else have you done with your audio?

3. Question • digitally recorded audio is better quality than analogue recorded audio because: (a) digital microphones are more accurate (b) digital formats are more accurate (c) digital equipment is newer (d) digital formats capture more information (e) no, digital audio is not better than analogue audio

4. Big questions • what are we actually recording? • what/who is it for? • what is the role of audio in language documentation?

5. An epistemology for audio in documentation • an audio recording is made in order to be experienced by a human listener • a recording conveys what a human listener would experience at a particular location in an event setting • documentation goals define recording methodology • a recording should capture spatial information • metadata about the recording and the recording setting are required for full interpretation • ethical recording respects speakers and honours their contribution through your effort and skill

6. Evaluating recordings • accuracy: how well is the signal captured, as true to its sources and without distortion? • intelligibility/information accessibility: can the desired content be identified? • signal vs. noise: is the ratio acceptable? can the focal source be separated from all sources of noise? • listenability/comfort/aesthetics: is it easy on the ears? will it be comfortable to listen to for an extended time?

7. Evaluating recordings • localisation of sources: is enough spatial information captured? • separation of noise: can all sources of noise be separated? • representation of environment: are the acoustic properties of the recording space appropriately represented?

8. Evaluating recordings • content (identity, performance, uniqueness, coverage): were the right people recorded doing the right things? • editability/repurposeability: is the recording suitable for turning to relevant purposes?

9. Recording audio • making it is both art and science • a critical and ethical responsibility • strongest relationship to communities • it’s not necessary to record everything, but it is necessary to record well

10. SIGNAL & NOISE

11. Evaluating recordings • signal • noise • signal to noise ratio • listenability (eg comfort, consistency) • fit for purpose

12. Evaluating recordings • audio professionals use their human ears as evaluator of audio quality and value, while many linguists (mistakenly?) look to formats, spectrographs, wave-forms, analyses etc   44.1 KHz, 24 bit

13. Signal - what you want • content • contextual and spatial information • fidelity • comfortable to listen to

14. Noise - what you don’t want • from environment: • near: people, animals, activities • far: traffic, generators, planes • machines: refrigerators, fans, computers • not hearable: mobile phones, electrical interference • acoustic: reflections/resonance

15. Noise - what you don’t want • generated by (unwanted parts of) event • shuffling papers, clothes • table banging • backchannel from interviewer • equipment handling, especially microphones and cables (and recorders with built-in mics)

16. Avoiding handling noise • use stands and cradles etc

17. Noise - what you don’t want • generated by equipment • wrong input levels • circuity noise (cheap or incompatible) • compression loss or distortion • ALC/AGC effects (pumping) • video camera motors

18. External noise sources • see also General principles

19. Dead cat

20. Close-up noise sources • machines

21. Dealing with noise sources • be prepared and aware • seek collaboration • monitor • use or modify room acoustics • location • direction • surfaces • reflection • absorption • isolation

22. Utilising room acoustics • location • away from doors, windows, traffic areas • direction • face away from noise sources • reflection • avoid parallel surfaces • surfaces • avoid hard smooth surfaces • choose or create soft or rough surfaces • isolation • find an ‘’airtight’’ place

23. When is a noise not a noise? • When it is part of the content, for some interpretation of the event John Cage performance Available on iTunes (150 yen)

24. PERCEPTION & PSYCHOACOUSTICS

25. Audio perception/psychoacoustics • a human listener has: • location, orientation in a physical setting • two ears - incredibly sensitive • a brain/mind • the mind selects from various sources of sound and other sensory information, using long- and short-term memory • listening is actually a “hallucination”

26. Psychoacoustics and recording • microphones don’t have a mind: they can't distinguish wanted from unwanted sound • microphones don’t have “edges” like camera lenses

27. Psychoacoustics and recording • the recording process loses acoustic information • if you only care about transcription, then you are going to throw away over 99% of the acoustic information anyway! • real world • record acoustic phenomena • represent (some) linguistic components • derive data

28. Implications for recording • typical recording methods are unscientific! • … so what should we do?

29. Implications for recording • plan and manage recording • goals • equipment preparation and settings • other preparation • environment and setup • sources • changes, actions, settings

30. Implications for recording • why is it important to record spatial information? • what other information (acoustic or non-acoustic) do we need?

31. “Sound stage” • spatial information is an essential part of audio • we are amazingly attuned to it • we should record in stereo

32. “Sound stage” • ... or in ORTF (binaural)

33. MICROPHONES

34. Microphones and audio quality • microphones are the greatest factor in audio recording quality • selection of appropriate microphone(s) for the task • placement and handling

35. Microphone types • principle: dynamic vs condenser • directionality: omni, cardoid, and shotgun • spatiality: mono, stereo, ORTF, binaural

36. Microphone physical principles • dynamic • generate signal from sound pressure • more robust, less accurate • used for musical and live performance • condenser • more fragile, sensitive and accurate • need power source - battery or phantom power • in general, use condenser microphones for language documentation

37. Omni • lavalier or tie-clip microphones are typically omni-directional

38. Microphone directionality - omni omni-directional

39. Cardioid • many “standard” handheld microphones are cardioid units

40. Microphone directionality - cardioid cardioid

41. Shotgun • shotguns are good for quiet sources, in some noisy environments, and for video work

42. Microphone directionality - shotgun shotgun/directional/hypercardioid

43. Stereo microphones • spatial information is an essential part of audio

44. Full “sound stage”: ORTF • now the “best practice” for field recordings” (Austrian Academy of Science, Vienna Phonogrammarchiv) Superlux S502 Full binaural on dummy head

45. 110° 17cm Simulating ORTF with 2 cardioids

46. Microphones - quality • generally, you get what you pay for • each model has its own subjective “colour” • good microphones for language documentation cost from $180 to $500

47. Reputable makers - examples • AKG • Audio Technica • Beyerdynamic • Røde • Sennheiser • Shure • Sony

48. Microphone placement

49. Microphone usage principles • where should the microphone be? • in general, about 20cm from the speaker’s mouth • the inverse square law is your friend ...

50. The inverse square law