The Next 5 weeks: A Journey into sound. In no particular order Intro to sound Editing, multitracking, processing sound M

1. The Next 5 weeks: A Journey into sound. • In no particular order • Intro to sound • Editing, multitracking, processing sound • Making a podcast (how to) • Digital synthesis • Streaming video/audio content from the helix server (how to) • Bluescreen / Chromakey • In camera effects

2. Designing with sound. Benyon – pages 394 - 402 • Elements of sound • Hardware, software and file types • Human hearing • Compression

3. Content Creation Bitmap imagesVector imagesTextVector animation3D animationMidi music Design and authoring Web pagesFlash moviesDirector movies Production and distribution CDRom Burning Stream to web Downloadable content Content Capture Digital audioDigital videoDigital photographs Coding hooks PythonJavascript / JavaASP / PHP and dynamic data Diagram showing all of the components of a multimedia production –content creation, assemblage and distribution

4. Traditionally sound has been used very unimaginatively in computer interface (with the exception of computer games) • Most often bleep and boings • Reason was the due to the rarity of soundcards in early PC’s and the expense of RAM needed to store sounds – no longer a problem.

5. Music • Sets place and time • Emotional effects (suspense, sadness, energy) • Covers over visual breaks • Voice • Informational “content of words” • Personality (comic, serious, cultured) • Connects to listener in ways text doesn’t • Spot effects • Information (birdsong, door closing) • Punctuation (gives feedback – Alert sound in Windows) • Ambient effects • Location • mood Communicative effects of sound

6. Auditory channel Sound effects Music Voice track Ambient sound Time Visual Channel Images Video Text Image and text Diagram showing ‘parallel’ nature of audio information in comparison to ‘linear’ video information

7. Informational use of sound: • Sound carries a lot of information. • Loud/soft – distance • Reverberation – sense of space or room size • Stereo – position in space • Combined audio – sound effects /music /speech • Music – emotional effects

8. Psychoacoustics • Continuous perception • We are aware of sound even when not ‘thinking about it’ • Primacy effect • Focussing on important sound • Masking • Ignoring unimportant sound

9. Copyright issues Because its easy to distribute ‘copied’ music across the internet does not make it legal! Both Napster and MP3.com were prosecuted. • Solutions? • Write and record it yourself – if you’re good enough and don’t mind giving your talents away for free! • Use royalty-free clips (but check the actual rights on offer) • license what you can’t produce yourself – either the song, the recording or both

10. Elements of sound • Loudness > Measured in ? • 0db as absolute upper limit • Pitch (frequency) > Measured in ? • Timbre – not measured, but is the ‘quality’ of sound – why a piano and violin playing at the same pitch and loudness sound different.

11. Elements of sound Drum Envelopes Loudness Time violin Diagram showing a graphical representation of two audio envelopes – one with a fast attack (drum) and one with a slow attack (violin) Time

12. Hardware Sound • Soundcard with mic / line input • Soundsource / microphone • Large HD (1 min of CD audio = 10 Mb) Music • Soundcard with midi instrument bank • Keyboard and leads

13. Software • Wav editor • Audacity • Vegas / Soundforge • Wavelab • Multitrack recorder • Cubase • Sonar • Logic Audio • FruityLoops • Processors (echo, EQ, reverb, distortion) • Native instruments • Sonic foundry

14. Screenshots of digital recording applications – Soundforge and Sonar

15. File types / soundsources • Midi (just musical data) • Wav • MP3 • Aiff • Digital synthesis Digital recordings

16. MIDI – a special case Musical Instrument Digital Interface (MIDI) the first plug and play multimedia networking protocol, introduced in the early eighties. Musical instruments were used to send messages to each other along the lines of “Give me a loud middle C on the piano for two seconds”. Control data is sent to synthesisers on the PC’s sound card A whole song can be transmitted in a few hundred bytes No vocals – only musical sounds! Challenge: Play with the piano roll in Fruityloops in AS125.

17. Screenshot of a ‘piano roll’ editor for midi music

18. Human hearing • 40 hz – 20 Khz at birth • 40 hz – 15 Khz at age 18 • Downhill from there (Ipod users please note!)

19. Nyquist rate • Scientist called Nyquist proposed that for humans not to hear digital ‘steps’ in recorded sound - it would need to be sampled at twice human hearing frequency • 22Khz X 2 = 44 Khz • CD quality audio is sampled at a rate of 44100 times a second

20. Amplitude Analogue Time Diagram showing a ‘smooth’ analogue waveform compared to a ‘stepped’ digital waveform. Samples Amplitude Digital

21. Problem with digital audio • 1 minute of digital audio takes about 10 Mb of disk space. How do we get round this to distribute sound on the web?

22. Problem with digital audio • Solution 1 – lower sample rate • 44 Khz = 10 Mb minute • 22 Khz = 5 Mb minuter • 11 Khz = 2.5 Mb minute

23. Problem with digital audio • Solution 2 – Compression • MP3 = .5 Mb per minute or less

24. MP3 Compression: How it works • When we listen: Important data • High frequency sounds • Short duration sounds • Rapid changes in tone Non-important data • Lower frequency sound • Continuous sound (hums and hiss)

25. Compression Works by “throwing away” the non important data • E.g. – there is no need for 2 channels of ‘bass’ when we can’t spatially locate low frequencies (e.g. sub woofer) • Finding “similar” parts of a sample and replacing with 1 occurrence. (just GIF in image compression) • Challenge: Render the same sound file from Vegas in AS127 – first as a .wav file, then as a .mp3 file – compare file sizes.

26. Digital Synthesis Google: VSTi, Native Instruments, virtual synthesizers. • ‘Old’ – hardware synthesisers – used oscillators (sound sources), filters (sound modifiers) and envelopes (sound shapers) – to produce sound. • Digital synthesisers ‘emulate’ this hardware in software– by modelling the ‘real’ hardware (with logic gates, filters, etc) • Challenge: Play with the virtual synthesisers that come with fruity loops in AS125.

27. Using sound imaginatively • There is no reason why you can’t use sound as a tool for communication in web design. • Remember a voice can ‘personalise’ information and ‘build a relationship’ with a listener in ways that plain text cannot. • Music can provide a sense of ‘place and time’ – a bit of militant scrap-house or D’n’B might help make pages seem more relevant to a student audience – it may also irritate them…. • Remember heuristic rules though – allow users to turn off sounds, or navigate through long sound files.