SWE 423: Multimedia Systems

1. SWE 423: Multimedia Systems Chapter 3: Audio Technology (2)

2. Audio Signal Representation • Waveform representation • Focuses on the exact representation of the produced audio signal. • Parametric form representation • Focuses on the modeling of the signal generation process. • Two major forms • Music synthesis (MIDI Standard) • Speech synthesis

3. Waveform Representation Audio Source Human Ear Audio Capture Playback (speaker) Sampling & Digitization Storage or Transmission Digital to Analog Receiver Audio Generation and Playback

4. Digitization • To get audio (or video for that matter) into a computer, we must digitize it (convert it into a stream of numbers). • This is achieved through sampling, quantization, and coding.

5. Example Signal Amplitude

6. Sampling • Sampling: The process of converting continuous time into discrete values.

7. Sampling Process • Time axis divided into fixed intervals • Reading of the instantaneous value of the analog signal is taken at the beginning of each time interval (interval determined by a clock pulse) • Frequency of clock is called sampling rate or sampling frequency • The sampled value is held constant for the next time interval (sampling and hold circuit)

8. Sampling Example Amplitude

9. Quantization • The process of converting continuous sample values into discrete values. • Size of quantization interval is called quantization step. • How many values can a 4-bit quantization represent? 8-bit? 16-bit? • The higher the quantization, the resulting sound quality .............

10. Quantization Example Amplitude

11. Coding • The process of representing quantized values digitally

12. Analog to Digital Conversion Amplitude

13. MIDI Interface • Musical sound can be generated, unlike other types of sounds. • Therefore, the Musical Instrument Digital Interface standard has been developed • The standard emerged in its final form in August 1982 • A music description language in binary form • A given piece of music is represented by a sequence of numbers that specify how the musical instruments are to be played at different time instances.

14. MIDI Components • An MIDI studio consists of • Controller: Musical performance device that generates MIDI signal when played. • MIDI Signal: A sequence of numbers representing a certain note. • Synthesizer: A piano-style keyboard musical instrument that simulates the sound of real musical instruments • Sequencer: A device or a computer program that records a MIDI signal. • Sound Module: A device that produces pre-recorded samples when triggered by a MIDI controller or sequencer

15. MIDI Components

16. MIDI Data MIDI File Organization • Describes • Start/end of a score • Intensity • Instrument • Basis frequency • .... Track 1 Track 2 Header Chunk Track Header Track Chunk Track Header Track Chunk Actual Music Data Status Byte Data Bytes Status Byte Data Bytes

17. MIDI Data • MIDI standard specifies 16 channels • A MIDI device is mapped onto one channel • E.g. MIDI Guitar controller, MIDI wind machine, Drum machine. • 128 instruments are identified by the MIDI standard • Electric grand piano (2) • Telephone ring (124) • Helicopter (125) • Applause (126) • Gunshot (127)

18. MIDI Instruments • Can play 1 single score (e.g. flute) vs. multiple scores (e.g. organ) • Maximum number of scores that can be played concurrently is an important property of a synthesizer • 3..16 scores per channel.

19. 3D Sound Projection • After experimentation, it was shown that the use of a two-channel sound (stereo) produces the best hearing effects for most people.

20. Spatial Sound • Direct sound path: The shortest path between the sound source and the auditor. • Carries the first sound waves to the auditors head. • All other sound paths are reflected • All sound paths leading to the human ear are influenced by the auditor’s individual head-related transfer function (HRTF) • HRTF is a function of the path’s direction (horizontal and vertical angles) to the auditor. Pulse response in a closed room