Motivation

1. Motivation • Application driven -- VoD, Information on Demand (WWW), education, telemedicine, videoconference, videophone • Storage capacity • Large capacity disks (several gigabytes) • Increase in storage bandwidth,e.g. disk array technology • Network bandwidth • high speed fiber optic networks - gigabit networks • fast packet switching technology

2. Challenges • Data intensive -- compression, high speed communication networks, high performance work stations • Time dimension -- strict time constraints for retrieval, transmission and presentation • Synchronization -- temporal and spatial relationships among different media • Techniques for indexing, retrieval, and recognition of multimedia information

3. Essentials • Data compression to reduce demands on storage space and transmission bandwidth • A multimedia operating system, transport protocol, schedulers to provide appropriate delay and jitter guarantees • High performance work stations and high speed networks to handle high bit rates under time constraints • Media synchronization to maintain both spatial and temporal relationships among related media • High performance retrieval engines • QoS specification and performance guarantees

4. Multimedia Information Representation • All media types are stored and processed in digital form • text, images (discrete, measured in units like bits) • audio, video (continuous, measured in bits per sec) • analog signals which must be converted into digital form

5. Analog Signals • A signal whose amplitude varies continuously with time (for example, electrical signals produced by microphones) • The amplitude of the signal indicates the magnitude of the sound-wave/image intensity at that time instant • Signal encoder to convert analog to digital (to store and process within a computer) • Signal decoder to convert digital to analog (to present and display on loudspeakers or computer monitors) • Advantage of all digital form -- different media types can be readily integrated and transmitted over a single all digital communication network

6. Encoder • Bandlimiting filter used to remove selected higher frequency components from the source signal • analog-to-digital converter (ADC) consists of a sample and hold circuit and a quantizer • sample and hold is used to sample the amplitude of the filtered signal at regular time intervals and to hold the sample amplitude constant between samples • quantizer circuit converts each sample amplitude into a binary value

7. Sampling Rate • How do we choose the sampling rate? • Nyquist sampling theorem which states that to obtain an accurate representation of a time-varying analog signal, its amplitude must be sampled at a minimum rate that is equal to or greater than twice the highest sinusoidal frequency component that is present in the signal.

8. Undersampling • Alias Signals • Frequency components are generated which were not present in the original signal • Sampling rate is determined by the transmission channel bandwidth which is lower than that of the source signal • A bandlimiting filter will remover higher frequencies before the signal is sampled

9. Quantization • Quantization Interval 2S/2n • Number of quantization levels 2n • Number of bits required to represent a sample n • Quantization error or Quantization noise (Quantization step) -- maximum difference between the quantized sample value and the corresponding analog signal values • larger the number of quantization levels, the smaller the quantization step and the smaller the quantization noise.

10. SNR • Signal-to-Noise ratio (SNR) : the digital signal quality relative to the original analog signal measured in dB SNR = 20log10(S/N) where S is the max. signal amplitude and N is the quantization noise. • High SNR means a good quality signal.

11. Summary • Sampling rate and quantization are high, then the digitized signal will be a close representation of the original analog signal.

12. Digital Audio • Speech signal bandwidth 50Hz to 10kHz • Music signal bandwidth 15 Hz to 20 kHz • Bit rate for speech : sampling rate 20 kHz or 20 ksps • at 12 bits per sample, 240 kbps bit rate • Bit rate for music at 16 bits per sample : 640 kbps (mono) or 2 X 640k = 1280 kbps (stereo) • Storage or memory required for a 10 minute Bach at stereo sound: • (1280 X 1000 X 600) / 8 = 96 MBytes

13. CD-DA Standard • Sampling rate is 44.1 ksps and 16 bits per sample and the bit rate is 1.411 Mbps • 60 minute multimedia title requires 634.95 Mbytes • Time to transmit a 30 sec portion of the title using a transmission channel of bit rate: at 64 kbps = (42.33 X 106)/(64 X 103) = 661.4 s or about 11 minutes at 1.5 Mbps = 28.22s

14. Digital Video • Video information units FILM -- NUMBER OF CLIPS CLIP -- NUMBER OF FRAMES FRAME -- NUMBER OF BLOCKS BLOCK -- 8x8 OR 16 x16 PIXELS PIXEL -- 6, 16 OR 24 BITS (pixel depth and determines the range of different colors that can be produced.

15. Digital Video • A collection horizontal scanning lines forms a frame • Horizontal and vertical blanking intervals • Movement of the scene is captured by a sequence of sampled frames • Frame rate, at least 25 frames per second for smooth motion • Aspect ratio -- the ratio of an image’s width to its height • Horizontal resolution - max number of black and white alternating vertical lines for given horizontal distance • Vertical resolution - number of horizontal scan lines in a frame

16. Color Specification • Primary colors, R, G, B -- Magnitude of the three signals in proportion 0.299R + 0.587G + 0.114B produces white color on the screen • Luminance (brightness) component -- amount of white light a source contains -- Y = 0.299R + 0.587G + 0.114B • Chrominance (hue and saturation) components -- represents the actual color of the source and strength or vividness of the color , Cb = B - Y and Cr = R - Y

17. 4:2:2 Format • In principle, for digital TV, RGB signals could be transmitted separately. The disadvantage is that they need to be sampled at the same rate and bits per sample -- 4:4:4 • In 4:2:2 format, for every 4Y samples, there are 2 Cb and Cr samples each.