1 / 63

Multimedia – Digital Audio & Video

Multimedia – Digital Audio & Video CSC1720 – Introduction to Internet Essential Materials Outline Audio / Video on the Web Basic Digital Audio Concepts Streaming Audio Web Audio Formats Basic Digital Video Concepts Video compression/decompression methods. Video File Formats

Audrey
Télécharger la présentation

Multimedia – Digital Audio & Video

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multimedia –Digital Audio & Video CSC1720 – Introduction to Internet Essential Materials

  2. Outline • Audio / Video on the Web • Basic Digital Audio Concepts • Streaming Audio • Web Audio Formats • Basic Digital Video Concepts • Video compression/decompression methods. • Video File Formats • Other Multi-media Formats • Summary All copyrights reserved by C.C. Cheung 2003.

  3. AV on the Web? • It is part of the Internet. • Radio Stations • Music, sound clips • Streaming Audio / Video • Video conferencing • Digital Cameras • Animation on the Web • Shockwave, Flash All copyrights reserved by C.C. Cheung 2003.

  4. Basic Terms All copyrights reserved by C.C. Cheung 2003.

  5. Analog vs. Digital • Analog: A signal that has a continuous nature rather than a pulsed or discrete nature. • Note: Electrical or physical analogies, such as continuously varying voltages, frequencies, or phases, may be used as analog signals. • Digital: A signal in which discrete steps are used to represent information. All copyrights reserved by C.C. Cheung 2003.

  6. Digital Signal • Use Pulse Code Modulation (PCM) to represent an audio signal by digital data. All copyrights reserved by C.C. Cheung 2003.

  7. ADC & DAC Figure 4.3 Conversion from Analog to Digital requires an Analog-to-Digital Converter Figure 4.4 Conversion from Digital to Analog requires a Digital-to-Analog Converter All copyrights reserved by C.C. Cheung 2003.

  8. Basic Digital Audio Concepts • Sampling rate • Number of sample taken of a signal in a given time (usually one second) • Bit depth • Describes the accuracy of the audio data • Channels • E.g. Stereo has two channels • Bit rate • Measured in kilobits per second (Kbps) is a function of the bit depth and sampling rate All copyrights reserved by C.C. Cheung 2003.

  9. Sampling rate • The more sample taken per second, the higher the accuracy. • Typically measured in kilohertz (KHz). • CD audio has 44,100 samples per second (44.1KHz). • 8 KHz produces lower quality radio sound. • Standard sampling rates include “8KHz”, “11.025KHz”, … • The high-end 96K is used in DVD, but is not applicable to the Web. All copyrights reserved by C.C. Cheung 2003.

  10. Sampling Rate demo All copyrights reserved by C.C. Cheung 2003.

  11. Bit depth • Also called “sampling resolution” or “word length”. • The more bits, the better is the quality of the audio (and a larger file of course). • Common bit depths are 8-bit (telephone like), 16-bit (CD quality), and 20, 24, 32, 48-bit depths. • How many signal can a 8-bit and a 16-bit data represent? • 0000 0000  1111 1111 • 0000 0000 0000 0000  1111 1111 1111 1111 All copyrights reserved by C.C. Cheung 2003.

  12. Quantization demo All copyrights reserved by C.C. Cheung 2003.

  13. Channels • Audio file can support one to six channels of audio formats. • Mono – one channel • Stereo– two channels • Some others– three, four channels. • Six channels– 5.1-channel surround sound. • More multi-channel formats announce in the coming years. All copyrights reserved by C.C. Cheung 2003.

  14. Channel Examples All copyrights reserved by C.C. Cheung 2003.

  15. Bit rate • Audio files are measured in terms of bit rate which is measured in kilobits per second (Kbps). • It can be calculated by dividing the file size by the time (in second) to play the audio clip. • E.g. 3Mb file play in 30 seconds • 3000k / 30 = 100kbps. • Quality at different compression rates All copyrights reserved by C.C. Cheung 2003.

  16. Optimizing for the Web • Length of the audio clip • Keep the audio clip as short as possible. • E.g. just keep the most sweetest part of your greeting. • Number of channels • A mono audio file is halved the space of a stereo file. • Depends on your actual needs. All copyrights reserved by C.C. Cheung 2003.

  17. Optimizing for the Web • Bit depth • Audio file on the Web are usually 8-bits. • Half the size of a 16-bit file. • Sampling rate • Half the sampling rate will also halve the space needed. • Voice only audio file can be reduced to 8KHz. • 22 KHz music clips are acceptable. • Putting all things together: Mono, 8-bit, 22KHz, MP3 compression. All copyrights reserved by C.C. Cheung 2003.

  18. Calculate audio size • 8-bit mono: seconds x KHz • 16-bit mono: (seconds x KHz) x 2 • 8-bit stereo: (seconds x KHz) x 2 • 16-bit stereo: ([seconds x KHz] x 2)x2 • E.g. the file size of 30 seconds of 16-bit, 11KHz mono audio: • (30 x 11) x 2 = 660K. All copyrights reserved by C.C. Cheung 2003.

  19. Streaming Audio • What is it? • Play almost immediately after the request, continues playing the transferring data. • Advantages: • Address the problem of long download time. • Control distribution and protect copyright, because the user cannot get a copy of the file. • Disadvantages: • Sound quality may be be affected by low speed or unstable Internet connection. • Reference All copyrights reserved by C.C. Cheung 2003.

  20. What is Streaming? All copyrights reserved by C.C. Cheung 2003.

  21. Packet are sent to a buffer on the receiving Computer, the RealPlayer will play the sound File when buffer full Web browser Web browser request a RealAudio from the web server RealAudio Server Web server How does it work?Streaming audio All copyrights reserved by C.C. Cheung 2003.

  22. Quality Comparison • http://www.cit.cornell.edu/atc/itsupport/streamcompare.shtml All copyrights reserved by C.C. Cheung 2003.

  23. Web Audio Formats • WAV/AIFF (.wav/.aif/.aiff) • The Waveform Audio File format (.wav) was developed by Microsoft, supports arbitrary sampling rates and bit depths. • The Audio Interchange File format (.aif, .aiff) was developed for Macintosh platform. • They are less likely used on the Web, because people use “mp3” or “streaming”. All copyrights reserved by C.C. Cheung 2003.

  24. http://www.nuance.com/prodserv/demo_vocalizer.html WAV/AIFF • Try the Bell-lab synthesis link. • http://www.bell-labs.com/project/tts/voices.html Select the Audio format All copyrights reserved by C.C. Cheung 2003.

  25. MP3 (.mp3) • Able to maintain excellent sound quality at very small file sizes. • The compression reduces an audio file to one-tenth of its original size. • E.g. 40MB file  3.5MB • MP3 is actually MPEG-1 Layer-III • Good for distribution of HQ audio. • Demo:www.mp3.com All copyrights reserved by C.C. Cheung 2003.

  26. What is MP3 digital encoding? • MP3 is actually the file extension for MPEG, audio layer 3. Layer 3 is one of three coding schemes (layer 1, layer 2, and layer 3) for the compression of audio signals. Layer 3 uses perceptual audio coding and psychoacoustic compression to remove all superfluous information. (More specifically, it removes the redundant and irrelevant parts of a sound signal--the stuff the human ear doesn't hear anyway). It also adds a MDCT (Modified Discrete Cosine Transform) that implements a filter bank, increasing the frequency resolution 18 times higher than that of layer 2. All copyrights reserved by C.C. Cheung 2003.

  27. MP3 Players All copyrights reserved by C.C. Cheung 2003.

  28. Apple QuickTime Audio (.mov) • QuickTime is a well-known video format, but it can create audio-only movies. • QuickTime is a container format, which is able to store still images, movie formats, … • Excellent compression, true streaming • Netscape and IE have Plug-in now. • Quicktime :demo All copyrights reserved by C.C. Cheung 2003.

  29. MIDI (.mid/.midi) • MIDI stands for “Musical Instrument Digital Interface” which is developed for electronic musical instruments. • MIDI files are very compact and very good for low-bandwidth delivery. • Instruments are “piano, drums, bass, orchestral strings, …” • It is very attractive for adding MIDI file to your website with very little download time. • Demo:www.findmidis.com All copyrights reserved by C.C. Cheung 2003.

  30. MIDI http://www.findmidis.com/ All copyrights reserved by C.C. Cheung 2003.

  31. RealMedia/RealAudio (.rm/.ra) • RealAudio is a server-based streaming audio format. • The RealServer responses to the requests and delivers the streaming packets, including the bandwidth negotiation. • A robust RealServer can support thousands of simultaneous listeners. • Good for continuous-playing audio and live broadcasts to a large group of people. • Example: RTHK Radio All copyrights reserved by C.C. Cheung 2003.

  32. RealMedia/RealAudio All copyrights reserved by C.C. Cheung 2003.

  33. Windows Media (.wma/.asf) • Windows Media is a streaming system. • It wraps all media elements into a Active Streaming File (.asf). • Audio may be saved as non-streaming Windows Media Audio format (.wma). • Good for continuous-play audio. • The encoder and player is Free, shipped with Windows OS. All copyrights reserved by C.C. Cheung 2003.

  34. Windows Media All copyrights reserved by C.C. Cheung 2003.

  35. Choosing an audio format All copyrights reserved by C.C. Cheung 2003.

  36. Add Audio to your Web Page • A simple link • <A HREF=“audio/song.wav”>Play the song </A> • <A HREF=“music.mp3”><IMG SRC=“buttons/play.gif”></A> • Background Sound • <BGSOUND SRC=“audio/song.mid” LOOP=3> • Link to RealMedia • <A HREF=“song.ram”>Link to the song</A> • pnm://domainname.com/song.rm All copyrights reserved by C.C. Cheung 2003.

  37. Break Time – 15 minutes All copyrights reserved by C.C. Cheung 2003.

  38. Basic Digital Video Concepts • Movie length • Frame size • Frame rate • Quality • Color bit depth • Data rate (bit rate) All copyrights reserved by C.C. Cheung 2003.

  39. Movie length • File size is proportional to the movie length. • Videos longer than 1 or 2 minutes cause long download times. • If it is a long video, consider to use streaming video. All copyrights reserved by C.C. Cheung 2003.

  40. Frame size • “Full-screen” video is 640x480 pixels. • The most common frame size for web video is 160x120 pixels. • Not recommend to use a frame size larger than 320x240. • The size depends on the CPU power and the Internet connection bandwidth. All copyrights reserved by C.C. Cheung 2003.

  41. Image and Video? All copyrights reserved by C.C. Cheung 2003.

  42. Frame rate • Frame rate is measured in number of frames per second (fps). • Standard TV-quality video uses 30 fps. • For the web, 15 or even 10 fps is more appropriate and produces fair smooth quality for the user. • Commercial Internet Broadcasts are using 0.5, 0.25 frames per second. All copyrights reserved by C.C. Cheung 2003.

  43. Quality • Many video-editors allow you to set the overall quality of the video. • The degree of compression controls the target quality. • The low or medium setting results a fairly high compression which is appropriate for web delivery. • Frame rate and quality are usually tradeoff in different applications. All copyrights reserved by C.C. Cheung 2003.

  44. Color bit depth • The number of pixel colors in each frame affects the size of the video. • The file size of the video will be greatly reduced by changing the number of colors from 24-bit to 8-bit. • It sacrifices the image quality of the video. All copyrights reserved by C.C. Cheung 2003.

  45. Data rate (bit rate) • This is the rate that the data must be transferred in order to ensure the video can play smoothly without interruption. • It is measured in kilobytes per second (K/sec or Kbps). • It can be calculated by dividing the size of the file (in K) by the movie length (in seconds). • E.g. the video file size is 1.9MB  1900K • Play 40 seconds long, Data rate = 47.5K/sec • Consider the Internet bandwidth! All copyrights reserved by C.C. Cheung 2003.

  46. Calculate space requirements of Video • NTSC video (640 x 480 and 29.97 fps) • Frame size = ([Pixel width x pixel height x bit depth]/8)/1024 • E.g. 200KB/Frame : 6.0 MB/sec • 200KB x 30 fps = 6000KB/s, 6 MB/sec • PAL video (768 x 576 and 25 fps) • E.g. 200KB/Frame : 5.0 MB/sec • 200KB x 25 fps = 5000KB/s, 5 MB/sec All copyrights reserved by C.C. Cheung 2003.

  47. Video CODECs • CODEC is “Compression/Decompression” algorithms. • The sound and frame images of a digital video must be compressed. • The vast amount of data • Compressed in a number of ways • Lossless and Lossy compression • Spatial and Temporal compression All copyrights reserved by C.C. Cheung 2003.

  48. Video Clip Demo • reference All copyrights reserved by C.C. Cheung 2003.

  49. Lossless and Lossy compression • Lossless compression means no information is lost and the final file is the same as the original. • Most compression methods are lossy. • Sacrifices some data from the file in order to achieve higher compression rates. • Use complicated algorithm to toss out some data that is not discernible to the human eye or ear. All copyrights reserved by C.C. Cheung 2003.

  50. Spatial and Temporal compression • Spatial (Intraframe) compression takes place on each individual frame of the video. • Temporal (Interframe) compression applies on a series of frames, it takes advantages of areas of the image remain unchanged from frame to frame. • Relies on the key frames and delta frames. • A key frame is placed once every second. • E.g. 15 fps, a key frame once 15 frames. All copyrights reserved by C.C. Cheung 2003.

More Related