Video

1. 7 Video

2. Moving Pictures • Vision is probably the most important among our senses • Persistence of vision allows images shown in rapid succession to produce the illusion of motion • Motion obtained above about 15 to 20 images (or frames) per second • At lower frame rates, individual frames are noticeable – “flickering” • At higher frame rates, motion becomes smooth • An image is worth a thousand words • Each second of video has about 25 images • Therefore, a second of video is worth about 25,000 words

3. Generate Moving Pictures • Video • Use video camera to capture a sequence of frames • Animation • Generate each frame individually either by computer or by other means

4. Basics of Video • Analog video is represented as a continuous (time varying) signal • Digital video is represented as a sequence of digital images • Digital video when duplicated will always retain the same original quality for an infinite amount of time • Crisp, and the highest quality video • Can be kept on DVD discs or CD-ROMs

5. Types of Color Video Signals • Component video • Each component is sent as a separate video signal • Three components: Y (luminance), U and V (color) • Often use in production and post-production • Best color reproduction • Requires more bandwidth and good synchronization of the three components • Composite video • Combine three components into a signal • Color component (U and V) is allocated half bandwidth as the luminance (Y) • Some interference between the two signals is inevitable • Often use in transmission

6. … Types of Color Video Signals • S-video (Separated video) • Separates the luminance from the two color (total two signals) • A compromise between component video and the composite video

7. Television Broadcast Standards • NTSC • 1952 by National Television Standards Committee • Used in: USA, Japan, Korea, and others • 525 lines/frame (vertical resolution) • Two passes to draw a single frame at a rate of 60Hz per second (interlacing) • 30 frames/second • Resolution approx. 640 x 480 (4:3 ratio) • uses YIQ color model • "Never The Same Color"

8. … Television Broadcast Standards • PAL • Phase-Alternative Line system • Western Europe except France • 625 lines/frame • Interlaced at 50Hz • 25 frames/second • uses YUV color model • SECAM • The Sequential Color and Memory system • France and Eastern Europe • 625 lines/frame • 50Hz • 25 frames/second

9. … Television Broadcast Standards • HDTV • High-Definition Television • The Image of the new millennium • screen picture quality similar to 35mm film, along with compact disc (CD) sound quality • Some American television stations began transmitting digital HDTV in 1998 • Format Wars (18 formats) active lines active aspect ratio frame rate horizontal pixels 720 1280 16/9 progressive 24, 30 or 60 1080 1920 16/9 interlaced 60 1080 1920 16/9 progressive 24, 30

10. … Television Broadcast Standards • Features • Higher-resolution picture • Wider picture • Digital surround sound (Dolby Digital (AC-3) ) • Additional data • DTV stands for digital television • All HDTV broadcasts in the United States would be digital

11. How Video Works • When light reflected from an object passes through a video lens, that light is converted into a electronic signal by a special sensor called a charged-coupled device (CCD) • The signal from the camera contains three channels of color (RGB) and synchronization pulses (sync) • The video signal is recorded on magnetic tapes • One or two channels of sound may also be recorded on the tape • Video Cassette Recorder (VCR) Audio track Each strip represents information for one field of a frame Video track Control track

12. Video Color • One luminance component • Two chrominance components • Original TV was black and white • Adding color had to be done in a compatible way • NTSC: YIQ • PAL: YUV • In general: YUV and YCrCb used as terms

13. Recording Formats • YUV: Luminance, Hue, Saturation R B G • Luminance (Y) .30 .11 .30 • Hue (U) .30 .87 .59 • Saturation (V) .70 .11 .59 • 4:1:1 YUV format • 7-bit Y + U & V data are averaged for 4 pixels • S-VHS Video (s-video) • video signal with enhanced quality used for recording • Color and luminance information are kept on two separate tracks • 4-pin DIN connectors • Oriented toward consumer with gaining acceptance among lower-end broadcaster 1 Ground 2 Ground 3 Intensity (Luminance) 4 Color (Chrominance)

14. … Recording Formats • Video recording format • VHS: Japan Victor Company • VHS-C: VHS-compact used in camcorders • Component (YUV) • Laying the signal on the tape in three channels • Four channels of audio • Sony Betacam SP • Component Digital • The signal is converted to digital information and stored as bytes • Can make unlimited copies without loss of quality • Digital Betacam • High price tag

15. Video Hardware Resolution • The lens used and the number, size, and quality of the CCDs determine the resolution • Resolution (in horizontal lines) • Video Type Resolution8 mm 230VHS 240S-VHS 400Hi-8 400Beta-SP 550MII (Panasonic) 550Broadcast-quality 1000 • Hi-8 is the most widely accepted format used for industrial and corporate video communication

16. Digital Video • A video sequence consists of a number of frames • Each frame is a single image produced by digitizing time-varying signal generated by video camera • CCIR 601 is a standard established for digitizing NTSC and PAL signals • specifies the image format, and coding for digital television signals • It uses Y'CBCR color with 4:2:2 chrominance sub-sampling. The data rate is 166Mbits per sec

17. … Digital Video • Think about the size of the uncompressed digital video • NTSC video format • Bitmapped images for video frame • 640  480 pixels with 24-bit color = 0.9 MB/frame • 30 frames per second • 900 kb/frame  30 frames/sec = 26 MB/sec • 60 seconds per minute • 26 MB/sec  60 secs/minute = 1,600 MB/minute • Strains on current processing, storage and data transmission !

18. Create Digital Video • Get analog/digital video signal from • video camera • video tape recorder (VTR) • broadcast signal • Digitize analog video & compress it

19. Digitizing Analog Video • In computer • Video capture card • Convert analog to digital & compress • Can also decompress & convert digital to analog • Compress through • Video capture card (hardware codec) • Software (software codec)

20. Digitizing Analog Video • In camera • Digitize and compress using circuitry inside camera • Transfer digitized signal from camera to computer through • IEEE 1394 interface (FireWire): 400 Mb/sec • USB: 12Mb/sec(version 1.1) ~ 480 Mb/sec(version 2.0)

21. Digitize in Computer v.s. Camera • Digitize in camera • Advantage • Digital signals are resistant to corruption when transmitted down cables and stored on tape • Disadvantage • User has no control between picture quality and data rate (file size)

22. Digital Video Camera • DV camcorders store up to 90 minutes of digital video on small DV cartridges • VHS tape format • Encoded digitally • Digital8 camcorders store digital video on Hi-8 tapes and can also play Hi-8 and 8mm tapes • Digital8 cameras are usually larger than DV cameras • DVD Camcorder • Video Resolution • The vertical resolution of a digital video image is determined by the number of scan lines in the image • The horizontal resolution is the rate at which the moving beam can turn on and off to paint "dots" of color on the screen

23. Better image quality than previous analog formats

24. Video Tape Formats

25. Videotape & Digital Recorders/Players

26. Digital Cameras

27. Video Capture Cards • Pinnacle Systems • DV series • TARGA series • ViewCast • Osprey series • Capture • to disk • to memory • full-screen • drop frames • non-real-time step-frame

28. Film & Video Editing • Traditional • Timecode • SMPTE timecode • Hours, minutes, seconds, frames • VHS • Two copying operations is to produce serious loss of quality • Constructed linearly

29. Video Production • Pre-production • script writing • storyboarding • production schedule for shooting the scene • Production involves shooting the scene • Post Production • editing the best scenes into the final video program • rough cut • final cut

30. Video Time • Time base • how time is divided in your project • time base of 30 means each second is divided into 30 units • frame rate • number of frames per second • frame rate of 30 means the camera records the scene every 1/30th of a second • there are 30 frames per second

31. Video Time • Time code • how the frames are counted • SMPTE time code (society of motion pictures and tv engineers) • hh:mm:ss:frames • 00:05:31:15 - 5 minutes, 31 seconds, 15 frames

32. Steps • Import clips • still image • audio • video (capture) • Editing • Compositing • Reverse shot • Conversation between two people • Transitions • Titles/motion • Export

33. Digital Video Editing • Analog editing systems • Avid video editing software • Avid Xpress DV • Adobe • Premiere • After Effects • Microsoft Windows Movie Maker • Pinnacle Systems • ReelTime • Digital video editing software • conversion and compression • capture • editing • transition effects • composition • audio editing • motion path control • text editing

34. … Digital Video Editing • Random access • Non-destructive • Premiere • The standard mid-rangevideo editing application • Three main windows • Project, timeline, monitor • Timelines • Have several video tracks • Transitions • Cuts and Transitions • In a cut, two clips are butted • In transitions, two clips overlap • Image processing is required to construct transitional frames

35. Digital Video Post-production • Over- or under-exposed, out of focus, color cast, digital artifacts • Provide image manipulation programs • Adjust level, sharpen, blur • The same correction may be needed for every frame, so the levels can be set for the first frame and the adjustment will be applied to as many frames as user specifies. • If light fades during a sequence, it will necessary to increase the brightness gradually to compensate. • Apply a suitable correction to each frame and allow their values at intermediate frames to be interpolated • Varying parameter values over time

36. Keying • Selecting transparent areas • Blue screening • Chroma keying: any color • Alpha channel • Luma keying: a brightness threshold is used to determine which areas are transparent • Select explicitly • Create mask • In film and video, mask is called matte • Matte out: removing unwanted elements • Split-screen effects • Alpha channel created in other application

37. Track matte • Chroma keying and luma keying • Color and brightness changes between frames • Use a sequence of masks as matte • Separate video track: track matte • Track matte • Painstaking by hand • Generated from a single still imageapplying simple geometrical transformations over time to create a varying sequence of mattes

38. Adobe After Effects • Apply a filter to a clip and vary it over time • A wide range of controls for the filter’s parameters • Premiere: parameter values are interpolated linearly between key frames • After effect: interpolation can use Bezier curves

39. Preparing Video for Multimedia Delivery • Frame size, frame rate, color depth, image quality • People sit close to monitors, so a large picture is not necessary • Higher frame rates are needed to eliminate flicker only if display is refreshed at the same rate. • Computer monitors are refreshed at a much higher rate from VRAM. • Limiting colors • Not all codecs support

40. The Bits and Bytes of Digital Video • Data rates of digital video depends on • Frame rate: Frames per second (TV/HDTV/DVD uses 24-30fps) • Frame dimensions: The width and height of the image expressed in number of pixels • TV: 640 x 480 pixels • DVD: 720 x 576 pixels • HDTV: 1,920 x 1,080 pixels • Pixel depth: The number of bits per pixel • TV/DVD: 16 bits • HDTV: 24 bits

41. … The Bits and Bytes of Digital Video • If we have DVD quality video, the uncompressed data stream would be • 720 x 576 pixels * 16 bits/pixel * 30 fps = 199 Mbits/second • HDTV quality requires significantly more • 1,920 x 1,080 pixels * 24 bits/pixel * 30 fps = 1,492 Mbits/second • Either way, this is huge amount of data

42. 7.2 Video Video Formats and Compression

43. Video Formats • VCD (Video Compact Disc) • A compact disc with full motion video on it • Once converted to MPEG (Motion Picture Experts Group), it can be put onto a CD • Has capability to hold 650-700MB of quality video (approximately 74 minutes) • Can be played on most DVD players, Computers with a CD-ROM drive, Sony Playstations, Sega Dreamcasts, etc. • DVD (Digital Video Disc) • A digital video disc that can store up to 2 hours of high quality video • 4.7 gigabytes of information • Played on DVD players and computers with DVD-ROM

44. Video Formats • Streaming video: • AVI • MOV • RM • WMV • MPEG • Other popular multimedia formats (not pure video formats): • Shockwave • Flash

45. Video Throughput • The filesize (download time) that a video requires is dependent on: • Window size (e.g. 640x480) • Frames per second (e.g. 30 fps) • Compression rate • Inversely impacts image quality • Quality also depends on compression algorithm

46. Video Throughput • If you thought we needed compression with audio… • At 24bbp, 640x480, 30fps (which would produce a fairly standard looking TV video) it would take about 100GB to store an hour of uncompressed video

47. Video Throughput • Videos can be fully downloaded to your computer and then played • On formats that support streaming you can start watching the video before it is done loading • Or videos can be played off a “streaming server” and sent to your computer via a special protocol • You don’t get a copy of the file on your system

48. Video Throughput • Because the streaming method is quite popular on the internet, video throughput is also often stated in terms of the speed of your connection • Your computer can received (and play) X bits per second of video information • So the actual size of the video file is not as important as the bandwidth required to play the video • MPEG2 videos require a connection that can handle 100Mbits/second in order to show the video in real-time (without large buffering pauses in the middle of the movie)

49. AVI • Microsoft’s Audio Video Interleave • The “Interleave” in the name is associated with the fact that packets of Video / image data are interwoven with audio data • One of the first video formats on the PC • Released with Windows 3.1 (early 90s) • Considered an “old” format • Compression is poor (long download times) • Image quality is poor • Cannot be streamed • Makes is a bad choice for the web

50. MOV • QuickTime Movie • Developed by Apple • Was one of the first multitrack formats • Can have different sound tracks aimed at different languages and then only using one of them • Also released in the early 90s • Compression is average (average bandwidth needed) • Image quality is good • Can be streamed • Also useful as an editing format -- not just for delivery