1 / 30

CS 414 – Multimedia Systems Design Lecture 5 – Digital Video Representation

CS 414 – Multimedia Systems Design Lecture 5 – Digital Video Representation. Klara Nahrstedt Spring 2009. Administrative. MP1 is out (January 28) Deadline of MP1 is February 9 (Monday) Demonstration of MP1 will be Monday, February 9 at 5-7pm in 216 Siebel Center

rae
Télécharger la présentation

CS 414 – Multimedia Systems Design Lecture 5 – Digital Video Representation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 414 – Multimedia Systems DesignLecture 5 – Digital Video Representation Klara Nahrstedt Spring 2009 CS 414 - Spring 2009

  2. Administrative • MP1 is out (January 28) • Deadline of MP1 is February 9 (Monday) • Demonstration of MP1 will be Monday, February 9 at 5-7pm in 216 Siebel Center • Sign-up sheet will be available in class on February 9 (during class) CS 414 - Spring 2009

  3. Color and Visual System • Color refers to how we perceive a narrow band of electromagnetic energy • source, object, observer • Visual system transforms light energy into sensory experience of sight

  4. Human Visual System • Eyes, optic nerve, parts of the brain • Transforms electromagnetic energy

  5. Human Visual System • Image Formation • cornea, sclera, pupil,iris, lens, retina, fovea • Transduction • retina, rods, and cones • Processing • optic nerve, brain

  6. Retina and Fovea • Retina has photosensitive receptors at back of eye • Fovea is small, dense region of receptors • only cones (no rods) • gives visual acuity • Outside fovea • fewer receptors overall • larger proportion of rods Retina Fovea

  7. Transduction (Retina) • Transform light to neural impulses • Receptors signal bipolar cells • Bipolar cells signal ganglion cells • Axons in the ganglion cells form optic nerve Bipolar cells Rods Ganglion Cones Optic nerve

  8. Contain photo-pigment Respond to low energy Enhance sensitivity Concentrated in retina, but outside of fovea One type, sensitive to grayscale changes Contain photo-pigment Respond to high energy Enhance perception Concentrated in fovea, exist sparsely in retina Three types, sensitive to different wavelengths Rods vs Cones Cones Rods CS 414 - Spring 2009

  9. Tri-stimulus Theory • 3 types of cones (6 to 7 million of them) • Red = L cones, Green = M cones, Blue = S cones • Ratio differentiates for each person • E.g., Red (64%), Green (32%), rest S cones • E.g., L(75.8%), M(20%), rest S cones • E.g., L(50.6%), M(44.2%), rest S cones • Source of information: • See ‘cone cell’ in wikipedia • www.colorbasics.com/tristimulus/index.php • Each type most responsive to a narrow band • red and green absorb most energy, blue the least • Light stimulates each set of cones differently, and the ratios produce sensation of color

  10. Color Perception (Color Theory) Hue Scale • Hue • distinguishes named colors, e.g., RGB • dominant wavelength of the light • Saturation • Perceived intensity of a specific color • how far color is from a gray of equal intensity • Brightness (lightness) • perceived intensity Original Saturation lightness CS 414 - Spring 2009 Source: Wikipedia

  11. Visual Perception: Resolution and Brightness • Spatial Resolution (depends on: ) • Image size • Viewing distance • Brightness • Perception of brightness is higher than perception of color • Different perception of primary colors • Relative brightness: green:red:blue=59%:30%:11% • B/W vs. Color CS 414 - Spring 2009 Source: wikipedia

  12. Visual Perception: Temporal Resolution CS 414 - Spring 2009 Effects caused by inertia of human eye Perception of 16 frames/second as continuous sequence Special Effect: Flicker

  13. Temporal Resolution • Flicker • Perceived if frame rate or refresh rate of screen too low (<50Hz) • Especially in large bright areas • Higher refresh rate requires: • Higher scanning frequency • Higher bandwidth CS 414 - Spring 2009

  14. Visual Perception Influence Viewing distance Display ratio (width/height – 4/3 for conventional TV) Number of details still visible Intensity (luminance) CS 414 - Spring 2009

  15. Television History • 1927, Hoover made a speech in Washington while viewers in NY could see, hear him • AT&T Bell Labs had the first “television” • 18 fps, 2 x 3 inch screen, 2500 pixels

  16. Television Concepts • Production (capture) • 2D array of light energy to electrical signals • signals must adhere to known, structured formats • Representation and Transmission • popular formats include NTSC, PAL, SECAM • Re-construction • CRT technology and raster scanning • display issues (refresh rates, temporal resolution) • relies on principles of human visual system CS 414 - Spring 2009

  17. Video Representations • Composite • NTSC - 6MHz (4.2MHz video), 29.97 fps • PAL - 6-8MHz (4.2-6MHz video), 25 fps • Component • Maintain separate signals for color • Color spaces • RGB, YUV, YCRCB, YIQ CS 414 - Spring 2009

  18. PAL video standard Y is luminance UV are chrominance YUV from RGB Y = .299R + .587G + .114BU = 0.492 (B - Y)V = 0.877 (R - Y) Color Coding: YUV U-V plane at Y=0.5 CS 414 - Spring 2009 Source: wikipedia

  19. YCrCb • Subset of YUV that scales and shifts the chrominance values into range 0..1 Y = 0.299R + 0.587G + .114BCr = ((B-Y)/2) + 0.5Cb = ((R-Y)/1.6) + 0.5 CS 414 - Spring 2009

  20. YIQ • NTSC standard • YIQ from RGB Y = .299R + .587G + .114B I = .74 (R - Y) - .27 (B - Y)Q = 0.48 (R - Y) + 0.41 (B - Y) YIQ with Y=0.5 CS 414 - Spring 2009 Source: wikipedia

  21. NTSC Video • 525 scan lines per frame; 29.97 fps • 33.37 msec/frame (1 second / 29.97 frames) • scan line lasts 63.6 usec (33.37 msec / 525) • aspect ratio of 4/3, gives 700 horizontal pixels • 20 lines reserved for control information at the beginning of each field • so only 485 lines of visible data CS 414 - Spring 2009

  22. NTSC Video • Interlaced scan lines divide each frame into 2 fields, each of which is 262.5 lines • phosphors in early TVs did not maintain luminance long enough (caused flicker) • scanning also interlaced; can cause visual artifacts for high motion scenes CS 414 - Spring 2009

  23. HDTV • Digital Television Broadcast (DTB) System • Twice as many horizontal and vertical columns and lines as traditional TV • Resolutions: • 1920x1080 (1080p) – Standard HDTV • Frame rate: options 50 or 60 frames per second CS 414 - Spring 2009

  24. Pixel Aspect Ratio • Computer Graphics parameter • Mathematical ratio describing horizontal length of a pixel to its vertical height • Used mainly in digital video editing software to properly scale and render video • Into a new format CS 414 - Spring 2009 Source: wikipedia

  25. CS 414 - Spring 2009

  26. HDTV • Interlaced and/or progressive formats • Conventional TCs – use interlaced formats • Computer displays (LCDs) – use progressive scanning • MPEG-2 compressed streams • In Europe (Germany) – MPEG-4 compressed streams CS 414 - Spring 2009

  27. Aspect Ratio and Refresh Rate • Aspect ratio • Conventional TV is 4:3 (1.33) • HDTV is 16:9 (2.11) • Cinema uses 1.85:1 or 2.35:1 • Frame Rate • NTSC is 60Hz interlaced (actually 59.94Hz) • PAL/SECAM is 50Hz interlaced • Cinema is 24Hz non-interlaced CS 414 - Spring 2009 Source: wikipedia

  28. SMPTE Time Codes • Society of Motion Picture and Television Engineers defines time codes for video • HH:MM:SS:FF • 01:12:59:16 represents number of pictures corresponding to 1hour, 12 minutes, 59 seconds, 16 frames • If we consider 30 fps, then 59 seconds represent 59*30 frames, 12 minutes represent 12*60*30 frames and 1 hour represents 1*60*60*30 frames. • For NTSC, SMPTE uses a 30 drop frame code • increment as if using 30 fps, when really NTSC has only 29.97fps • defines rules to remove the difference error CS 414 - Spring 2009

  29. Take Home Exercise • Given a SMPTE time stamp, convert it back to the original frame number • e.g., 00:01:00:10 CS 414 - Spring 2009

  30. Summary • Digitization of Video Signals • Composite Coding • Component Coding • Digital Television (DTV) • DVB (Digital Video Broadcast) • Satellite connections, CATV networks – best suited for DTV • DVB-S – for satellites (also DVB-S2) • DVB-C – for CATV CS 414 - Spring 2009

More Related