1 / 23

CS 414 – Multimedia Systems Design Lecture 12 – MPEG Video (Basic Principles)

CS 414 – Multimedia Systems Design Lecture 12 – MPEG Video (Basic Principles). Klara Nahrstedt Spring 2014. Administrative. MP1 Submission Deadline February 19 (Wednesday) 5pm You can use bonus day(s) – up to two if needed Demonstrations February 21 (Friday) 5-7pm

galena
Télécharger la présentation

CS 414 – Multimedia Systems Design Lecture 12 – MPEG Video (Basic Principles)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 414 – Multimedia Systems DesignLecture 12 – MPEG Video (Basic Principles) Klara Nahrstedt Spring 2014 CS 414 - Spring 2014

  2. Administrative • MP1 • Submission Deadline • February 19 (Wednesday) 5pm • You can use bonus day(s) – up to two if needed • Demonstrations • February 21 (Friday) 5-7pm • Demonstration room 0218 SC • Sign up process for demonstration – see piazza postings CS 414 - Spring 2014

  3. Administrative • ¼ Unit Project • Proposal submission deadline • February 28 (Friday) 11:59pm • Proposal topics and writeup requirements • See cs414 class website • Homework 1 • Posted on February 24 (Monday) • Deadline on March 3 (Monday), 5pm (no bonus days) • Midterm • March 7 (Friday) in class (closed book) • 1 site of cheat-sheet allowed and bring calculator!!

  4. Outline • MPEG Hybrid Coding • Reading: • Media Coding Book, Section 7.7.1-7.7.4 and lectures • A good book on JPEG/ MPEG Audio/Video Fundamentals is : • Haskell, Puri, Netravali, “Digital Video: An Introduction to MPEG-2”, Chapman and Hall, 1996 • Multimedia Systems Book (Havaldar and Medioni), Chapter 8, Sections 1-2. CS 414 - Spring 2014

  5. Comment on JPEG Losses Losses in JPEG compression occur because S(u,v)’ = Round(S(u,v)/Q(u,v)). Image Preparation => DCT Transform S(u,v) = DCT(S(x,y)) => Quantization with S(u,v)’ = Round(S(u,v)/Q(u,v) => Entropy Encoding => Transmission of compressed Image => Entropy Decoding => De-quantization S(u,v)’’ = S(u,v)’*Q(u,v) Compression Loss since S(u,v)” ≠ S(u,v) => Apply Inverse DCT => Display Image CS 414 - Spring 2014

  6. Motion Picture Expert Group (MPEG) • General Information about MPEG • Began in 1988; Part of Same ISO as JPEG • MPEG-1/Video • MPEG/Audio – MP3 • MPEG-2 • MPEG-4 • MPEG-7 • MPEG-21 CS 414 - Spring 2014

  7. MPEG General Information • Goal: data compression 1.5 Mbps • MPEG defines video, audio coding and system data streams with synchronization • MPEG information • Aspect ratios: 1:1 (CRT), 4:3 (NTSC), 16:9 (HDTV) • Refresh frequencies: 23.975, 24, 25, 29.97, 50, 59.94, 60 Hz CS 414 - Spring 2014

  8. MPEG Image Preparation (Resolution and Dimension) • MPEG defines exactly format • Three components: Luminance and two chrominance components (2:1:1) • Resolution of luminance comp:X1 ≤ 768; Y1 ≤ 576 pixels • Pixel precision is 8 bits for each component • Example of Video format: 352x240 pixels, 30 fps; chrominance components: 176x120 pixels CS 414 - Spring 2014

  9. MPEG Image Preparation - Blocks Each image is divided into macro-blocks Macro-block: 16x16 pixels for luminance; 8x8 for each chrominance component Macro-blocks are useful for Motion Estimation No MCUs which implies sequential non-interleaving order of pixels values CS 414 - Spring 2014

  10. MPEG Video Processing I B B P B B P B B P B B I • I-Frame: Intra frames (same as JPEG) • typically about 12 frames between I frames • P-Frame: Predictive frames • encode from previous I or P reference frame • B-Frame: Bi-directional frames • encode from previous and future I or P frames CS 414 - Spring 2014

  11. Selecting I, P, or B Frames • Heuristics • change of scenes should generate I frame • limit B and P frames between I frames • B frames are computationally intense CS 414 - Spring 2014

  12. MPEG Video I-Frames Intra-coded images I-frames – points of random access in MPEG stream I-frames use 8x8 blocks defined within Macro-block No quantization table for all DCT coefficients, only quantization factor CS 414 - Spring 2014

  13. MPEG Video P-Frames Motion Estimation Method Predictive coded frames require information of previous I frame and or previous P frame for encoding/decoding For Temporary Redundancy we determine last P or I frame that is most similar to the block under consideration CS 414 - Spring 2014

  14. Motion Computation for P Frames • Predictive search • Look for match window within a given search window • Match window – macro-block • Search window – arbitrary window size depending how far away are we willing to look • Displacement of two match windows is expressed by motion vector CS 414 - Spring 2014

  15. Matching Methods • SSD metric • SAD metric • Minimum error represents best match • must be below a specified threshold • error and perceptual similarity not always correlated CS 414 - Spring 2014

  16. Example of Finding Minimal SSD CS 414 - Spring 2014

  17. Example of Comparing Minimal SSD and SAD CS 414 - Spring 2014

  18. Syntax of P Frame Addr: address the syntax of P frame Type: INTRA block is specified if no good match was found Quant: quantization value per macro-block (vary quantization to fine-tune compression) Motion Vector: a 2D vector used for motion compensation provides offset from coordinate position in target image to coordinates in reference image CBP(Coded Block Pattern): bit mask indicates which blocks are present CS 414 - Spring 2014

  19. MPEG Video B Frames Bi-directionally Predictive-coded frames CS 414 - Spring 2014

  20. MPEG Video Decoding Display Order Decoding Order I1 B1 B2 P1 B3 B4 P2 B5 B6 P3 B7 B8 I2 I1 P1 B1 B2 P2 B3 B4 P3 B5 B6 I2 B8 B7 CS 414 - Spring 2014

  21. MPEG Video Quantization • AC coefficients of B/P frames are usually large values, I frames have smaller values • Adjust quantization • If data rate increases over threshold, then quantization enlarges step size (increase quantization factor Q) • If data rate decreases below threshold, then quantization decreases Q CS 414 - Spring 2014

  22. MPEG-1 Interchange Format Sequence Layer Seq Seq Seq … Seq SeqSC QT, misc Video Param Bitstream Param ... GOP Layer GOP GOP GOPSC Time Code GOP Param ... Picture Layer Pict Pict PSC Type Buffer Param Encode Param ... Slice Layer Slice Slice Vert Pos ... Macro-block Layer SSC QScale MB MB Addr QScale Type Motion Vector ... Block Layer CBP b0 b5 CS 414 - Spring 2014

  23. Conclusion • MPEG system data stream • Interchange format for audio and video streams • Interleave audio and video packets; insert time stamps into each “frame” of data • Synchronization • SRC – system clock reference; DTS – decoding time stamp; PTS – presentation time stamp • During encoding • Insert SCR values into system stream • Stamp each frame with PTS and DTS • Use encode time to approximate decode time • During decoding • Initialize local decoder clock with start values • Compare PTS to the value of local clock • Periodically synchronize local clock to SCR CS 414 - Spring 2014

More Related