Scope of video coding standardization

1. Introduction to H.264 / AVCVideo Coding StandardMultimedia SystemsSharif University of TechnologyNovember 2008

2. Scope of video coding standardization

3. Evolution of video coding standards of ITU-T VCEG and ISO/IEC MPEG.

4. H.264/AVC • H.264 exploits the same video compression principles as its predecessors • Simulation results have shown that it has substantial superiority of video quality over that achieved by H.263++ and MPEG-4. • H.264 offer significantly higher quality levels for the same bit rates Rate-distortion curve of H.264 codec compared with its predecessors, Foreman QCIF at 10 fps

5. Selected new features in H.264 • Advanced intra-coding • Enhanced motion estimation with variable block size • Multiple reference picture • Integer block transform • Improved deblocking filter • Entropy coding • H.264 Error Resilient Coding

6. H.264 advanced intra-coding (Cont.) • Efficient video encoders mainly use inter-frame prediction, • Use of intra-frame coding for parts of the picture is necessary to prevent error propagation • But intra-frame coding generates a large bit rate, and hence in order for H.264 to be efficient, special attention is paid to intra-frame coding. • H.264 takes advantage of correlations between neighboring blocks to achieve better compression in intra-coding.

7. H.264 advanced intra-coding (Cont.) • Every intra 16 X 16 pixel MB in a picture is first predicted in an appropriate mode from the already coded and re-constructed samples of the same picture.

8. H.264 advanced intra-coding (Cont.) • There are nine advanced intra-prediction modes for the samples when the MB is partitioned into 4 X 4 blocks

9. Advanced inter-coding • Inter-frame predictive coding is where H.264 makes most of its gain in compression efficiency. • Motion compensation (MC) on each 16 X 16 MB can be performed with various block sizes and shapes

10. Multiple reference picture • The H.264 standard also offers the option of using several previous pictures for prediction. • Every MB partition can have a different reference picture that is more appropriate for that particular block. • Increases the coding efficiency and produces a better subjective quality • Improve the robustness of the bitstream to channel errors

11. Deblocking filter • In H.264 codec every reconstructed picture, is filtered by default using an adaptive deblocking filter. • The filter removes visible block structures on the edges of the 4 X4 blocks caused by block-based transform coding and motion estimation

12. Transformation, Quantization, and entropy coding • H.264 employs a 4X4 integer transform • The transform is an approximation of the DCT • It has a similar coding-gain to the DCT transform. • Since the integer transform has an exact inverse operation, there is no mismatch between the encoder and the decoder which was a problem in all DCT based codecs

13. Transformation, Quantization, and entropy coding (Cont.) • A quantization parameter is used for determining the quantization of transform coefficients in H.264/AVC. The parameter can take 52 values. • The quantized transform coefficients of a block generally are scanned in a zig-zag fashion and transmitted using entropy coding methods.

14. Transformation, Quantization, and entropy coding (Cont.) • Before transmission, the generated data of all types are entropy coded. • H.264 supports two different methods of entropy coding • context adaptive variable length coding (CAVLC) • context adaptive binary arithmetic coding (CABAC)

15. Transformation, Quantization, and entropy coding (Cont.) • Their statistical distribution typically shows large values for the low frequency part decreasing to small values later in the scan for the high-frequency part. • CAVLC • The number of nonzero quantized coefficients (N) and the actual size and position of the coefficients are coded separately. • CABAC • The efficiency of entropy coding can be improved further if the Context-Adaptive Binary Arithmetic Coding (CABAC) is used • Allows the assignment of a non-integer number of bits to each symbol of an alphabet

16. H.264 Error Resilient Coding • Spatial Error Propagation. • The use of entropy coding means that every coded bit within a slice requires the previous bits for its decoding. Hence, a single bit error in the transmitted stream may destroy the whole remaining coded bits of one slice • Temporal error propagation • Dependencies of consecutive pictures, as a result of inter-coding. • Damage in one frame may propagate into many future frames, even if their information is received without error.

17. H.264 Error Resilient Coding (Cont.) • First, the coded video data are grouped in network abstraction layer (NAL) units • Each NAL unit can be considered as a packet that contains an integer number of bytes including a header and a payload • The header specifies the NAL unit type and the payload contains the related data. • Data Partitioning

18. H.264 Error Resilient Coding (Cont.) • In a video sequence, each frame can be divided into several slices; each containing a flexible number of MBs. • In each slice, the arithmetic coder is aligned and the spatial predictions are reset. • Every slice in the frame is independently decodable, and therefore can be considered as a re-synchronization point that prevents spatial propagation of a probable error to the next slice.

19. H.264 Error Resilient Coding (Cont.) • Intra-updates • Multiple-reference selection

20. References • [1] Thomas Wiegand, Gary J. Sullivan, Gisle Bjontegaard, and Ajay Luthra “Overview of the H.264 / AVC Video Coding Standard”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, JULY 2003. • [2] Gary J. Sullivan, Pankaj Topiwala, and Ajay Luthra, “The H.264/AVC Advanced Video Coding Standard: Overview and Introduction to the Fidelity Range Extensions”, Presented at the SPIE Conference on Applications of Digital Image Processing XXVII Special Session on Advances in the New Emerging Standard: H.264/AVC, August, 2004.