Overview of the H. 264/AVC video coding standard

1. Overview of the H. 264/AVC video coding standard

2. History and Naming

3. Goal of “The Standard” • Deploy over existing & future networks in these areas with substantially low bitrate: • Broadcast • Interactive or serial storage • Conversational services • Video-on-demand or multimedia streaming services • Multimedia messaging services • Flexibility and Customizability:

4. Comparison: PSNR • H.264 improves the coding efficiency over other coding methods

5. Design Feature Highlights • Enhancements of coding efficiency • Better prediction • Variable block-size motion compensation with small block sizes • Quarter-sample-accurate motion compensation • Multiple reference picture motion compensation • Weighted prediction • Other factors • Hierarchical block transform • Short word-length transform • Enhancements of robustness and flexibility for operation over a variety of networks

6. Structure of H.264 Video Encoder

7. Network Abstraction Layer • Facilitates the ability to map H.264 VCL data to transport layers such as RTP/IP, H.32x, MPEG-2. • NAL Units • VCL and non-VCL units • Parameter sets • Access units • Coded video sequence

8. NAL: Parameter Sets

9. VCL: Revolution or Evolution? • There is no single coding element in the VCL that provides the majority of the significant improvement in compression efficiency in relation to prior video coding standards. • It is rather a plurality of smaller improvements that add up to the significant gain. • The basic source-coding algorithm is a hybrid of inter-picture prediction to exploit temporal statistical dependencies and transform coding of the prediction residual to exploit spatial statistical dependencies.

10. VCL: Elements • Pictures, Frames, Fields • Macroblocks (MBs)(16x16) • Slices, Slices Groups

11. VCL: Coding the Slices • Each slice can be coded using different coding types as follows: • I slice: A slice in which all macroblocks of the slice are coded using intra prediction. • P slice: In addition to the coding types of the I slice, some macroblocks of the P slice can also be coded using inter prediction with at most one motion-compensated prediction signal per prediction block. • B slice: In addition to the coding types available in a P slice, some macroblocks of the B slice can also be coded • SP slice (switching P slice): coded such that efficient switching between different pre-coded pictures becomes possible. • SI slice (switching I slice): allows an exact match of a macroblock in an SP slice for random access and error recovery purposes.

12. Peek into Encoder

13. Adaptive Frame/Field Coding • 3 choices: • To combine the two fields together and to code them as one single coded frame (frame mode). • To not combine the two fields and to code them as separate coded fields (field mode). • To combine the two fields together and compress them as a single frame, but when coding the frame to split the pairs of two vertically adjacent macroblocks into either pairs of two field macroblocks or frame macroblocks before coding them.

14. Adaptive Frame/Field Coding • PAFF (Picture-Adaptive Frame/Field) • The choice between first 2 • >16% better than field only in general • MBAFF (MacroBlock-Adaptive Frame/Field ) • Non-moving: frame mode, moving: field mode • Use a pair of MBs (16x32 luma) • >14% better than PAFF in general

15. Intra-Frame Prediction 3 modes: Intra_4x4, Intra_16x16 (with chroma prediction) and I_PCM Constrained intra coding mode: prediction only from intra-coded neighboring MBs

16. Inter-Frame Prediction • Full sample, half sample, one-quarter sample • Half: apply one-dimensional 6-tap FIR filter • One-quarter: average full and half • Results: • Accurate motion representation • More flexibility in prediction filtering • Different degrees of low pass filtering could be chosen • better preserving hi-frequency content.

17. Transform, Scaling and Quantization • 4x4 Integer transform vs. 8x8 DCT • No mismatch in inverse transform • Less computations, short processing wordlength • Less noise • 52 quantization values; step size: 12% • one step means 12% bit rate change • 6 steps: factor of 2

18. Entropy Coding • CAVLC (Context-Adaptive Variable Length Coding) • VLC tables for various syntax elements are switched depending on already transmitted syntax elements. • Since the VLC tables are designed to match the corresponding conditioned statistics, performance is improved in comparison to schemes using a single VLC table. • CABAC (Context-Adaptive Binary Arithmetic Coding) • allows the assignment of a noninteger number of bits to each symbol of an alphabet • permits adaptation to nonstationary symbol statistics • uses only shifts and table look-ups • Compared to CAVLC, typically provides a reduction in bit rate 5%–15%

19. In-Loop Deblocking Filter • Goal: reduce blockiness and maintain sharpness

20. Profiles and Levels • Profiles: Facilitate interoperability between various applications. • Defines a set of coding tools or algorithms that can be used in generating a conforming bitstream • Incl. Baseline, Main, Extended, High(4) • Levels: place constraints on certain key parameters of the bitstream • Correspond to processing power and memory capability of a codec • Set limts for data bitrate, framesize, pic buffer size. Eg. QCIF, ITU-R601, HDTV etc. • Universal (15) used in different profile

21. Profiles: Coding Features

22. Computation Consumption • Use large amount of computation to achieve high quality, reliability and compression ratio • Intra/inter frame prediction: ~60% • Integer transform: ~10% • Error resilience: ~20% • De-blocking filter: ~10% • Encoding: ~10x of MPEG-2’s CPU usage • For current home PC users, realtime software decoding (HDTV quality) could be achieved, but not encoding • Recommend software: • QuickTime Alternative (www.free-codecs.com) • X264 (www.video-lan.org/x264.html)

23. Licensing • Standard: no free lunch • Lessons from MPEG-4 Visual • Was not friendly at the beginning… • H.264 • Codec manufactures pay for the consumer use by/between end users • Cheap for free television provider • Free internet broadcast till 2010.12.31

24. Applications • HD DVD Video (DVDForum); Blu-ray Disk (BDA) (High Profiles: mandatory) • DVB in Europe, DMB in Korea, ISDB-T in Japan • Direct broadcast satellite TV (multi-region) • 3GPP (optional feature) • IETF: RFC3984 for RTP • ITU-T: H.32x (in production) • VoD