Overview of Error Resiliency Schemes in H.264/AVC Standard

1. Overview of Error Resiliency Schemes in H.264/AVC Standard Sunil Kumar, Liyang Xu, Mrinal K. Mandal, and Sethuraman Panchanathan Elsevier Journal of Visual Communication and Image Representation, to appear in2005. S. Wenger, “H.264/AVC over IP,”IEEE Trans. Cir. Syst. Video Technol., vol. 13, pp. 645–656, July 2003

2. Outline • Introduction • Error-Resilience Tools in H.264/AVC • Experimental Results • References

3. Introduction • H.264/AVC Video Coding Standard • http://vc.cs.nthu.edu.tw/home/paper/codfiles/cycho/200310132110/Overview_of_the_H.264_AVC_Video_Coding_Standard.ppt

4. Introduction • Error-resilience tools in older video compression standard • Picture segmentation (slices, GOBs); • Placement of Intra MBs, Intra slices, and intra pictures; • Reference picture selection (with and without feedback); • Data partitioning MPEG 1 part 2 MPEG 2 part 2 H.261 H.263 H.263

5. p2 p4 p6 p8 I0 p9 p1 p3 p5 p7 Error-Resilience Tools in H.264/AVC • Reference picture selection • Feedback-based • Without feedback • e.g. Video redundancy coding • Picture segmentation • Slice • Flexible macroblock ordering (FMO) • 7 modes • Overhead bits highly depends on the picture format, the content, and the QP. (S. Wenger and M. Horowitz) • < 5% penalty at QP = 16; on average 20% at QP = 28. Encoder Decoder feedback channel Channel 1 Channel 2 Checker board mode Interleaving mode

6. P1, n-3 P1, n-2 P1, n-1 S1, n P1, n+1 S 21, n SI1, n Error-Resilience Tools in H.264/AVC • SP/SI frame http://vc.cs.nthu.edu.tw/home/paper/codfiles/cycho/200402181718/The_SP-_and_SI-Frames_Design_for_H.264_AVC.ppt • Instantaneous refreshing • Gradual refreshing

7. Intra inter Intra Intra Error-Resilience Tools in H.264/AVC • Intra placement • Used to combat drifting effects. • ConstrainedIntraPrediction flag • An Intra block can not be predicted from an Inter block. • IDR slice and IDR picture • An IDR picture contains only IDR slices. • An IDR slice can only be part of an IDR picture. • An IDR picture invalidates all short-term reference memory buffers. I P IDR P

8. Error-Resilience Tools in H.264/AVC • Data partitioning • For unequal error protection (UEP). • Three different partition types in H.264. • Type A • Header information (MB types, QP, MVs) • Type B • Intra coded block patterns (CBPs) and Intra coefficients • Type C • Inter CBPs and Inter coefficients • Importance • A > B > C

9. Error-Resilience Tools in H.264/AVC • Redundant slices (RSs) • For a MB, an encoder can place redundant representation of the same MBs into the same it stream. • e.g. • One slice is coded using different QP. • If the slice of low QP is available, the decoder discards the RS; otherwise, the RS is reconstructed by the decoder Slice AQP1 Decoder Slice AQP2

10. Error-Resilience Tools in H.264/AVC • Parameter sets • Including picture size, entropy coding method, MV resolution, and so on. • Sequence parameter set • Containing all information related to the picture sequence between two IDR pictures. • Picture parameter set • Containing all information related to all slices in a picture. • e.g. Sending multiple copies of SPSs so to enhance the arrival rate. • e.g. SPSs can be sent out-of-band.

11. Error-Resilience Tools in H.264/AVC • Intra block refreshing by RD control • Better than periodically inserting intra MBs.

12. Experimental Results Stephan Wenger

13. Experimental Results • Error concealment scheme • Intra • Inter

14. paris foreman Stephan Wenger

15. Experimental Results • Experiment 1: One picture, one packet, without any error-resilience • Packet loss rates > 3% unprotected H.264 video becomes unusable. Paris Foreman Stephan Wenger

16. PLR = 0% PSNR = 35 Experimental Results • Experiment 2: One picture, one packet, with intra macroblock refresh • Very good PSNR results • Very high amount of Intra MBs of significant size • A coarser quantizer is needed for CBR – less spatial detail. Stephan Wenger Foreman

17. Experimental Results • Experiment 3: 2-4 slices in one frame • Unfavorable PSNR results for Foreman. • 2nd best PSNR results for Paris. ? Packet header overhead Paris Foreman Stephan Wenger

18. Experimental Results • Experiment 4: Slices interleaving • Better than Exp. 3 for Foreman. • Worse than Exp. 3 for Paris. Paris Foreman Stephan Wenger

19. Experimental Results • Experiment 5: Data partitioning • Partition A is sent twice (3% PLR) or 3 times (>3% PLR). • Degraded PSNR due to CBR. Paris Foreman Stephan Wenger

20. Experimental Results • Experiment 6: FMO • Randomly inserting Intra MB. (3% of PLR) • Best PSNR results Paris Foreman Stephan Wenger

21. Experimental Results QCIF, 10fps,100Kbps M. M. Ghandi and M. Ghanbari

22. Experimental Results QCIF, 10fps,100Kbps M. M. Ghandi and M. Ghanbari

23. References • S. Kumar, L. Xu, M. K. Mandal, and S. Panchanathan, “Overview of Error Resiliency Schemes in H.264/AVC Standard,”Elsevier Journal of Visual Communication and Image Representation (Special issue on H.264/AVC Video Coding Standard), to appear in 2005. • S. Wenger, “H.264/AVC over IP,”IEEE Trans. Cir. Syst. Video Technol., vol. 13, pp. 645–656, July 2003. • S. Wenger and M. Horowitz, “Scattered slices: A new error resilience tool for H.26L,” in JVT-B027, Feb 2002. • M. M. Ghandi and M. Ghanbari, “Layered H.264 video transmission with hierarchical QAM,”Elsevier J. of Visual Communication and Image Representation (Special issue of H.264/AVC), to appear in 2005.