Transcoding from H.264/AVC to HEVC

1. Transcoding from H.264/AVC to HEVC Shantanu Kulkarni UTA ID: 1000789943

2. Introduction to Transcoding • Video transcoding is the operation of converting video from one format to another. • A format is defined by characteristics such as bit-rate, spatial resolution etc. • Transcoding is one of the most promising technologies, which provides video adaptation in terms of bit-rate reduction, resolution reduction and format conversion to meet various requirements

3. Following is the most basic transcoding architecture Fig. 1 Basic architecture for transcoding [8]

4. Need for Transcoding • Design of most video coding standards is primarily aimed at having the highest coding efficiency, which is the ability to encode the video at lowest possible bitrate while maintaining certain level of video quality. • HEVC, which is a recently emerged video coding standard, aims at high coding efficiency while retaining the video quality • With its hybrid coding architecture, motion compensation prediction and transform coding technique, it can be seen as an improved version of the previous standard H.264 [6]

5. Need for Transcoding contd. • Transcoding from H.264 to HEVC will enable lowering the bitrate resulting in a more efficient compression.[1] • AVC and HEVC share a similar prediction, transform, quantization, and entropy coding architecture. [1]

6. Overview of HEVC • The HEVC standard is based on the well-known block-based hybrid coding architecture, combining motion-compensated prediction and transform coding with high-efficiency entropy coding • It employs a flexible quad-tree coding block partitioning structure that enables the efficient use of large and multiple sizes of coding, prediction, and transform blocks. • It also employsimproved intra prediction and coding, adaptive motion parameter prediction and coding, new loop filter and an enhanced version of context-adaptive binary arithmetic coding (CABAC) entropy coding.

7. Overview of HEVC contd. Fig. 2 HEVC encoder block diagram

8. Overview of HEVC contd. Fig. 3 Block based architecture HEVC – Intra-Prediction [17]

9. Overview of H.264/AVC [6] • Directional spatial prediction for intra coding (9 directional prediction modes) • Variable block-size motion compensation with small block size • Quarter-sample-accurate motion compensation • Motion vectors over picture boundaries • Multiple reference picture motion compensation • Decoupling of referencing order from display order • In-the-loop deblocking filtering

10. H.264/AVC Encoder-Decoder Block Diagram Fig. 4 H.264/AVC Encoder [2] Fig. 5 H.264/AVC Decoder [2]

11. Comparison of AVC & HEVC • Larger block structure leading to maximum of 64x64 pixels per block • Intra prediction direction modes which are upto 35 (33 modes + DC + Planar) in case of HEVC while H.264 has 9 directional modes of intra prediction • Adaptive motion vector prediction, which allows codec to find more inter frame redundancies • Superior parallelization tools, including wavefront parallel processing, for more efficient coding in a multi core environment • Entropy using CABAC only, no more CAVLC • Improvements to de-blocking filter and addition of one more filter called Sample Adaptive Offset (SAO) that further leaves artifacts along block edges

12. Output HEVC Bitstream Input AVC Bitstream AVC Decoder HEVC Re-encoder Residual, modes and MVs CU, PU partitions and MVs Simplified Mode Selection HEVC Transcoder • The transcoding schemes discussed here avoid high computational complexity in terms of reduced RDO evaluations and motion compensation operation as well as fractional pixel interpolation operation. Fig. 6 Pixel domain AVC-HEVC transcoder[1]

13. HEVC Transcoder Contd. • The LCU will initially split according to the input MB modes in AVC • The initial CU partitions will be further merged to larger size according to the predict directions of its adjacent four sub-CUs. • For example, if the predict directions of adjacent four 8x8 CUs are the same, they will be merged to 16x16. Similar merge operations will also perform on CUs larger than 8x8. The merge process is applied from the 4x4 smallest 4x4 blocks to the blocks with size 32x32. • The input information from AVC can be used to reduce the candidate predict directions for SATD, or reducing the candidate SATD list

14. HEVC Transcoder contd. • The major complexity of Inter picture coding comes from the motion estimation (ME), MC operations when testing every set of possible coding parameters with possible CU size, PU and TU modes. • Thus, it is proposed to reduce these operations with the help of input • AVC information, e.g. residuals, modes and MVs.Since the largest CU (LCU) in HEVC consists of 16 MBs in AVC, after AVC decoding, the information of these MBs will transmit to the mode selection module.

15. HEVC Transcoder Contd. • Since the input AVC bitstream already contains useful information of the MB partitions and prediction directions, we extract the information which can be utilized by HEVC encoder without having to do any computations. • Key technology of AVC to HEVC transcoding consists of merging smaller blocks to a larger CU, especially for bit rate reduction transcoding. Since a large CU may consists of different 4x4 blocks, and probably, these blocks may have different MVs, merging these blocks now turns to measure the RD cost when the MV changes.

16. Cascaded encoder decoder transcoder architecture • Includes complete decoding and re-encoding • High Complexity • Error due to the lossy encoding of already decoded bit sequence Cascaded Decoder and Encoder Transcoded HEVC Bit stream Input Bit stream H.264 Bit stream Output Bit stream H.264 Encoder H.264 Decoder HEVC Encoder HEVC Decoder Reconstructed Bit stream Fig.7 Cascaded encoder – decoder transcoder

17. Simulation results Table 1. MSE and PSNR of akiyo_qcif.yuv video sequence for 100 frames

18. Simulation results continued… Table 2. MSE and PSNR of foreman_qcif.yuv video sequence for 100 frames

19. Simulation results continued… Table 3. MSE and PSNR of mobile_cif.yuv video sequence for 100 frames

20. Simulation results continued… Table 4. MSE and PSNR of coastguard_cif.yuv video sequence for 100 frames

21. Comparison of PSNR for colored images between ‘qcif’ video sequences akiyo and foreman PSNR (db)  PSNR (dB) Video Sequences  Video Sequences  Fig 8. PSNR Video sequences: akiyo_qcif and foreman_qcif

22. Comparison of PSNR for colored images between ‘cif’ video sequences mobile and coastguard PSNR (dB) Video Sequences  Fig 9. PSNR Video sequences: mobile_cif and coastguard_cif

23. Comparison of bitrate for akiyo_qcif video sequence Bitrate (kbps) Video Sequences  Fig 10. Bitrate comparison between H.264 encoded, HEVC encoded and transcoded output using foreman_qcif.yuv sequence. (100 frames)

24. Comparison of bitrate for foreman_qcif video sequence Bitrate (kbps) Video Sequences  Fig 10. Bitrate comparison between H.264 encoded, HEVC encoded and transcoded output using foreman_qcif.yuv sequence. (100 frames)

25. Comparison of bitrate for mobile_cif video sequence Bitrate (kbps) Video Sequences  Fig 11. Bitrate comparison between H.264 encoded, HEVC encoded and transcoded output using mobile_cif.yuv sequence. (100 frames)

26. Comparison of bitrate for coastguard_cif video sequence Bitrate (kbps) Video Sequences  Fig 12. Bitrate comparison between H.264 encoded, HEVC encoded and transcoded output using coastguard_cif.yuv sequence. (100 frames)

27. Comparison of computation time for akiyo_qcif video sequence Time (sec) Video Sequences  Fig 13. Computation time comparison between H.264 encoded, HEVC encoded and transcoded output using akiyo_qcif.yuv sequence. (100 frames)

28. Comparison of computation time for foreman_qcif video sequence Time (sec) Video Sequences  Fig 14. Computation time comparison between H.264 encoded, HEVC encoded and transcoded output using foreman_qcif.yuv sequence. (100 frames)

29. Comparison of computation time for mobile_cif video sequence Time (sec) Video Sequences  Fig 15. Computation time comparison between H.264 encoded, HEVC encoded and transcoded output using mobile_cif.yuv sequence. (100 frames)

30. Comparison of computation time for coastguard_cif video sequence Time (sec) Video Sequences  Fig 16. Computation time comparison between H.264 encoded, HEVC encoded and transcoded output using coastguard_cif.yuv sequence. (100 frames)

31. Images Akiyo_qcif and Foreman_qcif Fig 17a. Fig 17b. Fig 17c. Fig 17. Akiyo_qcif video sequence 17a) H.264 encoded, 17b) transcoded and 17c) HEVC encoded and reconstructed Fig 18a. Fig 18b. Fig 18c. Fig 18. Foreman_qcif video sequence 18a) H.264 encoded, 18b) transcoded and 18c) HEVC encoded and reconstructed

32. Images Mobile_cif Fig 19a. Mobile_cif H.264 encoded and reconstructed Fig 19b. Mobile_cif Transcoded Fig 19c. HEVC Encoded and reconstructed

33. Images coastguard_cif Fig 20a. Coastguard_cif H.264 encoded and reconstructed Fig 20b. Coastguard_cif Transcoded Fig 20c. Coastguard_cif HEVC Encoded and reconstructed

34. References • D. Zhang, B. Li, J. Xu, and H. Li, ‘Fast Transcoding from H.264/AVC to High Efficiency Video Coding’ IEEE International Conference on Multimedia Expo, pp. 651-656, July, 2012 • T. Wiegand et al, “Overview of the H.264/AVC video coding standard,” IEEE Trans. CSVT, Vol. 13, pp. 560-576, July 2003.J Xin, C.W. Lin and M.T. Sun, “Digital video transcoding”, Proceedings of the IEEE, Vol. 93, pp 84-97, Jan 2005. • T. Wiegand, G. J. Sullivan, G. Bjøntegaard, and A. Luthra, ‘Overview of the H.264/AVC Video Coding Standard’, IEEE transactions on circuits and systems for video technology, vol. 13, no. 7, pp. 560-576, July 2003 • I. Kim, J. Min, T. Lee et al, ‘Block Partitioning Structure in the HEVC Standard’, IEEE transactions on circuits and systems for video technology, vol. 22, no. 12, pp. 1697-1706, December 2012 • G. Sullivan, P. Topiwalla and A. Luthra, “The H.264/AVC video coding standard: overview and introduction to the fidelity range extensions”, SPIE Conference on Applications of Digital Image Processing XXVII, vol. 5558, pp. 53-74 Aug 2004. • T. D. Nguyen et al, “Efficient MPEG-4 to H.264/AVC transcoding with spatial downscaling”, ETRI Journal, vol.29, no.6, pp 826-828, Dec. 2007. • G.J. Sullivan, J. Ohm, W. Han et al, “Overview of High Efficiency Video Coding (HEVC) Standard” IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No.12, Dec 2012 • A. Vetros, C. Christopoulos and H. Sun, “Video transcoding architectures and techniques: An overview”, IEEE Signal Processing Magazine, Vol. 20, pp 18-29, March 2003 • HEVC open source software (encoder/decoder) https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-6.0 • JM Reference Software - http://iphome.hhi.de/suehring/tml/ - H.264 reference software • Eduardo Peixoto Fernandes da Silva, “Advanced Heterogeneous Video Transcoding” Queen Mary, University of London, PhD Thesis. • J. Padia, “Complexity Reduction For Vp6 To H.264 Transcoder Using Motion Vector Reuse”, MPL, University of Texas at Arlington, May 2012.

35. References Contd. Reference Books • K. Sayood, “Introduction to Data compression”, III edition, Morgan Kauffmann publishers, 2006. • I.E.G. Richardson, “H.264 and MPEG-4 video compression: video coding for next-generation multimedia”, Edition II ,Wiley, 2010. Websites • http://en.wikipedia.org/wiki/ : Website for Wikipedia, Encyclopedia • http://www-ee.uta.edu/Dip/Courses/EE5359/index.html: Course website • http://ieeexplore.ieee.org/: Website archive for IEEE papers online • http://www.v-net.tv/hevc-is-game-changer-for-multi-screen-and-iptv/: Impact of HEVC standard on digital media market like cell phones, TVs etc • http://www.streamingmedia.com/Articles/Editorial/What-Is-.../What-Is-HEVC-(H.265)-87765.aspx: Summary about HEVC , information site. • http://mrutyunjayahiremath.blogspot.com/2010/09/h264-video-codec_22.html: Diagram for H.264 prediction direction modes • http://opticalengineering.spiedigitallibrary.org/article.aspx?articleid=1352660: Diagram for Intra prediction block structure for HEVC

36. THANK YOU