An Early Block Type Decision Method for Intra Prediction in H.264/AVC

An Early Block Type Decision Method for Intra Prediction in H.264/AVC Jungho Do, Sangkwon Na and Chong-Min Kyung VLSI Systems Lab. Korea Advanced Institute of Science and Technology (KAIST)

Table of contents • Introduction • Intra prediction • Early block type decision • Block type decision point • Rate-distortion cost • Experimental results • Conclusion • Reference SiPS 2009

Introduction • Two prediction methods in H.264/AVC • Inter prediction • Using temporal correlation between the earlier frame and the current frame • Intra prediction • Using spatial correlation between neighboring blocks in the current frame • 9 prediction modes SiPS 2009

Introduction • Why do we focus on Intra prediction (IP)? • Rate-distortion optimization (RDO) in Intra prediction also improves an overall rate-distortion performance. • More block types and prediction modes inIntra prediction than inter prediction • Constraint: 560 cyclesfor 1920x1080@30fps (133MHz) • Total: 640 cycles SiPS 2009

Introduction • Why do we focus on Intra prediction (IP)? • Rate-distortion optimization (RDO) in Intra prediction also improves an overall rate-distortion performance. • More block types and prediction modes inIntra prediction than inter prediction • Constraint: 560 cyclesfor 1920x1080@30fps (133MHz) • Total: 640 cycles Pipeline for one prediction mode IP DCT Q Q-1 IDCT Recon. IP DCT Q Q-1 IDCT Recon. IP DCT Q Q-1 IDCT Recon. SiPS 2009

Introduction • Rate-distortion cost function for RDO [9] M.G. Sarwer and L.M. Po, “Bit rate estimation for cost function of 4x4 intra mode decision of h.264/avc,” ICME, 2007, pp. 1579 – 1582. SiPS 2009

Intra prediction • Nine prediction modes • Associated with the direction of prediction SiPS 2009

Earlyblock type decision • Motivation • Spatial correlation between sub-blocks and macroblock (MB) • The prediction mode of neighboring blocks have the similar direction. • We assume thatR-D cost of neighboring blocks also have the similar cost based on the above hypothesis. • R-D cost of macroblock can be substituted by R-D cost of sub-blocks for mode decision. SiPS 2009

Encoding order • Conventional encoding order • 8x8 blocks are performed after all 4x4 blocks are performed. 4x4 4x4 4x4 4x4 4x4 4x4 4x4 4x4 4x4 4x4 4x4 4x4 4x4 4x4 4x4 4x4 8x8 8x8 8x8 8x8 16 16x16 C8x8 SiPS 2009

Encoding order • Proposed encoding order • To use spatial correlation, the prediction of 8x8 block is performed after the predictions of the corresponding four 4x4 blocks are finished. • Three possible decision points SiPS 2009

1st block type culling • 16x16 block type • Probability of 16x16 block to be decided as the best mode when 4x4 block is selected by proposed early block type decision • 16x16 block type is bypassed when 4x4 block type is selected. SiPS 2009 11

2nd block type culling • Block type decision at 1/4MB point • If JRD,4x4(1/4) < JRD,8x8(1/4) • Prediction of only 4x4 block type is performed evaluated after 1/4MB. • Otherwise • 8x8 block type is taken. 16x16 block 4x4 8x8 SiPS 2009

2nd block type culling • Block type decision at 2/4MB and 3/4MB 4x4 8x8 4x4 8x8 SiPS 2009

Block type decision point • Which block type decision point is the best? • In terms of R-D performance and the number of reconstruction loops according to the positions of early block type decision method. SiPS 2009

Block type decision point • Which block type decision point is the best? • In terms of R-D performance and the number of reconstruction loops according to the positions of early block type decision method. Constraint SiPS 2009

Block type decision point • Proposed early block type decision • 2/4 MB point is effective in terms of distortion, bit-rate and computational complexity (the number of cycles for MB process). Constraint SiPS 2009

R-D cost function • Original cost function SiPS 2009

R-D cost function • Proposed R-D cost for sub-block • Assumption: the best mode of chroma block is decided independently regardless of luma mode. The best mode of luma block is decided independently. The difference of RCBP between 4x4 block type and 8x8 block type is frequently very small. The value of RMBis fixed as ‘intra’ for all block types. SiPS 2009

RCBP • Coded_block_pattern (CBP) variable 8 Chroma Luma 000000 Coded_block_patternindicates which 8x8 blocks(6) in the MB contain nonzero coefficients SiPS 2009

RCBP • Comparison of RCBP SiPS 2009

R-D cost function • Simplified R-D cost function • Proposed R-D cost function enables evaluation of Rheader to be performed at the early decision point (2/4MB) because RHeader can be derived from RLmode. SiPS 2009

Experimental results • R-D performance comparison Full search Proposed SiPS 2009

Conclusion • Computational complexity problem in RDO-enabled Intra prediction • An early decision at 2/4MB point for RDO-enabled Intra prediction • Block type decision based on spatial correlation • R-D cost computation is reduced by 90.1% with 0.93% bit-rate increase and 0.039 dB PSNR decrease compared to full search. SiPS 2009

Thank you SiPS 2009

Reference • [1] F. Pan, X. Lin, S. Rahardja, K.P. Lim, and Z.G. Li, “A directional field based fast intra mode decision algorithm for h.264 video coding,” ICME, 2004, vol. II, pp. 1147–1150. • [2] A.C. Tsai, J.F. Yang, andW.G. Lin, “Effective subblockbased and pixel-based fast direction detections for h.264 intra prediction,” Circuits and Systems for Video Technology, IEEE Transactions on, vol. 18, pp. 975–982, July 2008. • [3] Y.-K. Lin and T.-S. Chang, “Fast block type decision algorithm for intra prediction in h.264 frext,” ICIP, 2005, vol. I, pp. 585–588. • [4] T. Zhang, G. Tian, and S. Goto, “A frequency-based fast block type decision algorithm for intra prediction in h.264/avc high profile,” IAPCCAS, 2008, vol. II, pp. 1292–1295. • [5] W. Lee, Y. Jung, S. Lee, and J. Kim, “High speed intra prediction scheme for h.264/avc,” Consumer Electronics, IEEE Transactions on, vol. 53, pp. 1577–1582, Nov 2007. • [6] G. Jin, J.-S. Jung, and H.-J. Lee, “An efficient pipelined architecture for h.264/avc intra frame processing,” ISCAS, 2007, vol. II, pp. 1605–1608. • [7] Y.-K. Lin, C.-W. Ku, D.-W. Li, and T.-S. Chang, “A 140-mhz 94k gates hd1080p 30-frames/s intra-only profile h.264 encoder,” Circuits and Systems for Video Technology, IEEE Transactions on, vol. 19, pp. 432–436, Mar 2009. • [8] C.K. Huang and L.L. Youn, “An h.264/avc full-mode intra-frame encoder for 1080hd video,” ICME, 2008, pp. 1037–1040. • [9] M.G. Sarwer and L.M. Po, “Bit rate estimation for cost function of 4y4 intra mode decision of h.264/avc,” ICME, 2007, pp. 1579 – 1582. • [10] I.E. Richardson, “Draft itu-t recommendation and final draft international standard of joint video specification(itu-t rec.h.264 -iso/iec14496-10 avc),” Joint Video Team, 2003. • [11] Z. Kun, Y. Chun, L. Qiang, and Z. Yuzhou, “A fast block type decision method for h.264/avc intra prediction,” ICACT, 2007, vol. I, pp. 673–676. • [12] “Joint video team (jvt) reference software version 14.0,” http://iphome.hhi.de/suehring/tml/download/. • [13] I.E. Richardson, “H.264 and mpeg-4 video compression: video coding for next generation multimedia,” Chichester, U.K.: Wiley, 2003. SiPS 2009

Appendix SiPS 2009

Introduction • H.264/AVC coding flow Input Video Signal Split into Macroblocks 16x16 pixels Coder Control Control Data Transform/Scal./Quant. - Quant.Transf. coeffs Decoder Scaling & Inv. Transform Entropy Coding De-blocking Filter Intra-frame Prediction Output Video Signal Motion- Compensation Intra/Inter Motion Data Motion Estimation SiPS 2009

Introduction • Why we focus on Intra prediction • Computational bottleneck in rate-distortion (R-D) optimized encoder due to many block types and prediction modes • Constraint: 560 cyclesfor 1920x1080@30fps (133MHz) • Total: 640 cycles(= 15*16 + 60*4 + 160) • One 4x4 block: 15 cycles • One 8x8 block: 60 cycles • One 16x16 block: 160 cycles SiPS 2009

Early block type decision • Flow SiPS 2009

An Early Block Type Decision Method for Intra Prediction in H.264/AVC