Selective Disparity Estimation And Variable Size Motion Estimation Based on Motion Homogeneity for Multi-View Coding LiquanShen, Zhi Liu, Suxing Liu, Zhaoyang Zhang, and Ping An IEEE Transactions on Broadcasting Dec. 2009
Outline Introduction Observations and analysis Proposed algorithm Experimental results
Introduction (1/2) motion estimation disparity estimation Time = t-1 View-1 Time = t View-0 Coding structure proposed by HHI:
Introduction (2/2)Global Disparity Vector (GDV) Time Anchor frame Non-anchor frame Anchor frame …… Ref. view (view 0) …… View GDVcur GDVbehind GDVahead …… ……
Observations (1/2) • T: Temporal prediction • Static BG, homogeneous region • Small block size mode for complex motion • V: View prediction • Complex motion
Observations (2/2) Block size distribution: Only the MBs in the region with complex motion need DE and small mode size ME.
Goal • Try to decide in advance: • the optimal prediction direction (ME/DE) for MBs • the prediction size is 16×16 or not
Motion homogeneity determined (1/4) A uniform motion vector field at 4×4 block level is generated. MBm,n: a MB located at the mth row, nth column. : the MVs of its convered 4×4 blocks.
Motion homogeneity determined (2/4) 4 4 4 4 Current MB Neighbor MBs used in calculating the motion homogeneity:
Motion homogeneity determined (3/4) The motion homogeneities of MBm,n in horizontal and vertical directions are defined as: The motion homogeneities of MBm,n is defined as:
Motion homogeneity determined (4/4) • IfMD(m,n) <T then the MB is considered with homogeneous motion. Otherwise, the MB is considered with complex motion. • The threshold T is fixed for each QP level and different sequences, which is set to 0.1.
Selective disparity estimation MB with homogeneous motion is likely to choose temporal prediction. If a MB satisfies the criterion of spatially homogenous motion, inter-view prediction can be skipped.
Selective variable size motion estimation When a MB is with homogeneous motion, the best mode size of the MB has a very large probability to be 16×16.
Proposed fast DE/ME algorithm Derive MV from left, above, left-above MB, and the corresponding MB in the previously coded view. Compute the motion homogeneity for current MB. If a MB is a homogeneous motion, perform 16x16 ME, and go to step 6, otherwise, go to step 4. Perform variable size DE and ME. Perform intra 4x4 prediction. Perform intra 16x16 prediction. Determine the best prediction direction and prediction mode. Go to step 1 and proceed with next MB.
Experimental results (1/4)Experimental environment • JMVM 6.0 • Test sequences (total of 9): • Downflamence2, Flamencol, Golf1, Golf2, Race1, Exit, Ballroom, Jungle, Uli • Full temporal prediction modes and inter-view prediction (FMD) • 3 views are coded • QP: 20, 24, 28, 32 • CABAC, loop filter are enabled
Experimental results (2/4) Comparison between the proposed method and FMD in JMVM:
Experimental results (4/4)  X. Li, D. Zhao, X. Ji, Q. Wang, and W. Gao, “A fast inter frame prediction algorithm for multi-view video coding,” in ICIP, 2007. Compares with other method:
Fast Mode Decision Using Global Disparity Vector for Multiview Video coding Dong-Hoon Han, Yung-Lyul Lee 2008 Second International Conference on Future Generation Communication and Networking Symposia
Outline Goal Proposed algorithm Experimental results
Goal Using both MB-based region segmentation information and global disparity vector (GDV) among view to reduce encoding time. Fast mode decision using GDV.
Region partition (1/2) The proposed segmentation of the background and objects block modes for fast mode decision in inter-view prediction: An MB is decided as background block mode if a derive motion vector is smaller than ¼ in integer pixel unit in case of Direct mode , Inter 16x16, P_SKIP or B_SKIP mode.
Region partition (2/2) Black block: object region White block: background region
Fast mode decision forinter-view prediction Region segmentation information of base-view Region segmentation information of non-base view using GDV and (a) Regions of the vies using inter-view prediction are estimated using MB-based GDV and region segmentation map of reference view.