1 / 19

An H.264-based Scheme for 2D to 3D Video Conversion

An H.264-based Scheme for 2D to 3D Video Conversion. Mahsa T. Pourazad Panos Nasiopoulos Rabab K. Ward. IEEE Transactions on Consumer Electronics 2009. Outline. Introduction to 3D television 2D-to-3D Conversion Scheme Camera motion Correction Correction of Displacement Estimates

nerys
Télécharger la présentation

An H.264-based Scheme for 2D to 3D Video Conversion

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An H.264-based Scheme for 2D to 3D Video Conversion Mahsa T. Pourazad PanosNasiopoulos Rabab K. Ward IEEE Transactions on Consumer Electronics 2009

  2. Outline • Introductionto 3D television • 2D-to-3D Conversion Scheme • Camera motion Correction • Correction of Displacement Estimates • Perceptual Depth Enhancement • Performance Evaluation • Conclusion

  3. Introduction • 3D television • Stereoscopic • Multi-view • 2D plus depth • 3D display

  4. Introduction • 2D to 3D video streams • 2D video stream + Depth map • Depth Image Based Rendering(DIBR) [1] • 2 different viewpoints (projected on left and right retinas) [1] L. Zhang, “Stereoscopic image generation based on depth images for 3D TV,” IEEE Trans. Broadcasting, vol. 51, no.2, pp191-199, 2005.

  5. Introduction • Depth map estimation • Light, shade, relative size, motion parallax, partial occlusion, textural gradient, geometric perspective…… • Manual, semi automatic or automatic • Machine learning • Extract depth from blur • Edge information • Motion vector information • H.264/AVC standard • Can’t work on static objects

  6. 2D-to-3D Conversion Scheme • Use abs(MVx) for estimating the depth map • Depth of point P can be easily obtained if the disparity d is known.

  7. 2D-to-3D Conversion Scheme • H.264/AVC Motion vector estimation • variable block sizes • Quarter-pixel matching accuracy • Correction • Moving camera • Object boundary • Perceptual depth enhancement

  8. Camera Motion Correction • Camera panning • Recognize camera motion • Adjust “Skip Mode” • Adjust net motion • Zoom in/out • Check the tendency of the camera • MVs are scaled accordingly [2] [2] D. Kim, D. Min, K. Sohn, “Stereoscopic video generation method using motion analysis,” 3DTV Conf. pp. 1-4, 2007.

  9. Correction of Displacement Estimates • Is this motion vector correct? • Readjust MVs by making it equal to the median MV Motion vector is very different from neighbors’ ? Yes Check the variance of the corresponding block in residual frame Object boundary pixels? No MV=median of neighbors’ MV

  10. Perceptual Depth Enhancement • Non-linear scaling model • The further the object is, the smaller the scaling factor. • The enhanced disparity value (N uniformly spaced depth layer) Ex: Layer 0(i=0, S(0)=Zfar/Znear) Layer N-1(i=N-1, S(N-1)=1)

  11. Performance Evaluation • Video sequences • “Interview”, “Orbi” • True Depth Maps • Captured by 3D-depth range camera (Zcam) • 0 to 255 (256 depth layers) • JM12.2 version of the H.264/AVC standard • Compare with [3] [3] I. Ideses, L. P. Yaroslavsky, and B. Fishbain, “Real-time 2D to 3D video conversion,” Journal of Real-Time Image Processing, vol. 2, no. 1, pp. 3-9, 2007.

  12. Performance Evaluation Orbi Interview Video sequence Recorded depth map

  13. Performance Evaluation Estimated depth map by [3] Estimated depth map by our approach

  14. Performance Evaluation • 15 people graded the videos from 1 to 10 of 3D perception and visual quality

  15. Performance Evaluation

  16. Performance Evaluation

  17. Performance Evaluation

  18. Performance Evaluation • Badly matched pixelsin the estimated depth (Th=1) Percentage of correctly matched pixels

  19. Conclusion • This paper present a efficient method that estimates the depth map of a 2D video sequence using its H.264/AVC estimated motion information. • It can be implemented in real-time at the receiver-end, without increasing the transmission bandwidth requirement.

More Related