1 / 27

Methods of Video Object Segmentation in Compressed Domain

Methods of Video Object Segmentation in Compressed Domain. Cheng Quan Jia. Presentation Outline. Features for Segmentation in Compressed Domain Using Motion Vectors in Segmentation Confidence Measure Conclusion Q & A. Features for Segmentation in Compressed Domain.

osborn
Télécharger la présentation

Methods of Video Object Segmentation in Compressed Domain

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Methods of Video Object Segmentation in Compressed Domain Cheng Quan Jia

  2. Presentation Outline • Features for Segmentation in Compressed Domain • Using Motion Vectors in Segmentation • Confidence Measure • Conclusion • Q & A

  3. Features for Segmentation in Compressed Domain An introduction to Compressed Domain

  4. Compressed Domain: Definition • Compressed Domain refers to video compression techniques that expliots Spatial and Temporal Redundancy using • DCT & Quantization • Motion Compensation • Examples include MPEG-1/-2/-4, H.261 and H.263

  5. Compressed Domain: Definition • Opreations in the Compressed Domain involves processing of • DCT coefficients (from I-macroblocks) • Motion Vectors (from P-/B-macroblocks)

  6. Compressed Domain: Parsing • Unlike pixel domain, operations in the compressed domain do not require the input bitstream to be decoded • Instead, they are Parsed

  7. Compressed Domain: Parsing

  8. Features for Segmentation • After Parsing, we have • DCT coefficients (from I-macroblocks) • Motion Vectors (from P-/B-macroblocks) • Which coresspond to • Frequencies of texture change • Motion of the macroblock

  9. Using Motion Vectors in Segmentation

  10. Acquiring Dense Motion Field • Many video object segmentation methods attempt to acquire a dense smooth motion field in order to create object masks • For this end spatial interpolation and motion accumulation are employed

  11. Motion Accumulation

  12. Motion Accumulation • Due to the different magnitude and signs of motion vectors, the obtained MVs are normalized, e.g. MVs in B-macroblocks would have their signs reversed • Filtering is applied to remove non-uniform MV and smooth the motion field

  13. Motion Accumulation • Chen and Bajic [chen2009] employs MV Integration block-wise and pixel-wise to enhance the Motion Field

  14. Motion Accumulation Chen and Bajic [chen2009] Babu et al. [babu2004]

  15. Porikli et al.’s Investigation • The Compression Domain segmentation system published by Porikli et al. [porikli2010] experimented the effect of DCT coefficients and MV on segmentation performance • The DC parameters(for Y, U, V channels) of the I-frame • Low vertical and horizontal frequency AC values • A spatial energy term • Aggregated motion flow of the corresponding macroblock

  16. Porikli et al.’s Investigation • They create a Frequency-temporal data structure for each macroblock with the features and perform volume segmentation • Their results show that using DCT terms in FT segmentation and using MV in the hierarchical clustering, on average, gives better results.

  17. Porikli et al.’s Investigation

  18. Porikli et al.’s Investigation • The Block Matching Process in encoding stage looks for only the best match for a macroblock rather than object motion

  19. Confidence Measure of Motion Vectors

  20. Approximating Optical Flow • Coimbra and Davies [coimbras2005] try to approximate Lucas–Kanade optical flow in MPEG-2 Compressed Domain

  21. Confidence Measure • They argue that AC[1] and AC[8] in an I-macroblock can be used as confidence measure • The confidence update step will have a 8×8 macroblock referencing a 16×16 image block in the I-frame, and the confidence of the motion vector of the macroblock is the weighted average of confidence in the 16×16 window

  22. Confidence Measure Original image MPEG-2 smooth motion field after confidence threshold

  23. Conclusion

  24. Conclusion • Due to block matching process, motion vectors in P-/B- frames do not necessary relate to object motion • To ensure a motion vector is correlated to object motion, some sort of confidence measure is required • [coimbras2005] demonstrated that edge strength can be an effective measure

  25. Conclusion • Problems not discussed here • Camera motion • Changes in illumination • Occlusions

  26. References • R. V. Babu, K. R. Ramakrishnan, and S. H. Srinivasan. Video Object Segmentation: A Compressed Domain Approach. IEEE Transactions on Circuits and Systems for Video Technology, 14(4):462–473, April 2004. • Y.-M. Chen and I. V. Bajic. Compressed-Domain Moving Region Segmentation with Pixel Precision using Motion Integration. In IEEE Pacific Rim Conference on Computers and Signal Processing, 2009, pages 442 – 447, August 2009. • M. T. Coimbra and M. Davies. Approximating Optical Flow Within the MPEG-2 Compressed Domain. IEEE Transactions on Circuits and Systems for Video Technology, 15(1):103–107, January 2005. • F. Porikli, F. Bashir, and H. Sun. Compressed Domain Video Object Segmentation. IEEE Transactions on Circuits and Systems for Video Technology, 20(1):2–14, January 2010.

  27. Comments and Suggestions Q & A Section

More Related