Symmetric Disparity Estimation in Distributed Coding of Stereo Images

1. Symmetric Disparity Estimation in Distributed Coding of Stereo Images Xin Li Lane Dept. of CSEE West Virginia University Morgantown, WV 26506-6109

2. Outline • Background: distributed source coding (DSC) • The gap between theory and practice: modeling nonstationary and location-related correlation structure • Distributed coding of stereo images • Asymmetric vs. symmetric approach • Side information (SI) generation: EM-like iterative disparity estimation • Coded information (CI) exploitation: jointly refine intensity and disparity uncertainty • Experimental results and summary

3. What is Distributed Coding? Traditional (centralized) Coding Paradigm (e.g., TV broadcasting) New (distributed) Coding Paradigm (e.g., sensor network) source Y source X source Y source X centralized encoder distributed encoder X distributed encoder Y Power-hungry component centralized decoder centralized decoder

4. Current State of the Art • Theoretic sources (e.g., i.i.d. binary or Gaussian) • Trellis codes: Pradhan and Ramchandran’1999, Wang and Orchard’2001 • Turbo codes: J. Garcia-Frias’2001, Aaron and Girod’2002 • LDPC codes: Stankovic, Liveris and Xiong’2002 • Practical sources (e.g., image and video) • Wyner-Ziv coding of video still significantly falls behind H.263+ (refer to “Distributed Video Coding” by Girod, Aaron, Rane and Rebollo-Monedero, Proc. of IEEE, pp.71-83 2005)

5. Why is Practical Source Difficult? • The gap between theory and practice: the correlation structure of practical sources is often characterized by • Nonstationary instead of stationary • Location-based instead of intensity-based • The distributed constraint of exploiting source correlation • Adaptation at the encoder is more difficult • Optimization at the decoder instead of encoder

6. Distributed Coding of Stereo Images Y Y X X Disparity estimation & compensation Distributed encoder X Distributed encoder Y Quantization & entropy coding Centralized decoder How to shift disparity estimation from encoder to decoder? Centralized encoder

7. Asymmetric vs. Symmetric Coding Left eye: X Right eye: Y Left sensor: X Right sensor: Y Amblyopia (Lazy Eye) Interlaced sampling Interlaced versions of left and right images are jointly decoded along with disparity estimation Left-eye and right-eye images with varying acuity are jointly decoded by HVS to estimate disparity. Asymmetric coding protocol Symmetric coding protocol

8. Symmetric Coding Protocol Primary Xeven,Yodd (finely-quantized) Secondary Xodd,Yeven (coarsely-quantized) Left image: X Right image: Y Correlation model: D: disparity compensation W: additive noise term Side Information (SI) from primary Coded Information (CI) from secondary

9. Side-Information (SI) Generation To estimate missing rows (SI), We need to know disparity maps To estimate disparity maps, We need to know missing rows How to solve such problem of chicken-and-egg flavor ?

10. EM-like Iterative Decoding Initialization (spatial interpolation) Update the estimation of missing rows (disparity-based compensation) Update the estimation of disparity maps (stereo matching/ disparity estimation)

11. Convergence and Caveat original reconstructed Intensity profile Energy of intensity update right left Iteration number column variable Phase misalignment (trapped in local minimum) Fast convergence of our iterative algorithm

12. Coded Information (CI) Exploitation Primary SI Xeven,Yodd (finely-quantized) Secondary CI Xodd,Yeven (most significant bit) Left image: X Right image: Y Kill two Birds with one Stone (secondary channel) • Work as the synchronization marker to resolve location ambiguity • Work as the quantization constraint to correct intensity errors

13. Benefits of Secondary Channel • Phase Alignment • Intensity Refinement Work as the synchronization marker to resolve location uncertainty CI SI original Work as the quantization constraint to correct intensity errors 201 50

14. Experimental Setup square tsukuba room low middle high Complexity of disparity field

15. Improvement of Generated SI room tsukuba Blue line: left image, green line: right image

16. Benefits of Secondary Channel Original EM-like scheme (no phase alignment or intensity refinement) Improved EM-like scheme (with phase alignment but no intensity refinement) Improved EM-like scheme (with both phase alignment and intensity refinement)

17. Disparity Map Comparison Distributed decoding Centralized encoding (The stereo matching scheme we used is fast yet less accurate)

18. Disparity Field Comparison Distributed decoding (uncorrected errors due to local minimum trap) Centralized encoding

19. Preliminary Coding Results H264 I+I Ours H264 I+P Room pair (256×256)

20. Summary • Distributed coding of stereo images • Why? – It calls for the modeling of nonstationary location-based correlation • How? – We propose a symmetric coding protocol in which SI is generated by EM-like iterative disparity estimation and CI is exploited to jointly refine intensity and disparity uncertainty at the decoder • What is next? – We are working on modeling more complicated correlation structure in distributed video coding