Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013

1. Joint Histogram Based Cost Aggregationfor Stereo Matching - TPAMI 2013 Dongbo Min, Member, IEEE, Jiangbo Lu, Member, IEEE, Minh N. Do, Senior Member, IEEE M.S. Student, Hee-Jong Hong Sep 24, 2013

2. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Outline • Introduction • RelatedWorks • Proposed Method:Improve CostAggregation • ExperimentalResults • Conclusion

3. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Introduction • Goal：Perform efficient cost aggregation. • Solution : Jointhistogram+reduceredundancy • Advantage : Low complexitybutkeephigh-quality.

4. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Related Works N : all pixels (W*H) B : window size L : disparity level • Complexityofaggregation：O(NBL) • Reducecomplexityapproach • Scaleimage: Multi Scale ApproachD. Min and K. Sohn, “Cost aggregation and occlusion handling with WLS in stereo matching,” IEEE Trans. on Image Processing, 2008. • Bilateralfilter: Bilateral Approximation C. Richardt, D. Orr, I. P. Davies, A. Criminisi, and N. A. Dodgson, “Real-time spatiotemporal stereo matching using the dual-cross- bilateral grid,” in European Conf. on Computer Vision, 2010S. Paris and F. Durand, “A fast approximation of the bilateral filter using a signal processing approach,” International Journal of Computer Vision, 2009. • Guidedfilter: Run in constant time => O(NL)C.Rhemann,A.Hosni,M.Bleyer,C.Rother,andM.Gelautz,“Fast cost-volume filtering for visual correspondence and beyond,” in IEEE Conf. on Computer Vision and Pattern Recognition, 2011

5. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Proposed Method

6. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Local Method Algorithm • Cost initialization : Truncated Absolute Difference => • Cost aggregation : Weighted filter • Disparity computation : Winner take all [4,8]

7. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Improve CostAggregation • New formulation for aggregation • Remove normalization • Joint histogram representation • Compact representation for search range • Reduce disparity levels • Spatial sampling of matching window • Regularly sampled neighboring pixels • Pixel-independent sampling

8. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 New formulation for aggregation • Remove normalization => • Joint histogram representation

9. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Compact Search Range • Cost aggregation => • MC(q)：a subset of disparity levels whose size is Dc. N : all pixels (W*H) B : window size D : disparity level O( NBD ) O( NBDc )

10. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Compact Search Range • Non-occluded region of ‘Teddy’ image Dc= 60 Final Accuracy = 93.7% Dc= 6 Final Accuracy = 94.1% Dc= 5 (Best) Final Accuracy=94.2%

11. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Spatial Sampling of Matching Window • Reason • A large matching window and a well-defined weighting function leads to high complexity. • Pixels should aggregate in the same object, NOT in the window. • Solution • Color segmentation => Time consuming (Heavy) • Spatial Sampling => Easy but powerful • Regular Sampling => Independent from reference pixel => Reduce Complexity

12. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Spatial Sampling of Matching Window • Cost aggregation => • S : sampling ratio O( NBDc ) O( NBDc / S2)

13. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Parameter definition N : size of image B : size of matching window N(p)=W×W MD : disparity levels size=D MC : The subset of disparity size=DC<<D S : Sampling ratio Pre-procseeing

14. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Experimental Result

15. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 ExperimentalResults • Pre-processing • 5*5 Box filter • Post-processing • Cross-checking technique • Weightedmedian filter (WMF) • Device：Intel Xeon 2.8-GHz CPU (using a single core only) and a 6-GB RAM • Parametersetting ( ) = (1.5, 1.7, 31*31, 0.11, 13.5, 2.0)

16. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 ExperimentalResults (a) (b) (c) (d)

17. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 ExperimentalResults • Using too large box windows (7×7, 9×9) deteriorates the quality, and incurs more computational overhead. • Pre-filtering can be seen as the first cost aggregation step and serves the removal of noise.

18. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 ExperimentalResults The smaller S, the better Fig. 5. Performance evaluation: average percent (%) of bad matching pixels for ‘nonocc’, ‘all’ and ‘disc’ regions according to Dc and S. 2 better than 1

19. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 ExperimentalResults The smaller S, the longer The bigger Dc, the longer

20. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 ExperimentalResults • APBP : Average Percentage of Bad Pixels

21. Original images Results Error maps Ground truth Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 ExperimentalResults

22. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 ExperimentalResults

23. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Conclusion

24. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Conclusion • Contribution • Re-formulate the problem withthe relaxed joint histogram. • Reduce the complexity of the joint histogram-based aggregation. • Achieved both accuracy and efficiency. • Futurework • Moreelaborate algorithms for selecting the subset of label hypotheses. • Estimate the optimal number Dc adaptively. • Extendthemethodtoanopticalflowestimation.

25. Thank you!