Visual Detection of Lintel-Occluded Doors from a Single Image

1. Visual Detection of Lintel-Occluded Doors from a Single Image Zhichao Chen and Stan Birchfield Dept. of Electrical and Computer Engineering Clemson University Clemson, South Carolina USA

2. Types of Maps Metric map Topological map Either way, doors are semantically meaningfullandmarks

3. Previous Approaches to Detecting Doors • Range-based approaches • sonar [Stoeter et al.1995] • stereo [Kim et al. 1994] • laser [Anguelov et al. 2004] • Vision-only (uncalibrated) • low cost • low power • non-contact (passive) measurement • rich capturing ability

4. Vision-Based Door Detection fuzzy logic [Munoz-Salinas et al. 2004] color segmentation [Rous et al. 2005] • neural network • [Cicirelli et al 2003] • Limitations: • require different colors for doors and walls • simplified environment (untextured floor, no reflections) • limited viewing angle • high computational load • assume lintel visible

5. Problem Statement Detect doors in complex environments: • textured and untextured floors • walls and doors with similar colors • specular reflections • variable lighting conditions • wide range of robot poses

6. Another Challenge Lintel-occluded • post-and-lintel architecture • camera is low to ground • cannot point upward b/c obstacles lintel post

7. Video

8. concavity door gap vanishing point kick plate color texture vertical lines Our Approach Standard features Door detected Adaboost Novel features Assumptions: • Both door posts are visible • Posts appear nearly vertical • The door is at least a certain width

9. Pairs of Vertical Lines vertical lines detected lines Canny edges non-vertical lines • Edges detected by Canny • Line segmentsdetected by modified Douglas-Peucker algorithm • Clean up (merge lines across small gaps, discard short lines) • Separate vertical and non-vertical lines • Door candidates given by all the vertical line pairs whose spacing is within a given range

10. Cue #1: Color • Threshold the histogram intersection between the wall color model φwall and the color histogram φdoor computed between two vertical line segments. • Wall color model can be built either by hand, or automatically. positive negative

11. Cue #2: Texture • The bottom part of the door is usually untextured. • Texture energy is computed by summing the magnitude of the gradient in the lower region of the door. positive negative

12. Cue #3: Gap Below the Door Intensity along the line darker (light off) positive brighter (light on) negative no gap

13. Cue #4: Kick Plate R positive negative • Kick plates occur in 30% of images • Segmentation algorithm of Felzenszwalb et al., IJCV 2004 is used (15 fps on 160x120 downsampled image) • A region R in the segmented image is considered as kick plate if: • the region R is located between two vertical lines • the bottom of R is near the bottom of the two vertical lines • the width and height of R are within a specified range

14. Cue #5: Vanishing Point • The vanishing point is computed as the mean of the intersection of pairs of non-vertical lines. • If the bottom door edge passes near the vanishing point, then the test succeeds. A distracting line caused by shadows does not intersect the vanishing point The bottom line of a door should intersect the vanishing point

15. Cue #6: Concavity Slim “U” vertical door lines wall door wall Lleft bottom door edge intersection line of wall and floor extension of intersection line LRight ε floor

16. w Detect Concavity Slim “U” Extended line negative positive Concavity is declared if at least two of the three tests (Hrec(L), Hrec(R), and HU) succeed.

17. dallowed Douglas-Peucker Algorithm Line segmenting Edge labeling Canny edges

18. dallowed Modified Douglas-Peucker Algorithm original algorithm: dallowed is constantmodified algorithm: dallowed is given by half-sigmoid function original modified Note: important for concavity cue

19. Adaboost • Bayes decision rule: declare a door if the a posteriori probability of the predicate is greater than that of its complement: where • The strong classifier is given by a weighted sum of the weak classifiers: where weight , error

20. Experimental Results: Similar or Different Door/Wall Color

21. Experimental Results: High Reflection / Textured Floors

22. Experimental Results: Different Viewpoints

23. Experimental Results: Cluttered Environments

24. Results • 20 different buildings • 309 images: • 100 training • 209 testing • 90% accuracy with • 0.05 FP per image Speed: 5 fps (unoptimized)

25. False Negatives and Positives strong reflection concavity and bottom gap tests fail distracting reflection two vertical lines unavailable concavity erroneously detected distracting reflection

26. Navigation in a Corridor • Doors were detected and tracked from frame to frame. • Fasle positives are discarded if doors were not repeatedly detected.

27. Video

28. Conclusion • Conclusion • Detect doors using a single uncalibrated camera in a variety of environments • Augment standard features (color, texture, and vertical edges) with novel features (concavity, door gap, kick plate, and vanishing point) • The features are combined in an Adaboost framework • Suitable for real-time mobile robot applications using an off-the-shelf camera • Future work • on-line learning of hall color • building of a geometric map • detecting open doors