Biologically inspired Mobile Robot Vision Localization

1. Biologically inspired Mobile Robot Vision Localization Presenter FolamiAlamudun Authors Christian Siagian Laurent Itti

2. IntroductionVision-based LocalizationScene RecognitionTopological MapsBiological Vision Localization System Experimental ResultsDiscussionRelated work

3. What? • Robot localization system using biologically inspired vision • Why? • Provide machines with a human-like perceptual system capable conducting intelligent localization in an unstructured environment. • How? • Biologically inspired scene summarization (gist) and landmark identification (saliency). Introduction

4. IntroductionVision-based LocalizationScene RecognitionTopological MapsBiological Vision Localization System Experimental ResultsDiscussionRelated work

5. Vision • Primary perceptual system for localization in most animals (including humans). • Effective in most environments where sonar, radar and GPS are unavailable or inoperable. • The human process of localization is performed using two processes: • Gist – A holistic statistical signature of the image, thereby yielding abstract scene classification and layout. • Saliency – A measure of interest at every image location and landmark-identification. Vision-based Localization

6. Vision-based localization systems use vision information to classify systems using: • Global features – A general summary of information over the entire image. • Local features – Computed over a limited area of the image Vision-based Localization

7. Global feature methods generally consider an input image as a whole and extract a low-dimensional signature. Advantage: • Provides a summary of the image statistics or semantics. • Robust because random local pixel noise averages out on global scale. Disadvantages: • Sacrifices spatial information such as feature location and orientation. • Unable to define accurate pose estimation • Harder to deduce positional change even with significant robot movement. Vision-based Localization – Global Features

8. Local feature methods limit their scope to image regions and their respective configuration relationships to form a signature of a location. Advantage • local features encode scene characteristics that are more focused in scope. • Invariant in scale, in-plane rotation, viewpoint and lighting invariance. Disadvantage • Very slow. Vision-based Localization – Local Features

9. IntroductionVision-based LocalizationScene RecognitionTopological MapsBiological Vision Localization System Experimental ResultsDiscussionRelated work

10. Human visual processing system uses visually interesting regions within the field of view. • Saliency-based selection of landmarks that are most reliable in a particular environment. • Focusing on specific regions for comparing different images makes for a less computationally expensive process Scene Recognition

11. Scene Recognition

12. Scene Recognition

13. IntroductionVision-based LocalizationScene RecognitionTopological MapsBiological Vision Localization System Experimental ResultsDiscussionRelated work

14. A topological map is a graph annotation of an environment. • Topological Maps assign nodes to particular places and edges as paths if direct passage between pairs of places (end nodes) exist. • Humans manage spatial knowledge primarily by topological information. • This information is used to construct a hierarchical topological map that describes the environment. Topological Maps

15. IntroductionVision-based LocalizationScene RecognitionTopological MapsBiological Vision Localization System Experimental ResultsDiscussionRelated work

16. Biological Vision Localization System

17. The localization system is divided into three stages: • Feature extraction – Processes image to produce: • Gist features; • Salient regions. • Recognition - compares features with memorized environment visual information. • Localization – compute where the robot is situated. Biological Vision Localization System

18. Feature extraction involves processing of raw low-level filter outputs into gist and saliency modules. • Gist feature extraction • Computes average values from sub-regions of feature maps. • Dimensionality reduction using PCA/ICA • Salient region selection and segmentation • Uses feature maps to detect conspicuity regions in each channel. Biological Vision Localization System – Feature extraction

19. Biological Vision Localization System – Gist Feature extraction

20. Biological Vision Localization System – Gist Feature extraction

21. This stage attempts to match salient regions and gist features with stored environment information. • Segment estimator: • Three-layer neural network classifier trained using the back-propagation on gist features • Salient Region Recognition: • Recalls stored salient regions • Uses SIFT key points and salient feature vector to recognize regions. Biological Vision Localization System – Segment and Salient Region Recognition

22. Segment Estimation computes likelihood that a scene belongs to a segment: • Salient region localization provides a saliency map which highlights coordinates of peak values (salient points). • These points are used for identification purposes in subsequent viewing. Biological Vision Localization System

23. Recollection of stored salient regions for localization involves: • SIFT keypoints • SIFT recognition system using parameters and thresholds. • Salient feature vector • A set of values taken from 5x5 window centered at the salient point location. Biological Vision Localization System – Salient Region Recognition

24. (continued) • Salient feature vectors form two salient regions (sreg1, sreg2) are compared using: • Similarity • Proximity Biological Vision Localization System – Salient Region Recognition

25. Biological Vision Localization System – Salient Region Recognition

26. When a landmark is recognized its associated location is used to deduce robot location. • Accumulated temporal context is used to distinguish between identical landmarks. • Robot position is estimated by implementing Monte-Carlo Localization (MCL) which utilizes Sampling Importance Resampling (SIR). Biological Vision Localization System – Monte Carlo Localization

27. St as a set of weighted particles: • St = {xt,i, wt,i}, (i = 1, . . . , N) • xt,i = {snumt,i , ltravt,i} (possible robot location) • snum – segment number • Ltrav – length traveled along segment edge • wt,i = weight likelihood. • at time t; and • N is the number of particles. • Bel(St) = location belief at time t. • ut = motion measurement Biological Vision Localization System – Monte Carlo Localization

28. Belief estimation algorithm: • Apply motion model to St−1 to create St’ . • Apply segment observation model to St’ to create St’’. • IF (Mt > 0): • apply salient region observation model to St’’ to yield St ; • ELSE St = St’’. Where: • St’ = is the belief state after application of motion model. • St’’ = is the state after the segment observation Biological Vision Localization System – Monte Carlo Localization

29. Biological Vision Localization System – Monte Carlo Localization

30. IntroductionVision-based LocalizationScene RecognitionTopological MapsBiological Vision Localization System Experimental ResultsDiscussionRelated work

31. Experimental Results – rigid environment Lighting conditions test

32. Experimental Results – rigid environment Lighting conditions test

33. Experimental Results – rigid environment Test System response on sparser scenes

34. Experimental Results – rigid environment Test System response on sparser scenes

35. IntroductionVision-based LocalizationScene RecognitionTopological MapsBiological Vision Localization System Experimental ResultsDiscussionRelated work

36. This paper introduced new ideas (the use of complementary gist and saliency features). • Saliency model lets the system automatically select persistent salient regions as localization cues. • Low computation cost gist features approximate the image layout and provide segment estimation. • System is able to compute coordinate level localization in multiple environments • Performance is comparable to GPS database guided systems. Discussion

37. Determining Patch Saliency Using Low-Level Context. European Conference on Computer Vision (ECCV), 2008. D. Parikh et. al. Related Work