A High Dynamic Range Vision Approach to Outdoor Localization

1. A High Dynamic Range Vision Approach to Outdoor Localization Kiyoshi Irie, Tomoaki Yoshida, and Masahiro Tomono 2011 IEEE International Conference on Robotics and Automation Shanghai International Conference Center May 9-13, 2011, Shanghai, China Andy {andrey.korea@gmail.com}

2. Problem setting Auto exposure HDR 9am 4pm 9am 4pm Main goal: design algorithm for localization robust to changing in illumination conditions based on SIFT

3. HDR Keypoint set - differently exposed images ordered by shutter speed - sets of SIFT keypoints detected at corresponding images - all detected keypoints - duplicated keypoints - HDR keypoint set

4. Detecting duplicated keypoints Essential matrix Computed from odometry Translation matrix Rotation matrix • Only keypoints of neighboring images are compared together • Every keypoint in Kj (j=0,…,n-1) is compared with every keypoint in Kj+1 • Keypoints with smallest euclidean distance of their feature vectors are selected and stored as Mj • Check each matched pair in Mj to satisfy the following condition:

5. Detecting keypoints example Keypoints detected by SIFT Matched pairs of keypoints False matching removed by constraint

6. Localization • Basic assumptions: • Robot assumed to navigate on flat 2D surface • 2D pose of robot defined as • odometry information is known. Errors in odometry assumed to have normal distribution. • Localization is based on Monte-Carlo method

7. Localization Prediction phase motion model previous state Main task in robot localization: estimate robot state at time-step k, given knowledge about the initial step and all measurements Zk={z1,z2,…,zk} In terms of probability we need to construct posterior density At each step we know all previous measurements z1,…,zk-1 and control input uk-1 Typical localization process Update phase

8. Monte-Carlo Localization method Main idea: represent as a set of N random samples Prediction phase Update phase Apply motion model to each particle Weight each sample taking measurement into account Resample according to weight

9. Localization For each particle i the HDR keypoint set that is close to particle position is choosen Detected keypoints are matched with keypoints in map forming a set of pairs: Particles are scored by counting the number of matched pairs. Relative camera pose is calculated using robot pose of and the pose of the particle at time t 1. In the prediction step for each particle a new set of particles is generated: 2. Set of HDR keypoints Ht is computed 3. Particles are updated by weighting each particle using the likelihood of Ht given map M and particles

10. Robot Resolution: 2448 x 2048 Framerate: 15 FPS Angles: 185° x 185°

11. Conclusions

12. Accuracy of localization

13. Computational time

14. Accuracy of localization

15. Position tracking results

16. Conclusions • Advantages : • Localization method using High Dynamic Range vision is proposed • Localization based on Monte-Carlo method • Using HDR images increases number of key-points • HDR key-point set improves localization in terms of accuracy and computational cost • Future work: • How to determine optimal number of exposures? • Shadows could effect matching

17. Images captured by robot in 3 different points