1. PCL Range Images Bastian Steder

2. Topics • Range Images • NARFs (Normal Aligned Radial Features) • Example applications

3. Preparation • Download: http://www.informatik.uni-freiburg.de/~steder/data/office_scene.pcd • … and save it somewhere on your disc,e.g., directly in your home:~/office_scene.pcd

4. What are Range Images? http://www.informatik.uni-freiburg.de/~steder/data/office_scene.pcd

5. Implementation • The RangeImage class is derived from PointCloud<PointWithRange> So every pixel has a range value but is alsoa 3D point.

6. Range Images in the Code • Header:range_image/include/pcl/range_image/range_image.h • How to create a range image from a point cloud: • Corresponding Tutorial:Range Images → How to create a range image from a point cloud RangeImage range_image; range_image.createFromPointCloud ( point_cloud, angular_resolution (deg2rad(0.5°)), max_angle_width (360°), max_angle_height (180°), sensor_pose, coordinate_frame (CAMERA_FRAME), noise_level (0.0), min_range (0.0), border_size(0) );

7. Compile & Try $ cd $PCL_ROOT/doc/tutorials/content/sources/range_image_creation $ mkdir build $ cd build $ cmake .. $ make $ ./range_image_creation range image of size 40x34 with angular resolution 1deg/pixel and 1360 points

8. Compile & Try • We can visualize range images using the RangeImageVisualizer class.(We already covered this in the visualization talk) • Corresponding Tutorial:Visualization → How to visualize a range image

9. Border extraction • How to differentiate foreground from background?

10. How to detect these borders? • Analyze changes in distances toneighboring points

11. How to detect these borders? • Analyze changes in distances toneighboring points

12. How to detect these borders? • Analyze changes in distances toneighboring points

13. Border extraction example

14. In the Code • Header:features/include/pcl/features/range_image_border_extractor.h • Code Example: • Corresponding Tutorial:Range Images → How to extract borders from a range image RangeImageBorderExtractor border_extractor (&range_image); PointCloud<BorderDescription> border_descriptions; border_extractor.compute (border_descriptions);

15. Compile & Try $ cd $PCL_ROOT/doc/tutorials/content/sources/range_image_border_extraction $ mkdir build $ cd build $ cmake .. $ make $ ./range_image_border_extraction ~/office_scene.pcd

16. NARF Keypoints • Keypoints:Points on 3D structure that can be reliably detected in the same place, even if observed from different viewpoints.

17. NARF Keypoints Procedure

18. In the Code • Header:keypoints/include/pcl/keypoints/narf_keypoint.h • Code Example: • Corresponding Tutorial:Keypoints → How to extract NARF keypoints from a range image RangeImageBorderExtractor range_image_border_extractor; NarfKeypoint narf_keypoint_detector (&range_image_border_extractor); narf_keypoint_detector.setRangeImage (&range_image); narf_keypoint_detector.getParameters ().support_size = 0.2; PointCloud<int> keypoint_indices; narf_keypoint_detector.compute (keypoint_indices);

19. Compile & Try $ cd $PCL_ROOT/doc/tutorials/content/sources/narf_keypoint_extraction $ mkdir build $ cd build $ cmake .. $ make $ ./narf_keypoint_extraction ~/office_scene.pcd

20. Descriptors • Describe an area by a vector of real numbers • Allow fast similarity comparison using standard norms (0.23, 0.45, 0.65, …)

21. NARF descriptors calculation Wanna try this visualization on another scene? Use the code from the tutorial ‘Visualization → Visualization of the NARF descriptor and descriptor distances’ ($PCL_ROOT/doc/tutorials/content/sources/narf_descriptor_visualization)

22. In the Code • Header:features/include/pcl/features/narf_descriptor.h • Code Example: • Corresponding Tutorial:Features→ How to extract NARF Features from a range image NarfDescriptor narf_descriptor(&range_image, &keypoint_indices); narf_descriptor.getParameters().support_size = 0.3; narf_descriptor.getParameters().rotation_invariant = true; PointCloud<Narf36> narf_descriptors; narf_descriptor.compute(narf_descriptors);

23. Compile & Try $ cd $PCL_ROOT/doc/tutorials/content/sources/narf_feature_extraction $ mkdir build $ cd build $ cmake .. $ make $ ./narf_feature_extraction ~/office_scene.pcd Extracted 60 descriptors for 55 keypoints.

24. Application Examples I • Object Recognition

25. Application Example II • Place Recognition Hanover2 dataset (Courtesy of Oliver Wulf)