1 / 37

Exploiting Inertial Planes for Multi-sensor 3D Data Registration

Exploiting Inertial Planes for Multi-sensor 3D Data Registration. PhD dissertation Hadi Aliakbarpour Faculty of Science and Technology October 2012, University of Coimbra. Introduction. Introduction: Problem Statement .

ofira
Télécharger la présentation

Exploiting Inertial Planes for Multi-sensor 3D Data Registration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Exploiting Inertial Planes forMulti-sensor 3D Data Registration PhD dissertation Hadi Aliakbarpour Faculty of Science and Technology October 2012, University of Coimbra

  2. Introduction

  3. Introduction: Problem Statement • This dissertation investigate the problem of multi-sensor 3D data registration using a network of IS-camera pairs. • Target applications: Surveillance, human behaviourmodelling, virtual-reality, smart-room, health-care, games, teleconferencing, human-robot interaction, medical industries, and scene and object understanding

  4. Introduction: Motivation • Performing 3D data registration and scene reconstruction using a set of planar images is still one of the key challenges of computer vision. • A network of cameras, whose usage and ubiquitousness have been increasing in the last decade, can provide such planar images from different views of the scene. • Recently, IS has been becoming much cheaper and more available so that nowadays most smart-phones are equipped in both IS and camera sensors. 3D earth cardinal orientation (North-East-Down) is one of the outputs of an IS. • How can we benefit from having a network of IS and camera couples, for the purpose of 3D data registration?

  5. Introduction: Overall View

  6. Main Contributions • A homographic framework is developed for 3D data registration using a network of cameras and inertial sensors. Geometric relations among different projective image planes and Euclidean inertial planes involved in the framework are explored. [AD12a] [AD11c] [AD10b] [AD11b] [AD10a] [AFKD10] [AFQ+11]. • A real-time prototype of the framework is developed which is able to perform fully reconstruction of human body (and objects) in a large scene. The real-time characteristic is achieved by using a parallel processing architecture on a CUDA-enabled GP-GPU [AAMD11]. • A two-point-based method to estimate translations among virtual cameras in the framework is proposed and verified [AD12a] [AD11a] [AD10a] [AFQ+11]. • The uncertainties of the homography transformations involved in the framework and their error propagations on the image planes and Euclidean planes have been modelized using statistical geometry.

  7. Additional Contributions • Within the context of the proposed framework, a genetic algorithm is developed to provide an optimal coverage of the camera network to a polygonal object (or a scene). • A method to estimate extrinsic parameters among camera and laser range finder is developed [ANP+09]. A related SLaRF; available to download at http://isr.uc.pt/~hadi is prepared.

  8. Methodologies onMain Contributions

  9. Virtual Camera: Concept & Geometry

  10. Mapping: From Scene onto Inertial Planes A 3D point X is registered on different Euclidean planes using homographies

  11. Extension to a Network of IS-Camera Pairs A network of IS-camera couples is used to observe the scene from different views

  12. 3D Reconstruction Using Inertial Planes: Illustrative example An exemplary case: a person is observed by three cameras Off-the-plane point Y is subject to parallax and on-the-plane point X with no parralax Top-view of the registration plane. Area in white is the intersection to the person.

  13. Translation Among Virtual Cameras Knowing the heights of two 3D points (X1 and X2) is sufficient to recover the translation (t) among two cameras

  14. Parametric Homography RelationsAmong Different Planes in the Framework

  15. Real-time implementation using GP-GPU

  16. Parallelized Operations: Virtual Planes Generations

  17. Modelization of Uncertainties The uncertainties of the homography transformations involved in the framework and their error propagations on the image planes and Euclidean planes have been modelized using statistical geometry.

  18. Modelization of Uncertainties: examples • Uncertainty of point μ’X , where mapped from μref to μ’ : • Uncertainty of point μ(k)X , where mapped from μk-1 to μk :

  19. Methodologies onAdditional Contributions

  20. Optimization for Sensor Configuration (Camera Placement) The quality of reconstruction using a camera network depends to mainly three parameters: • Number of cameras • The quality of the applied background subtraction technique • The cameras configurations (e.g. positions)

  21. Optimization for Sensor Configuration: Problem Statement An exemplary convex polygon with 5 edges are observed by two camera. The problem is how to arrange cameras to have optimum registration of the polygon with most completeness. C2 C1 e1 e2 e5 e3 e4 Y {W} X After registering with the present camera configuration: An extra part colored in red is registered as a part of the object!

  22. Optimization for Sensor Configuration: Solution Solution: To use geometry (e.g. normal of the edges etc. ), define some cost functions and applying GA.

  23. Integration of a Laser Range Finder LRF is an active sensor which can be used as a complementary sensor to the cameras: Comparison table Estimation of the rigid transformation, C TL(α) , among a stereo camera and a LRF

  24. Experiments

  25. Volumetric 3D Reconstruction: Offline 47’th virtual plane Setup and scene Second virtual plane Statue First virtual plane

  26. Analysis on Translation Estimation: Certainties w.r.t. Noise in IS Empirical analysis the effects of IS noise to the translation estimation method Input noise in degrees (roll, pitch and yaw of inertial sensor) Output uncertainty in cm (on three elements of the estimated translation vector)

  27. Analysis on Translation Estimation: Certainties w.r.t. other Noise Output uncertainty (cm) Output uncertainty (cm) Input noise (cm) Input noise (pixels)

  28. Volumetric 3D Reconstruction using Real-time using GPU: video

  29. Uncertainty of Virtual Image’s Points The uncertainties for pixels of the virtual camera’s image plane are demonstrated by covariance ellipses, where they are scaled 1000 times for clarity.

  30. Uncertainty of Inertial plane’s Points The uncertainties for different registered points on the Euclidean inertial plane, demonstrated by covariance ellipses. The blue and red ellipses stand for points registered by the first and second camera, respectively. For the sake of clarity the covariance values are scaled 500 and 600 times, respectively for the first and second cameras

  31. Changes of Uncertainty in Virtual Image Plane Uncertainties for an exemplary pixel x = [ 450 450 1 ]Twhere s = [ π/2 -π/2 0 ]T

  32. Optimization for Sensor Configuration (Camera Placement) 1200x1200 cm2

  33. Estimate of Extrinsic Parameters Among Camera And Laser Range Finder Image α = 2o Result (during 6 months) + α = 12o Range data Reprojection of LRF data on the image (blue points) α = 23.2o

  34. Conclusion & Future Work

  35. Conclusion • We investigated the use of IS for 3D data registration by using a network of cameras and inertial sensors. • A volumetric data registration algorithm was proposed. • Normally the volumetric reconstruction of a scene is time consuming due to the huge amount of data to be processed. In order to achieve a real-time processing, a prototype was built using GP-GPU and CUDA • A method to estimate the translation among cameras within the network was proposed. The certainty of the method has been evaluated in the presence of different noise. • The issue of sensor configuration, particularly the cameras’ positions in the scene was investigated and a geometric method to find an optimal configuration was proposed using genetic algorithm. • A method to estimate the extrinsic parameters among camera and LRF was proposed as a step towards applying range data in the framework.

  36. Future Work • Integration of range data within the proposed inertial-based data registration framework. • To develop a probabilistic algorithm for fusion of heterogeneous data, capable of dealing with the uncertainty of each sensor node. • To investigate a multi-layer 3D tracking of human/objects. In this future investigation, we will provide contribution to model and recognize the state of scene and to analyse the behavior of small group using probabilistic approaches.

  37. Thank you! 

More Related