1 / 26

Kinect calibration

Kinect calibration. Facoltà di Ingegneria. Ilya Afanasyev. Trento, 2012. Content. The description. About the existing software. Tests of Nicolas Burrus software binaries. Open Kinect drivers installation. Compilation of Nicolas Burrus source code. Results . Conclusions

deron
Télécharger la présentation

Kinect calibration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Kinect calibration Facoltà di Ingegneria • Ilya Afanasyev • Trento, 2012

  2. Content • The description. • About the existing software. • Tests of Nicolas Burrus software binaries. • Open Kinect drivers installation. • Compilation of Nicolas Burrussource code. • Results. • Conclusions • References.

  3. The description. • The software for Kinect Calibration should be used for: • Saving the RGB images and IR-Depth data from Kinect sensor under OS Windows. • Calibration the Kinect sensor with obtaining [1]: - the intrinsic parameters (i.e. matrixes of intrinsic parameters for IR camera (K_ir) & RGB camera (K_rgb)).- the extrinsic parameters (i.e. extrinsic mapping between IR (depth) and RGB Kinect camera (R,T)).- distortion coefficients (kc_ir, kc_rgb). - the intrinsic and extrinsic parameters (i.e. matrixes of intrinsic parameters for IR camera (K_ir) & RGB camera (K_rgb) and extrinsic mapping between IR (depth) and RGB Kinect camera (R,T)). • Developing the interface of different algorithm with Kinect for different real-time object recognition applications.

  4. About the existing software. • The existing analogues of the software: • - The author doesn’t know analogues under Windows7/64 bit . It can be created, basing on Microsoft Kinect SDK for developers [6] under the platform .NET Framework ver. 4.0. • - There is thesoftware developed by Nicolas Burrus [2] under OS Linux / Ubuntu 10.10 / 32 bit, which is claimed to be compiled also with Windows-32 bit. Can it be used under Windows7/64 bit ?? -> It seems that NO!!

  5. Microsoft Kinect SDK for developers [6] • SDK (Software Development Kids) includes drivers, APIs for raw sensor streams and human motion tracking, installation documents, and resource materials. It provides capabilities offered by the Microsoft Kinect device connected to computers under Windows 7 / Windows 8 to developers who build applications with C++, C#, or Visual Basic by using Microsoft Visual Studio 2010.

  6. Nicolas Burrus software [2] • Nicolas Burrus software [2]: • There are different versions of the Burrus software:1. Test programs running frombinaries -> Windows binaries -> Kinect RGBDemo v0.6.1. -> supports OpenNI/Nitebackends[3] and has the experimental infrared support with OpenNI (still buggy) -> I tried to check this work for OpenNI backend under Windows 64bit -> there is no the extrinsic mapping between IR and RGB camera at Kinect calibration.2. Compiling from source -> 2.1 Compilation on Windows -> with OpenNI/Nitebackendsdoesn’t give the extrinsic mapping between IR and RGB camera at Kinect calibration.with Libfreenect backend -> I wasn’t succeed to compile under Windows 7/64 bit?! 2.2 Compilation on Linux (Ubuntu10.10 - 32 bit) -> supports Libfreenect backend -> the only workable variant with getting all calibration parameters (intrinsic, extrinsic and distortion) was realized on Alberto Fornaser’s PC (UniTN)

  7. The using of Nicolas Burrus software [2] with OpenNI backend under Windows 1st attempt • 1. OpenNI backend: • Nicolas Burrus software [1]: Kinect RGBDemo v0.6.1. -> supports OpenNI/Nitebackends [2] and has the experimental infrared support with OpenNI (still buggy) -> I tried to check this work for OpenNI backend under Windows 64bit. • Install Sensor Kinect drivers under Windows 32 bit [4] and OpenNI/Nite modules (Win32) [3] according to the instruction [5] and verified that they work properly. • Use RGB-D Capture [1] to grab Kinect RGB and IR images with intensity.raw and depth.raw files. -> the software doesn’t grab IR images!! -> only RGB image, intensity.raw, depth.raw and calibration file (calibration.yml without distortion parameters)!! • “calibrate-openni-intrinsics --pattern-size 0.0405 grab1 calibration.yml” -> gives “openni_calibration.yml” with Intrinsic matrix and distortion coefficients for Kinect RGB camera !! • “calibrate-openni-depth.exe --pattern-size 0.0405 grab1” gives partly processed figures with a message about errors. • The software doesn’t produce a file with xyz- coordinates and RGB-colors info of every point!!!

  8. The comparison of Kinect calibration results for - Libfreenect backend (under OS Linux / Ubuntu 10.10 - 32 bit) - OpenNI backend (under OS Windows 32bit) Libfreenect backend OpenNI backend The Depth Camera Calibration for OpenNI backend is NOT available!!! • The same figures of Intrinsic Matrixes for Kinect RGB and Depth Cameras

  9. The comparison of Kinect calibration results for - Libfreenect backend (under OS Linux / Ubuntu 10.10 - 32 bit) - OpenNI backend (under OS Windows 32bit) Libfreenect backend OpenNI backend The reprojection of 3D points from Kinect Depth Camera to RGB image for OpenNI is NOT possible!! • There is NO extrinsic mapping between Kinect Depth and RGB cameras for OpenNI backend

  10. Install OpenKinect for Windows 7 [7] • It is necessary to have:- Visual Studio 2010 (as a compiler on a 32 bit Windows 7 installation).- Kinect Sensor drivers for Windows 7 (OpenKinect-libfreenectofTheo/ofxKinect) [9].- freenect windows dependencies [8]: libusb-win32 1.2.4.0 (Currently there is a breaking change in libusb 1.2.5.0, which I used!!); pthreads-win32  Glut

  11. Kinect Sensor drivers installation[7] • - Download & extract the latest GitHub repository [9]. • - Make sure your Kinect sensor is plugged into a USB port on your computer. • - Open Device Manager (Start  Control Panel  System and Security  System  Device Manager) • - Find “Human Interface Device” (Xbox NUI Motor)  Update Driver Software  uninstall the previous Kinect drivers (if there are)  set path like C:\Special\OpenKinect_libfreenect_v0_1_1_56\platform\windows\inf\xbox nui camera\  the drivers will correspond to the processor type.

  12. Kinect Sensor drivers installation[7] • Check the files: libusb0.dll (x64) in C:\Windows\System32\and libusb0.sys in C:\Windows\System32\drivers\and libusb0.dll (x86) in C:\Windows\SysWOW64\

  13. Cmake configuration & Generation of a Visual Studio 10 solution [7] • Open Cmake GUI and set the Browse Source path to /libfreenect/openkinect. • Set the browse build path to /libfreenect/openkinect/build • The software should be configured when Kinect turned off!!

  14. Cmake configuration & Generation of a Visual Studio 10 solution [7] • You will now need to point certain build paths to the freenect dependencies (pthread). • The software should be configured when Kinect turned off!!

  15. Cmake configuration & Generation of a Visual Studio 10 solution [7] • You will now need to point certain build paths to the freenect dependencies (libusb). • If take "msvc" instead of "msvc_x64" -> 1 fatal error of compilation less !!! • It doesn't matter if choose THREADS\LIB\pthreadVC2.lib or pthreadVSE2.lib

  16. Cmake configuration & Generation of a Visual Studio 10 solution [7] • Run Configure again. It should successfully complete the process this time but we still need to add the Glut references. • Run "Configure" and it should successfully complete. Click Generate afterwards and then open the libfreenect.sln file from the build folder the output was placed in - /openkinect/build/

  17. Visual Studio 10 solution [7] • Disable the Antivirus if it disturbs the creation of the VS10 solution! • Choose Visual C++ 2010 and then select Build Solution from the Debug Menu or press F7. • RESULT: Rebuild All: 13 succeeded, 2 failed, 2 skipped. • You will receive a few errors when building the solution but it should compile correctly regardless. The binaries will be placed in build/bin/Debug.

  18. The result of building the c for Windows 7 [7] • The final step is to place a few .dll files in the appropriate folder and then we can run some of the samples provided: Copy freenect.dll and freenect_sync.dll from \build\lib\Debug •  Copy from dependencies folders: pthreadVC2.dll, glut32.dll, libusb0.dll (choose the correct version x86/x64) •  Copy tiltdemo.exe, regtest.exe, glview.exe, glpclview.exe, glpclview.exe, regview.exe, hiview.exe from \build\bin\Debug •  It can be helpful to copy freenect_sync.lib, freenect.lib, freenect.exp from \build\lib\Debug to \lib\ • Successful start of demo-samples Kinect works!

  19. Demo-samples of OpenKinectfor Windows 7 •  tiltdemo.exe drives the Kinect motor. •  regview.exeshows the combined IR&RGB video (IR up to 1 m) •  regtest.exe grabs bad-quality pictures: -- registration_test_depth_registered.pgm, -- registration_test_rgb.ppm,-- registration_test_depth_raw.pgm, -- registration_test_depth_mm.pgm •  glview.exe gives IR&RGBvideo in separate windows (IR up to 3.5 m)

  20. Demo-samples of OpenKinect for Windows 7 • hiview.exe gives IR&RGBvideo in separate windows (IR up to 3.5 m, RGB has hi-quality & hi resolution) • glpclview.exe gives color 3D video with big shadows behind the objects. • cppview.exe produces a continuous list with messagesunder the phrase:“thread execution started”

  21. Compiling RGBDemo-0.6.1 under Min GW from source with OpenKinect on Windows 7 [2] 2ndattempt The step-by-step procedure for Min GW, in case using libfreenect: 1. Install QT opensource for Windows. This will also install Min GW. 2. Add C:\Special\QtSDK\mingw\bin\ to the Path environment variable: Start  Computer (properties)  Advanced System Settings  Environmental Variables (button)  Variable: PATH  Edit  Add. 3. Install and run cmake on rgbdemo Disable the NESTK_USE_OPENNIcmake variable  “Could not find module FindOpenCV.cmake”  “Configuring incomplete, errors occurred!” 4. Ungrouped Entries   OpenCV_DIR (NOT FOUND)  C:/Special/opencv. QT_QMAKE_EXECUTABLE (NOT FOUND)  C:\Special\QtSDK\Desktop\Qt\4.7.4\mingw\bin\qmake.exe it is better to use qmake.exe from QtSDK\mingw ??

  22. Compiling RGBDemo-0.6.1 under Min GW from source with OpenKinect on Windows 7 [2] The step-by-step procedure for Min GW, in case using libfreenect:  Open the CMakeLists.txt in Qt Creator (from C:\Special\RGBDemo-0.6.1-Source\RGBDemo-0.6.1-Source\CMakeLists.txt open with C:\Special\QtSDK\QtCreator\bin\qtcreator.exe).  It creates the folder: qtcreator-build  Add QT-qmake & OpenCV path to the Path environment variable:C:\Special\QtSDK\Desktop\Qt\4.7.4\mingw\bin\C:\Special\opencv\ to the Path environment variable: Start  Computer (properties)  Advanced System Settings  Environmental Variables (button)  Variable: PATH  Edit  Add.

  23. Compiling RGBDemo-0.6.1 under Min GW / MSVS 2010 from source with OpenKinect on Windows 7 [2] The step-by-step procedure for Min GW / MSVS2010, in case using libfreenect:  Cmake 2.8.6  Ungrouped Entries  PKG_CONFIG_EXECUTABLE  C:\Special\opencv\OpenCVFindPkgConfig.cmake Results: Cmakecan’t compile the QT & OpenCV_DIR  OpenCVConfig.cmake  We should use Qt!!  Qt  Run Cmake  “CMAKE_MODULE_PATH C:\Special\qtcreator-build\”  Run Cmake “CUDA_TOOLKIT_ROOT_DIR not found or specified” [10]  if set path to C:\Special\PCL_1_4_0\ which contains CUDA libraries, then we have the messages “CMake Error: The source directory "C:/Special/PCL_1_4_0/3rdParty/flann/bin" does not appear to contain CMakeLists.txt. Specify --help for usage, or press the help button on the CMake GUI”. • Burrus software “RGBDemo-0.6.1” is not installed on Window7-64bit!!

  24. Conclusions • 1. Nicolas Burrus software with OpenNIbinaries [2] works under Windows 7-64 bit, but the software doesn’t give the extrinsic mapping between Kinect Depth and RGB cameras and doesn’t produce a file with xyz- coordinates and RGB-colorsinfo of every point!!! • 2. Although the drivers of OpenKinect has been successfully installed under Windows 7-64 bit, the Nicolas Burrussource code RGBDemo-0.6.1 (with LGPL License)for Kinect Calibration [2] is not compiled properly under Windows 7-64 bit.  It should be used another software or another Kinect drivers!! • 3. Both Microsoft Kinect SDK for developers [6] and OpenKinect drivers [9] can be applicable to save RGB images and IR-Depth data to calibrate Kinect sensor and use it under Windows 7-64 bit, but no suitable software has not been produced yet.

  25. 5. References Camera Calibration and 3d Reconstruction. // OpenCV (Open Source Computer Vision) v2.1 documentation: http://opencv.willowgarage.com/documentation/cpp/camera_calibration_and_3d_reconstruction.html Nicolas Burrus. Kinect. RGBDemo, calibrate and visualize Kinect output. // http://nicolas.burrus.name/, 2011. OpenNI Modules. // www.openni.org, SensorKinect drivers. // github social coding. https://github.com/avin2/SensorKinect. How-to: Successfully install Kinect on Windows (OpenNI and NITE). // VangosPterneas blog: http://studentguru.gr/b/vangos/archive/2011/01/20/how-to-successfully-install-kinect-windows-openni-nite.aspx Microsoft Kinect SDK for developers. http://www.microsoft.com/en-us/kinectforwindows/

  26. 5. References Install OpenKinect for Windows 7 and XP. // http://kinect.dashhacks.com/kinect-guides/2011/01/09/install-openkinect-windows-7-and-xp Getting Started using OpenKinect. // http://openkinect.org/wiki/Getting_Started#Windows. https://github.com/ofTheo/ofxKinect “OpenCV: CUDA_TOOLKIT_ROOT_DIR not found or specified“. http://xinyustudio.wordpress.com/2011/11/30/opencv-cuda_toolkit_root_dir-not-found-or-specified/

More Related