1. Visual Obstacle Detection SystemProgress Report Aamer Ahmad Lawrence Chan Raymond Llonillo Advisors: Narasimhamurthi Natarajan John Miller

2. Purpose • Use a camera to detect obstacles • Mount the camera on a panning unit • Pan back and forth, from 0° to 180 ° • Place camera on pole located on top of Rhino • Full horizontal View

3. System Overview

4. Panning Unit • PIC16F88 • Programmed using MikroC • DC Motor • At 5V has 43 oz-in of Torque • SN754410 Quadruple Half-H driver • Bidirectional movement • L7805 Voltage regulator • Converts 9V input to 5V for TTL

5. Panning Unit • Wheel Encoder with 24 edges • Placed under Camera mount • Provides way to measure position of motor • QRB1134 Infrared Sensor • Detects edges from encoder • Capture Images every 15°

6. Panning Unit • Panning Unit Progress • Panning unit built • Software to capture images completed • Debugging software to control panning unit

7. Image Stitching • Design Steps • Detecting Interesting Points  Matching Cross Ratios  Scaling  Stitching • Cross Ratio • Given 5 interesting points on an image, can calculate the cross ratio • If 2 sets of 5 points have the same cross ratio, then they are the same 5 points from the same object • Helps when 2 images are taken from 2 different locations • Images show the same objects, but from different perspectives

8. Image Stitching Cross Ratio = 2.4222 Cross Ratio = 2.4078 Percent Error = 0.298%

9. Image Stitching • Progress • Can detect interesting points in an image • Edges & corners • Cross Ratio algorithm designed & implemented • Given 5 points, can calculate the cross ratio • Current algorithm • Randomly choose 5 points on the 1st image & find the closest match with every combination of 5 points on the 2nd image • After finding a match, the 2nd image can be scaled so that the sets of points can overlay each other

10. Image Processing • Highest priority right now • IGVC competition is on June 5-8 • For the competition, will only use: • single stationary camera • image processing software • All group members are currently focusing in completing this part of the VODS • Suggested by Prof. Natarajan

11. Image Processing (Segmentation) • Method 1: Chi-squared (x2) distribution • Probabilities distribution • Determines if the distribution of a sample of data is close to expected results • If the chi-square is a low value, then the sample is a much better match than if it is a high value • In short, it is a “goodness of fit” test to see how well a set of observations fit a statistical model

12. Image Processing (Segmentation) • Method 1 (cont.)

13. Image Processing (Segmentation) • Method 1 (cont.) • Choosing the statistical model • The model that will then be used to compare with everything else • Choose a small box in front of the robot to create the statistical model • The box will most likely consist of the ground the robot is traveling on • Calculate the variance & mean from the model • Calculate the x2 value for each pixel in reference to the model

14. Image Processing (Segmentation) • Method 1 (cont.) Original Image Chi-Squared Image Binary Image

15. Image Processing (Segmentation) • Method 1 Progress • Algorithm is completely designed & implemented • Previous slide are the actual results • The program execution is slow • In relation to the size of the input image • Use of built-in OpenCV functions would be faster • Being compared to alternatives to determine optimal visual system to use for the IGVC

16. Image Processing (Segmentation) • Method 2: Matching Hue • “Similar” • Using Probability and statistics • Find m=mean and s=std deviation of hue patch • Calculate low=(m-3*s) and high=(m+3*s) of the hue patch • Finally, only show values that are within that range Result: Path’s are white obstacles are black

17. Image Processing (Segmentation) • Method 2: Matching Hue (cont.) Original Image Hue Channel “Similar” Hue

18. Image Processing (Detection) • Determine best path • Draw lines for possible paths in segmented image • Move in the direction of the longest line • Note – longest line isn’t longest path due to 3D perspective RHINO ROBOT

19. Image Processing (Controlling) • Once best path is determined • Send control signal to motor control • Ex) Forward, left, sharp right, etc. (no reverse) • Other systems also affect motor control • If VODS fails sonar range finder will detect obstacle • Patch used for obtaining statistical data must not be an obstacle, which can be confirmed using sonar Motor Control Vision Sonar Range Finder Direction & Speed GPS

20. Specifications • IGVC robot will move at a max speed of 1.1 m/sec • Panning unit will rotate at a speed of 275°/sec • Camera will capture images at 30 frames/sec • Stitching process will take less than 1 minute • Each thread will be given a time slice of 10 msec

21. ANY QUESTIONS ????