Introduction to Computer Vision

1. Introduction to Computer Vision Olac Fuentes Computer Science Department University of Texas at El Paso El Paso, TX, U.S.A.

2. What is Computer Vision? Computer Vision is the process of extracting knowledge about the world from one or more digital images

3. Digital Images are 2D arrays (matrices) of numbers:

4. Digital Images Color Images are formed with three 2-D arrays, representing the Red, Green and Blue components of the image.

5. Computer Vision – Main Tasks • Model generation • Object Recognition • Object Detection • Tracking

6. Computer Vision – Object DetectionDetecting Faces

7. Computer Vision – Object DetectionDetecting Faces

8. Computer Vision – Object DetectionDetecting Pedestrians

9. Computer Vision – Object DetectionDetecting Cars

10. Computer Vision – Object DetectionHow to do it? Idea: Use Machine Learning Training: Training Set: • Positive examples are images of objects that belong to the class of interest • Negative examples are images of objects that don’t belong to that class Train classifier using the training set Detection Given an image to analyze, apply classifier to every subimage (there are lots of them, so a low false positive rate is important!)

11. Face Detection – Training Images

12. Efficient Object DetectionViola & Jones, 2005 Idea #1: Classifier Structure Build a cascade classifiers: • Where stage i is simpler (and faster) than stage i+1

13. Efficient Object DetectionViola & Jones, 2005 Idea #2: Features Use a large number of very simple features:

14. Efficient Object DetectionViola & Jones, 2005 Idea #3: Feature Computation Compute the features very efficiently using the integral image:

15. Efficient Object DetectionViola & Jones, 2005 Idea #4: Dealing with multiple scales

16. Efficient Object DetectionViola & Jones, 2005 Idea #4: Dealing with multiple scales Obvious solution: Build a detector for each possible scale

17. Efficient Object DetectionViola & Jones, 2005 Idea #4: Dealing with multiple scales Obvious solution: Build a detector for each possible scale

18. Efficient Object DetectionViola & Jones, 2005 Idea #4: Dealing with multiple scales Obvious solution: Build a detector for each possible scale Better idea: Build a detector for a single scale During detection, scale the image

19. Efficient Object DetectionThe Modified census transform (Froba and Ernst, 2004) Used local intensity descriptors as features

20. Efficient Object DetectionThe Modified census transform (Froba and Ernst, 2004) Used local intensity descriptors as features Used simple voting classifiers and Adaboost to build a cascade of classifiers

21. Efficient Object DetectionHistograms of Gradients (Dalal, 2005) Histograms of Gradients (Dalal, 2005) Used histograms of oriented gradients as features Used Support Vector Machine as classifier Best results to date

22. Object Recognition Training Testing Owl ?? Duck Toucan ?? Egret

23. Object Recognition – Face Recognition Eigenfaces are a set of "standardized face ingredients", derived from statistical analysis of many pictures of faces. First four eigenfaces from the AT&T database

24. Eigenfaces • One person's face might be made up of 10% from face 1, 24% from face 2 and so on. Very few eigenvector terms are needed to give a fair likeness of most people's faces Eigenfaces provide a means of applying data compression to faces for identification purposes.

25. Eigenfaces • Let E1,...,En, be the eigenfaces obtained from a face database Let F1,...,Fm be the images in our training/testing sets. (For the training images we also know the person’s identity) The attributes of Fi are given by the sum of the pixel by pixel products of Fi and E1,...,En, that is, Fi is represented by n numbers: [Fi·E1, Fi·E2, ..., Fi·En] Using the attribute vectors and the class information we can now construct a classifier

26. Tracking Continuous detection of objects of interest in video streams

27. Tracking Continuous detection of objects of interest in video streams

28. Reconstruction • Build a 3D models of world given 2D Images • Most-common Approach: Stereo Vision • Inspired by human 3D perception • Use two cameras of known geometry

29. Reconstruction • Build a 3D models of world given 2D Images • Most-common Approach: Stereo Vision • Inspired by human 3D perception • Use two cameras of known geometry • Take images

30. Reconstruction • Build a 3D models of world given 2D Images • Most-common Approach: Stereo Vision • Inspired by human 3D perception • Use two cameras of known geometry • Take images • Find correspondences • Reconstruct using correspondences and known geometry

31. Reconstruction

32. Reconstruction Problems with Stereo Vision: Finding matches reliably is difficult Calibration is difficult It hard to deal with featureless areas Computationally expensive

33. Reconstruction Microsoft to the rescue!

34. Reconstruction Microsoft to the rescue! Seriously!

35. Reconstruction Microsoft Kinect Reconstruction using active illumination Project a known pattern of light at an invisible wavelength Learn the appearance of that pattern at different distances Fast and easy

36. Reconstruction Microsoft Kinect

37. Reconstruction Microsoft Kinect