A Brief Overview of Computer Vision

1. A Brief Overview of Computer Vision Jinxiang Chai

2. What is Computer Vision? • Computer vision is the science and technology of machines that see. • Concerned with the theory for building artificial systems that obtain information from images. • The image data can take many forms, such as a video sequence, views from multiple cameras, or multi-dimensional data from a medical scanner

3. Applications • Robot perception (e.g. an industrial robot or an autonomous vehicle, autonomous helicopter, humanoid robots).

4. Honda ASIMO Humanoid Robot • Face detection • Face recognition • Posture/gesture recognition (e.g., hand waving) • Environment recognition (e.g., obstacles)

5. Applications • Robot perception (e.g. an industrial robot or an autonomous vehicle, humanoid robots). • Detecting events (e.g. for visual surveillance or people counting).

6. Detecting Events • Customer tracking and activity analysis

7. Applications • Robot perception (e.g. an industrial robot or an autonomous vehicle, humanoid robots). • Detecting events (e.g. for visual surveillance or people counting). • Modeling objects or environments

8. Modeling objects or environments • Modeling buildings, plants, faces, cars etc.

9. Applications • Robot perception (e.g. an industrial robot or an autonomous vehicle, humanoid robots). • Detecting events (e.g. for visual surveillance or people counting). • Modeling objects or environments • Interaction (e.g. as the input to a device for computer-human interaction).

10. Interactions • Interactions with computers and video games, etc. Face recognition for automatic login Computer vision for game interfaces (Sony eyetoy, Microsoft Kinect)

11. Applications • Robot perception (e.g. an industrial robot or an autonomous vehicle, humanoid robots). • Detecting events (e.g. for visual surveillance or people counting). • Modeling objects or environments • Interaction (e.g. as the input to a device for computer-human interaction). • Organizing information (e.g. for indexing databases of images and image sequences).

12. Organizing information • Flickr (www. Flickr.com) has 3 billion images • Youtube has tons of videos. • Need new ways to search, analyze, summarize a large collection of internet images and videos

13. Image Representation An image is a 2D rectilinear array of Pixels - A width X height array where each entry of the array stores a single pixel

14. Image Representation A pixel stores color information Luminance pixels - gray-scale images (intensity images) - 0-255 - 8 bits per pixel Red, green, blue pixels (RGB) - Color images - Each channel: 0-255 - 24 bits per pixel

15. Image Representation An image is a 2D rectilinear array of Pixels - A width X height array where each entry of the array stores a single pixel - Each pixel stores color information (255,255,255)

16. Images • Which kind of information you can obtain from images

17. Images • Which kind of information you can obtain from images Edge detection

18. Images • Which kind of information you can obtain from images Edge detection Corner& feature detection

19. Images • Which kind of information you can obtain from images Edge detection Corner& feature detection Geometric primitive detection

20. Images • Which kind of information you can obtain from images Edge detection Corner& feature detection Geometric primitive detection Object detection

21. Images • Which kind of information you can obtain from images Edge detection Corner& feature detection Geometric primitive detection …… Face alignment and recognition Object detection

22. How about multiple images? • What can we obtain if we have multiple images?

23. How about multiple images? • What can we obtain if we have multiple images? Two images of the same scene

24. Structure and motion analysis • Given two or more images of the same scene or object, estimate camera motion and 3D object structure (e.g., depth) unknown camera viewpoints

25. Structure and motion analysis • Given two or more images of the same scene or object, estimate camera motion and 3D object structure (e.g., depth) unknown camera viewpoints How to estimate camera parameters? - where is the camera? - where is it pointing? - what are internal parameters, e.g. focal length?

26. Structure and motion analysis • Given two or more images of the same scene or object, estimate camera motion and 3D object structure (e.g., depth) unknown camera viewpoints How to estimate camera parameters? - where is the camera? - where is it pointing? - what are internal parameters, e.g. focal length? Camera calibration!

27. Structure and motion analysis • Reconstruct the depth information. Input images How to find the depth information of this point?

28. Structure and motion analysis • Reconstruct the depth information. Input images How to find the depth information of this point? - find the corresponding point in the right image.

29. Structure and motion analysis • Reconstruct the depth information. Input images

30. Structure and motion analysis • Reconstruct the depth information. Input images Depth images

31. Structure and motion analysis • Reconstruct 3D models from multiple images Reconstruction results from 23 images

32. All together video • Click here - feature detection - feature matching (epipolar geometry) - structure from motion - stereo reconstruction - triangulation - texture mapping

33. How about video sequences? • What can we obtain from video?

34. How about video sequences? • What can we obtain from video? Optical flow: where are pixels moving to?

35. How about multiple video sequences • Modeling dynamic objects (video click here)

36. Modeling human motion from video • Single-view camera • Interactively construct human motion form video