Capturing, Processing and Experiencing Indian Monuments

1. Capturing, Processing and Experiencing Indian Monuments BTP Presentation Dr. C.V. Jawahar Syed Ahsan Ishtiaque Kumar Srijan

2. Experiencing a monument • For instance, Golkonda Fort • Many Photographs are taken • How to visualize such a large collection of images.

3. Previous and Related Work • Technologies for visualizing image collection in 3D exist • Microsoft’s Photosynth • It uses Bundle Adjustment step. • Computes 3D and refines results based on the errors • Highly Iterative • Computationally intensive • Not Scalable: Due to high computational complexity, unable to deal with large image datasets • Not Incremental: Adding a new image needs the whole computation to be done again from the scratch. http://photosynth.net/

4. Improvements • We present an approach which is • Incremental • Deals with updation or addition of a new image • Scalable • Works on large image datasets • We are trying to eliminate the need of bundle adjustment, or limit it to smaller dataset. • Achieved by introduction of Matching graph, and dividing it into subgraphs.

5. Solution Overview • Creating a Matching Graph • Compute SIFT features in the images • Cluster features into Visual Words • Presence of similar visual words indicate similarity • Pose and orientation estimation • Essential matrix(E) between 2 images is computed. • E is decomposed to get R and t between images. • Visualization • Place image in a 3d world according to R and t. • Project neighbouring images onto the current image plane. • For transition synthesize proper intermediate views.

6. Incremental and Scalability • The matching graph makes the process scalable. • Graph is further decomposed into subgraphs. • Computations are performed on these subgraphs. • Relationship between subgraphs is computed using common images. • New image is added incrementally • Compute neighbouring images • Add to the matching graph • Modify the subgraphs which got affected.

7. Generating Matching Graphs Given a set of images Find images which are geometrically close to each other Place an edge between the two Repeat for all the images Edge represents that images are geometrically close.

8. Example* *Manually Constructed Graph *Manually Constructed Graph

9. Pose and Orientation Estimation SIFT Features are detected. For an edge in Matching graph Features are matched using RANSAC, and spurious matches are eliminated Fundamental and Essential Matrix are estimated. x x’ F Spurious Matches Correct Matches Essential Matrix is decomposed to obtain R and T between the two cameras

10. Example Relative Position and Orientation between two cameras

11. Visualization • Known pose and orientation • Two cases exist • Viewing an Image Ci • Translating from Ci to Cj 1-t t Cj Ci Plane(Ci) Ci

12. Example • Video: Gate Dataset, Golkonda. • Video: Way to Hill top, extracted from Photosynth. • Video: Way to Hill top, from our browser. http://photosynth.net/

13. Conclusion • Matching graph computation serves as most important step in the entire process • An offline process, consumes time at start. • Growing this graph is efficient which makes the process incremental. • Computations on subgraph and computing relationships between subgraphs make the process scalable. • Future Work • Automatic construction of Matching graph • Currently done manually • Visualization can be improved with more features.

14. Thank You