Application of light fields in computer vision

1. Application of light fields in computer vision Amari Lewis – reu student Aideansharghi- ph.dstuent

2. Main objective • increase object recognition through using the EPI of light field images • Using the light field camera

3. Using the Lytro light field camera • conventional methods- involve using 2D information • Light field images- captures all 3D information in a single shot. • Using the Lytro light field camera to collect dataset • camera captures light field direction, intensity and color

4. Datasets- • 1. Collected own dataset using the Lytro light field camera • Bikes • Buildings • Trees • Vehicles - Studying the 7 different image perspectives

5. 2. Dataset from Switzerland using the iphone video • Buildings – 50 categories • Ranging from 4-30 videos • Extracted 300 frames from each video

6. Epipolar planar images- EPI • It is a 2D representation or slice of an image • Taking the same line from each image and putting it on top of each other • Using the multiple shots taken from the camera and the extracted frames

7. Light field 7 lines from each of the images concatenated- total of 1080

8. Concatenated the 300 lines – total 720

9. Implementing DCT • Steps: • Separate the RGB into 3 channels • Calculate the row-wise mean- calculates the mean of each row to create a vector • Calculate the DCT for each channels • Concatenate some coefficients, using as a feature vector (smaller)

10. For classification • Apply Principal component analysis (PCA) • gmm- Gaussian mixture model • Linear SVM

11. Best Results • Using this method on EPIs • Lytro Light field camera dataset 77% accuracy • Switzerland dataset 96% accuracy

12. Thank you