Self - similarity and points of interest

1. Self-similarity and points of interest Jasna Maver

2. What are self-similarity detectors? Self-similarity detectors detect locations in images where local circular regions can be well approximated by average intensity values computed either for annuli or circular sectors

3. A circular region is divided either into • annuli or • circular sectors Each annulus or circular sector is represented by the average intensity value of its pixels

4. Weighting by 1/r is used • This effectively scales all circles to have the same circumference, so all annuli are equally important • This means polar representation of a local region • The number of annuli in a local regionrepresents scale

5. An example Blob-like features approximate local regions by averages computed for annuli Original image with resolution (66×72) pixels (b) Green circles denote blob-like features obtained for scales between 4 to 12 (c) Reconstruction obtained with the feature approximations (d),(e) Features and their approximations

6. How are feature locations detected? • By computing a proportional reduction error for each pixel location and scale • Proportional reduction error is: Its meaning is explained by the next slides

7. How are feature locations detected? • consists of intensity values: • Sum of intensity values of -th circular sector: • Sum of intensity values of -th annulus: P

8. How are feature locations detected? • The total sum-of-squares • A local region of intensity values is represented by the average intensity value • This representation is evaluated by the total sum-of-squares:

9. How are feature locations detected? • The between-sets sum of squares • A local region is represented by averages computed either for annuli or circular sectors • This representation is evaluated by the within-sets sum of squares For circular sectors: For annuli: For circular sectors: For annuli:

10. How are feature locations detected? • A proportional reduction error tells how much of the total sum of squarederrors is removed by representing a local region with the set averages, sets are here circular sectors or annuli • The maximal value means that there is no variations of intensity values within sets, variations are only between sets, representation by the set averages is error-free and sets are determined in the best way • The minimal value means that representation by the set averages is no better than representation by the average of a local region

11. How are feature locations detected? • The total sum of squares can be decomposed into three partial sums ofsquares: = • By dividing the above equation by three different proportional reduction errors or saliency measures are obtained:

13. Three different types of features (b) (c)

14. How are feature locations detected? Saliency maps are computed for different number of annuli or scales Image resolution: pixels as an RGB colour image for scale Local scale-space maxima computed on saliency maps arefeature locations

15. Local scale-space maxima(LSSM) LSSM of LSSM of LSSM of 120 best LSSM 120 best LSSM 400 best LSSM

16. Properties The triple • is invariant to rotation, photometric shift, and the magnitude of the local region contrast • is covariant with translation and scaling • is robust to intra-class variations LSSM of

17. Intra-class variations Regions belonging to the highest 30 local scale-space maxima of obtained for scales between and. Image resolution is approximately pixels. Image shows all women from the Caltech Human-Facesimage set

18. Image reconstruction from LSSM of tangential saliency Features Feature approximations Image resolution pixels LSSM of tangential saliency Image reconstructed from feature approximations

19. Reconstructions for different scales LSSM of LSSM of LSSM of Resolution pixels

20. Image reconstruction from LSSM of radial saliency Features Feature approximations Image resolution pixels Features Reconstruction from feature approximations

21. Higher image resolution Image reconstruction obtained for LSSM of Image reconstruction obtained for LSSM of Imageresolution pixels

22. Local region classification Image resolution pixels Scale:

23. Lenna (a) (a)120 bestLSSM of ; (b)120 best LSSM of ; (b)1000 best LSSM of (b) (c)

24. Lenna Image reconstruction obtained for LSSM of 2269 features; Local region classification;