Minimum Likelihood Image Feature and Scale Detection

1. Minimum Likelihood Image Feature and Scale Detection Kim Steenstrup Pedersen Collaborators: Pieter van Dorst, TUe, The Netherlands Marco Loog, ITU, Denmark

2. What is an image feature? • Marr’s (1982) primal sketch (edges, bars, corners, blobs) • Geometrical features, Marr’s features defined by differential geometry: Canny (1986), Lindeberg (1998) • Iconic features: Koenderink (1993), Griffin & Lillholm (2005) Observation: Features are usually points and curves, i.e. sparsely distributed in space (unlikely events). Features have an intrinsic scale / size. How blurred is the edge?What is the size if a bar? Gaussian Processes in Practice

3. A probabilistic primal sketch • Our definition: Features are points that are unlikely to occure under an image model. Similarly the scale of the feature is defined as the most unlikely scale. • We use fractional Brownian images as a generic model of the intensity correlation found in natural images. Captures second order statistics of generic image points (non-feature points). • The model includes feature scale naturally. • This leads to a probabilistic feature and scale detection. • Possible applications: Feature detection, interest points for object recognition, correspondance in stereo, tracking, etc. Gaussian Processes in Practice

4. Probabilistic feature detection • Feature detection: • Konishi et al. (1999, 2002, 2003) • Lillholm & Pedersen (2004) • Scale selection: • Pedersen & Nielsen (1999) • Loog et al. (2005) Gaussian Processes in Practice

5. Linear scale-space derivatives • Scale-space derivatives: Gaussian Processes in Practice

6. Scale Space k-Jet Representation • We use the k-jet as representationof the local geometry: • (The coefficients of the truncatedTaylor expansion of the blurredimage.) • Biologically plausiblerepresentation (Koenderink et al., 1987) Gaussian Processes in Practice

7. Probabilistic image models • Key results on natural image statistics: • Scale invariance / Self-similarity: Power spectrum, : Field (1987), Ruderman & Bialek (1994) • In general non-Gaussian filter responses! • Fractional Brownian images as model of natural images: • Mumford & Gidas (2001), Pedersen (2003), Markussen et al. (2005) • Jet covariance of natural images resembles that of fractional Brownian images: Pedersen (2003) Gaussian Processes in Practice

8. Fractional Brownian images Gaussian Processes in Practice

9. FBm in Jet space • (Result from Pedersen (2003)) Gaussian Processes in Practice

10. Detecting Features and Scales • Detecting points in scale-space that are locally unlikely (minima): • (We could also have maximised .) Gaussian Processes in Practice

11. Why minimum likeli scales? • Lindeberg (1998) maximises polynomials of derivatives in order to detect features and scales. • Similarly, we maximise in order to detect features and scales. • The difference lies in the choice of polynomial! We use an image model and Lindeberg uses a feature model. Gaussian Processes in Practice

12. Synthetic examples: Double blobs Gaussian Processes in Practice

13. Synthetic examples: Blurred step edge Gaussian Processes in Practice

14. Real Example: Sunflowers Gaussian Processes in Practice

15. Sunflowers: Multi-scale Gaussian Processes in Practice

16. Sunflowers: Fixed scale Gaussian Processes in Practice

17. Summary • Minimising the likelihood of an image point under the fractional Brownian image model detects feature points and their intrinsic scale. • There is a relationship between feature types and the  parameter. • Why over estimation of the scale? • Preliminary results look promising, a performance evaluation is needed (task based?). • The method is pointwise. How to handle curve features (edges, bars, ridges)? Gaussian Processes in Practice