Non-linear filtering

Non-linear filtering • Example: Median filter • Replaces pixel value by median value over neighborhood • Generates no new gray levels

Median filters Advantage (?): the “odd-man-out” effect e.g. 1,1,1,7,1,1,1,1  ?,1,1,1.1,1,1,?

Median filters: example filter width = 5 Input Median Linear (averaging)

Median filters: example filter width = 5 Input Discards spike Median Preserves jump exactly at correct location Linear (averaging)

Median filters: example filter width = 5 Input Median Smooths spike Linear (averaging) Blurs jump

Median filter Original 3 x 3 median filter Keeps edges sharp  Odd Man Out can destroy details  Sharpens edges  Destroys edge cusps and protrusions 

Median filter vs Gaussian Gaussian 3 x 3 median filter Eye has more detail. Blurrier edges

Median filter: summary • Purpose • Smoothimagewithout blurring boundaries Input Median Filter

Median filter: summary • Purpose • Smooth image without blurring boundaries • Problem • Loses small details Input Median Filter

Another approach: bilateral filter Input Gaussian Smoothed

Another approach: bilateral filter [ Input Gaussian Smoothed OK to average these brightness values

Another approach: bilateral filter [ Input Gaussian Smoothed Don’t average these (brightness too different, probably from different objects)

Another approach: bilateral filter Input Gaussian Smoothed • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

Another approach: bilateral filter Input Gaussian Smoothed Sum over all 3 pixels • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

Another approach: bilateral filter Input Gaussian Smoothed Sum over all 3 pixels (similar brightness) • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

Another approach: bilateral filter Input Gaussian Smoothed Don’t sum over 3rd pixel (too dark) • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

Another approach: bilateral filter Input Gaussian Smoothed Don’t sum over 3rd pixel (too bright) • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

Another approach: bilateral filter Input Gaussian Smoothed Sum over all 3 pixels (similar brightness)) • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

Another approach: bilateral filter Input Bilateral Smoothing • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

Another approach: bilateral filter Smoothed Input Bilateral Smoothing • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

Another approach: bilateral filter Not smoothed Input Bilateral Smoothing • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

Another approach: bilateral filter Input Bilateral Smoothing • Bilateral smoothing: implementation • Ordinary smoothing… • But only sum over pixels with similar brightness • Pixels with similar brightness weighted more

Bilateral filter • Weight pixel contributions by nearness in distance and nearness in intensity • Old: Gaussian smoothing

Bilateral filter • Weight pixel contributions by nearness in distance and nearness in intensity • Old: Gaussian smoothing Normalization (so weights sum to 1)

Bilateral filter • Weight pixel contributions by nearness in distance and nearness in intensity • Old: Gaussian smoothing • New: bilateral smoothing

Bilateral filter • Weight pixel contributions by nearness in distance and nearness in intensity • Old: Gaussian smoothing • New: bilateral smoothing Suppresses contribution from pixels with different brightness

Bilateral filter

Pattern detection with filters • Biggest response to brightness patterns resembling them! • Example: 1D correlation with mask

Pattern detection with filters • Biggest response to brightness patterns resembling them! • Example: 1D correlation with mask Compute at position 3

Pattern detection with filters • Biggest response to brightness patterns resembling them! • Example: 1D correlation with mask Compute at position 3 Image (1D) … mask weights

Pattern detection with filters • Biggest response to brightness patterns resembling them! • Example: 1D correlation with mask Compute at position 3 Vector i Image (1D) … mask weights Vector w

Pattern detection with filters • Biggest response to brightness patterns resembling them! • Example: 1D correlation with mask Compute at position 3 Vector i Image (1D) … mask weights Vector w dot product

Pattern detection with filters • Gaussian filter • Detects blobs • “Derivative” filter e.g., [-1/2 0 1/2] • Detects brightness jumps • eye detector?

Pattern detection with filters • Summary: detect pattern (object) by correlating with similar mask • Problems when lighting changes • Overall brightness can change between images • Contrast may change • Or both…

Want to detect pattern Old Correlate image with Pattern “detected” wherever is large To improve detection: Compensate for changes in brightness and contrast

Want to detect pattern Old Correlate image with Pattern “detected” wherever is large New: at each pixel (i,j) Subtract average brightness (to eliminate problems from additive shifts) Do weighted sum To improve detection: Compensate for changes in brightness and contrast

New: at each pixel (i,j) Subtract average brightness Do weighted sum Normalize (avoids problems from contrast changes) To improve detection: Compensate for changes in brightness and contrast

Normalized Correlation Important: for each (x,y)(i.e., each pixel in result ) you must compute the average for window centered over (x,y)

Normalized Correlation • MATLAB demo (eye detection)

Detection with Normalized Correlation • Problem Result of normalized correlation Suppose you want 3 best candidates for an eye…

Detection with Normalized Correlation • Problem Result of normalized correlation The three highest NCvalues are all here (at best eye position plus 2 nearby pixels) Suppose you want 3 best candidates for an eye…

Detection with Normalized Correlation • Solution Result of normalized correlation Only consider pixels with higher NC value than at all other “nearby” pixels

Detection with Normalized Correlation • Solution Result of normalized correlation (Non Ma Only consider pixels with higher NC value than at all other “nearby” pixels (Non maximum suppression)

Detection with Normalized Correlation • Solution Result of NC and NonMax Suppression Result of normalized correlation

Non-linear filtering