Robust statistical method for background extraction in image segmentation

1. Robust statistical method for background extraction in image segmentation Doug Keen March 29, 2001

2. Source Paper • Rodriguez, Arturo A., Mitchell, O. Robert. “Robust statistical method for background extraction in image segmentation” Stochastic and Neural Methods in Signal Processing, Image Processing, and Computer Vision. Vol. 1569, 1991

3. Problem • Given a digital image, how can we differentiate objects of interest from the background?

4. Problem One simple and fast method is thresholding • Create a graytone histogram of a sample of the background • Find threshold values for the right and left shoulders of the background histogram • Compare graytone values of all other pixels of the image to the background histogram • If pixel falls between right and left background thresholds, that pixel belongs to the background

5. Problems with thresholding • Local background variations may be small, but background variations across the entire image may be substantial • Thresholding also assumes just a single background

6. Problems with thresholding • In applications with a fixed background, one could do an empirical analysis of a background only image • Once an object is introduced, graytone properties of the background could change (due to reflectance properties of the object, scene illumination, automatic gain control of the camera, and/or shadows)

7. Alternate solution • A background extraction method that is based on local statistical measurements performed on log-transformed image data • Works despite smooth changes in background • Doesn’t matter if objects are darker or brighter than the background • Works with multiple backgrounds • Works without a priori knowledge of the background of the image

8. Log-Transformation • Log-transformation can help reduce various illumination effects in the image g(x, y) = i(x, y) * f(x, y) g(x,y) is the grayscale value of a pixel i(x,y) is the illumination component f(x,y) is the reflectance component • A log-transformation would cause multiplicative illumination effects to become additive

9. Log-Transformation In an image with graytone values: g  [gL, gR] The log-transformation is expressed as: Where Pis a fraction of the original gray scale resolution

10. Background Extraction Segmentation Method Step 1: • Decompose the image into a grid of non-overlapping blocks Blocks along the periphery are boundary blocks, and blocks that are not boundary blocks are interior blocks.

11. Background Extraction Segmentation Method Step 2: • For each block in the image, calculate the mean graytone, the left and right standard deviations of the log-transformed histogram (denoted by , Tleft, and Trightrespectively)

12. Background Extraction Segmentation Method • Step 2 (cont.): • A block can be considered homogenous if Q percent or more of its pixels belong to the same class. Specifically, a block can be considered homogenous if: • and • Where S is the standard deviation of the image • Homogenous blocks can be object-homogenous or background-homogenous. If the block is not homogenous, it is considered uncertain.

13. Background Extraction Segmentation Method • Step 2 (cont.): • Homogenous boundary blocks are assumed to be background blocks, and their measured statistical parameters are considered their background distributions

14. Background Extraction Segmentation Method • Step 3: • Non-homogenous boundary blocks are examined by starting from an arbitrary homogenous boundary block and proceeding [counter]clockwise around the boundary blocks • The background parameters of a non-homogenous boundary block are estimated from the two nearest background-homogenous blocks • Once the background parameters of the non-homogenous boundary block have been estimated, that block is marked background-homogenous

15. Background Extraction Segmentation Method • Step 4: • Interior blocks are then examined in a certain sequence that assures that the block being examined has 3 adjacent blocks that have already had their background parameters estimated. One of these blocks must be horizontally adjacent, one must be vertically adjacent, and the third must be diagonally adjacent between the two other adjacent blocks.

16. Background Extraction Segmentation Method • Step 4 (cont.): • A homogenous interior block is considered object-homogenous if its measured graytone mean and the background mean of a vertically or horizontally adjacent block are significantly different. Otherwise it is background-homogenous. • Background parameters of background-homogenous blocks are measured, while background parameters of object-homogenous or uncertain blocks are estimated.

17. Background Extraction Segmentation Method • Step 5: • Once the background parameters of each block have been measured or estimated, calculate the left and right shoulder thresholds of the background distribution using the following formulas: • Where is a prespecified constant (in the case of this study’s experiments, 2.5) • Assign these threshold values to the center of each block

18. Background Extraction Segmentation Method • Step 5: • (dots indicate positioning of threshold values)

19. Background Extraction Segmentation Method • Step 6: • Left and right shoulder thresholds for a pixel (x, y) are obtained by bilinear interpolation of the left and right shoulder thresholds assigned to the four block centers surrounding that pixel • Let L[g(x,y)] be the log-transformed graytone of pixel (x,y) • Pixel (x,y) is darker than the background if L[g(x,y)] < tleft • Pixel (x,y) is lighter than the background if L[g(x,y)] > tright

20. Background Extraction Segmentation Method • Step 6 (cont.): • In order to preserve local brightness relationships between object pixels and background pixels, keep two floating histograms as pixels are being classified: one of bright pixels (Fb) and the other of dark pixels (Fd). • The abscissa of these floating histograms represents the difference between the log-transformed data value of a pixel and the log-transformed data value of the corresponding background shoulder threshold at that pixel

21. Background Extraction Segmentation Method • Step 7: • Once every pixel has been classified by the background extraction procedure, calculate the mean and the left and right standard deviations of Fb and Fd. • Calculate left and right shoulder thresholds of Fb and Fd:

22. Background Extraction Segmentation Method Final pixel classification can then be obtained from: (Where represents the label associated with each of the descriptive categories)

23. Experimental Results • See figures in handout

24. Assessment

25. Possible Improvements • Color image support • Processing spatial and local information in addition to brightness information to reduce misclassified pixels • Robust and real-time performance on natural scenes