Purpose of Background Subtraction • Reduce problem set for further processing • Only process part of picture that contains the relevant information • Segment the image into foreground and background • Add a virtual background
Encountered Problems • Lighting • Shadows • Gradual/Sudden illumination changes • Camouflage • Moving objects • Foreground aperture • Foreground object becomes motionless
Lighting and Shadows • Weight the luminance with other characteristics • Depth of object • Region/Frame information • Adjust the background model with time • Store a history of previous backgrounds
Frame Differencing The accuracy of this approach is dependent on speed of movement in the scene. Faster movements may require higher thresholds
Frame Differencing This approach will only work for cases where all foreground pixels are moving and all background pixels are static.
Mean Filter where N is the number of preceding images taken for averaging. This averaging refers to averaging corresponding pixels in the given images. N would depend on the video speed (number of images per second in the video) and the amount of movement in the video.
Median Filter Nathan Johnson
Median Filter Nathan Johnson
Normal Gaussian Distribution As a result, a pixel, once it has become foreground, can only become background again when the intensity value gets close to what it was before turning foreground. This method, however, has several issues: It only works if all pixels are initially background pixels (or foreground pixels are annotated as such).
Running Gaussian Average • The pdf of every pixel is characterized by mean and variance. • In order to initialize variance, we can, for example, use the variance in x and y from a small window around each pixel. • Note that background may change over time (e.g. due to illumination changes or non-static background objects). To accommodate for that change, at every frame , every pixel's mean and variance must be updated
Algorithm Overview If a pixel is categorized as foreground for a too long period of time, the background intensity in that location might have changed (because illumination has changed etc.). As a result, once the foreground object is gone, the new background intensity might not be recognized as such anymore.
Mixture Model Nathan Johnson
Model Adaptation Nathan Johnson
Background Model Estimation Nathan Johnson
Relevant papers Nathan Johnson
Limitations (Selectivity) Nathan Johnson
Kernel Density Estimators Kernel density estimation is a fundamental data smoothing problem where inferences about the population are made, based on a finite samples. Kernel density estimates are closely related to histograms, but can be endowed with properties such as smoothness or continuity by using a suitable kernell.
Mean Shift Based Estimation Mean shift is a procedure for locating the maxima of density function from given discrete data sampled density function. The simplest such algorithm would create a confidence map in the new image based on the color histogram of the object in the previous image, and use mean shift to find the peak of a confidence map near the object's old position. The confidence map is a probability density function on the new image, assigning each pixel of the new image a probability, which is the probability of the pixel color occurring in the object in the previous image.
Eigen Backgrounds • The method uses the difference between input image and the reconstructed background image to detect foreground objects based on eigenvalue decomposition. • Foreground regions are represented in the reconstructed image using eigenbackground. • As the principle components of a scene are its background in general, the eigenbackgrounds that are used to reconstruct the background depict the characteristics of background.
Spatial Correlation Nathan Johnson
Binary Morphology • Binary Morphological techniques probe an image with a small shape or template called a structuring element. • The structuring element is positioned at all possible locations in the image and it is compared with the corresponding neighbourhood of pixels. • Some operations test whether the element "fits" within the neighbourhood, while others test whether it "hits" or intersects the neighbourhood.
Summary Nathan Johnson
Summary Nathan Johnson
Definition of Motion Detection • Action of sensing physical movement in a give area • Motion can be detected by measuring change in speed or vector of an object
Motion Detection Goals of motion detection • Identify moving objects • Detection of unusual activity patterns • Computing trajectories of moving objects Applications of motion detection • Indoor/outdoor security • Real time crime detection • Traffic monitoring Many intelligent video analysis systems are based on motion detection.
Two Approaches to Motion Detection • Optical Flow • Compute motion within region or the frame as a whole • Change detection • Detect objects within a scene • Track object across a number of frames
Background Subtraction • Uses a reference background image for comparison purposes. • Current image (containing target object) is compared to reference image pixel by pixel. • Places where there are differences are detected and classified as moving objects. Motivation: simple difference of two images shows moving objects
a. Original scene b. Same scene later Subtraction of scene a from scene b Subtracted image with threshold of 100
Model the background and subtract to obtain object mask Filter to remove noise Group adjacent pixels to obtain objects Track objects between frames to develop trajectories Static Scene Object Detection and Tracking
Approaches to Background Modeling • Background Subtraction • Statistical Methods(e.g., Gaussian Mixture Model,Stauffer and Grimson 2000) Background Subtraction: • Construct a background image B as average of few images • For each actual frame I, classify individual pixels as foreground if |B-I| > T (threshold) • Clean noisy pixels