Background Detection work in progress

1. Background Detectionwork in progress Roger S. Gaborski Roger S. Gaborski

2. Main Reference Papers • “Robust Tracking of Human Motion” – Dan Buzan, Boston University • “Adaptive Background Mixture Model for Real-time Tracking,” Stauffer and Grimson, MIT • Both of these papers operate on the premise that each pixel in an image can be modeled with a Gaussian distribution(s) • The parameters of the distribution change with changes in the background environment Roger S. Gaborski

3. Approach- develop understand of current methods • First, implement Buzan’s approach • Evaluate approach under wide range of conditions • Implement Stauffer and Grimson’s approach • Evaluate approach under wide range of conditions • Read the literature • Propose, implement and test new algorithms Roger S. Gaborski

4. Test Video'Test1Sidewalk3.avi'; Roger S. Gaborski

5. Consider Two Locations:no movement in white locationmovement in yellow location Roger S. Gaborski

6. White Dot Region Consider pixel in upper right corner ( white circle, brick area). No motion in this area. Mean of RED 124.7700 STD of RED 2.3044 Mean of GREEN 88.2700 STD of GREEN 2.1736 Mean of BLUE 78.4900 STD of BLUE 2.2133 Roger S. Gaborski

7. Data for 100 frames Roger S. Gaborski

8. Consider area where there is motion (yellow circle, student with dark jacket walks through this point. Roger S. Gaborski

9. Red, Green and Blue Data Roger S. Gaborski

10. Opening an avi Video File in MATLAB % Read in the avi file fname = 'Test1Sidewalk3.avi'; %fname = 'sidewalk10_09.avi'; a = aviread(fname); % Get the information for the avi frameInfo = aviinfo(fname); totalFrames = frameInfo.NumFrames height = frameInfo.Height width = frameInfo.Width Roger S. Gaborski

11. Computing Statistics Use 100 frames of data to calculate statistics: First smooth data to reduce noise: Smoothing each frame by [1 1 1; 1 1 1; 1 1 1]/9 reduces the std deviation of a non moving region by a factor of 2. filterLP=[1 1 1;1 1 1;1 1 1]/9; For each frame in image: ss(:,:,1)=conv2(double(currentImage(:,:,1)),filterLP,'same'); ss(:,:,2)=conv2(double(currentImage(:,:,2)),filterLP,'same'); ss(:,:,3)=conv2(double(currentImage(:,:,3)),filterLP,'same'); Roger S. Gaborski

12. Mean For each pixel in the image, calculate its red, green and blue mean using the first 100 frames redAvg=zeros(height,width); greenAvg=zeros(height,width); blueAvg=zeros(height,width); redAvg(:,:)= a(1,1).cdata(:,:,1); %initial values, redAvg is an array greenAvg(:,:)= a(1,1).cdata(:,:,2);%initial values, greenAvg is an array blueAvg(:,:)= a(1,1).cdata(:,:,3);%initial values, blueAvg is an array numberFrames = 100; for i=2:100 %process 100 images for mean, first frame in initial array from above for k=1:width for j=1:height redAvg(j,k)= redAvg(j,k)+double(a(1,i).cdata(j,k,1)); greenAvg(j,k)= greenAvg(j,k)+double(a(1,i).cdata(j,k,2)); blueAvg(j,k)= blueAvg(j,k)+double(a(1,i).cdata(j,k,3)); end end end redAvg=redAvg/numberFrames; greenAvg=greenAvg/numberFrames; blueAvg=blueAvg/numberFrames; Roger S. Gaborski

13. Covariance for Each Pixel Covariance for each pixel is estimated using N frames (say, 100) Let up = (redAvgp, greenAvgp,blueAvgp)T , where p is pixel p and T is transpose I(p.t) is the intensity of pixel p at frame t Kp = (1/N-1)  (I(p,t)-up) (I(p,t)-up)T Dimensions of K I(p,t) and up is 3x1 , so (I(p,t)-up) is 3x1 (I(p,t)-up)Tis 1x3, soKp is 3x3 Each pixel in the image has a red, green and blue mean And a 3x3 covariance matrix GAUSSIAN MODEL IS DESCRIBED BY THE MEAN AND COVARIANCE (MATLAB has a cov function for calculating covariance) Roger S. Gaborski

14. Updating • Because of changes in the scene (environmental, shadows, etc.) it is necessary to update the background model. • One approach is to only update the mean of the model • Assume we have a binary map Mt associate with frame t. 1’s in the map represent foreground objects •  is a learning rate indicates how much of current background should be maintained •  is a learning rate indicates how much of current background occupied by foreground pixels in the current frame should be maintained •  and  are close to 1 Roger S. Gaborski

15. Update-2 u(t) = [ (1- ) I(t-1) +  u(t-1) ] Mt-1 + [ (1- ) I(t-1) + u(t-1) (1- Mt-1) Roger S. Gaborski

16. Blob Detection We now have a background model. We will compute the distance Between the current frame and the background using the log-likelihood measure: dp(t)= -.5 [ (I(p,t)-up(t)) Kp-1 (I(p,t)-up(t))T +ln | Kp | +n*ln(2) ] n is dimension of color space (note: MATLAB has functions for determinants, transpose and matrix inverse) The magnitude of the distance determines if the pixel p belongs to the background or the foreground Roger S. Gaborski

17. Set of all dp(t) distances generates a gray level image We apply morphological operations to eliminate small holes and regions First threshold gray level difference image to obtain a binary image Apply closing operation Perform connected component analysis to generate a set of blobs b(t)= ConnCompAnalysis(U[dp(t) < thres] Remove small blobs with size filter Result is binary map M Roger S. Gaborski

18. Movies- sidewalk10_09.avi Note: first 100 frames (approx) contain motion. Exclude these frames When calculating means and covariance matrices Roger S. Gaborski

19. Processed: oct11thres15meanA90 Roger S. Gaborski

20. Processed: oct10thres15.avino mean update Roger S. Gaborski

21. Problems • Background recovered as foreground • Loose portions of walking individuals • Analysis: • Analyze individual pixels – why are background pixels being labeled as foreground? • Why are foreground pixels being labeled as background • Is dp(t) being calculated as you would expect, is threshold ok? • Is the mean update formula working correctly? Roger S. Gaborski