Evaluating Reliability of Motion Features in Surveillance Videos

1. Evaluating Reliability of Motion Features in Surveillance Videos Longin Jan Latecki and Roland Miezianko, Temple University Dragoljub Pokrajac, Delaware State University November 2004

2. Motion Detection Goals of motion detection • Identify moving objects • Detection of unusual activity patterns • Computing trajectories of moving objects Benefits of reliability assessment • Reduction of false detections (e.g., false alarms)

3. Applications of Motion Detection • Many intelligent video analysis systems are based on motion detection. Such systems can be used in • Homeland security • Real time crime detection • Traffic monitoring …

4. Motion Measure Computation • We use spatial-temporal blocks to represent videos • Each block consists of NBLOCK x NBLOCK pixels from 3 consecutive frames • Those pixel values are reduced to K principal components using PCA (Kahrunen-Loeve trans.) • In our application, NBLOCK=8, K=10 • Thus, we project 192 gray level values to a texture vector with 10 PCA components

5. Frame t-1

6. Frame t

7. Frame t+1

8. 3 principal components -0.5221 -0.0624 -0.1734 4*4*3 spatial-temporal block Location I=7, J=7, time t 48-component block vector (4*4*3)

9. Why texture of spatiotemporal blocks can work better? • More robust in comparison to pixel-based approach • Integrates texture- and movement (temporal) information • Faster

10. 499 624 863 1477

11. Trajectory of block (24,8) (Campus 1 video) Moving blocks corresponds to regions of high local variance Space of spatiotemporal block vectors

12. Trajectory of a pixel from block (24,8) Space of RGB pixel values

13. Detection of Moving Objects Based on Local Variation For each location (x,y) of the frames • Consider vectors of derived attribute values corresponding to a symmetric window of size 2W+1 around each time instant t • Derived attribute vectors: RGB; first 10 PCA projectionsof spatial-temporal blocks, etc. • Compute the covariance matrix for the vectors • motion measureis defined as the largest eigenvalue of the covariance matrix

14. Feature Vectors in Space Feature vectors 4.2000 3.5000 2.6000 4.1000 3.7000 2.8000 3.9000 3.9000 2.9000 4.0000 4.0000 3.0000 4.1000 3.9000 2.8000 4.2000 3.8000 2.7000 4.3000 3.7000 2.6500 Covariance matrix Current time 0.0089 -0.0120 -0.0096 -0.0120 0.0299 0.0201 -0.0096 0.0201 0.0157 Motion Measure Eigenvalues 0.0499 0.00350.0011 0.0499

15. Feature Vectors in Space Feature vectors 4.3000 3.7000 2.6500 4.4191 3.5944 2.4329 4.1798 3.8415 2.6441 4.2980 3.6195 2.5489 4.2843 3.7529 2.7114 4.1396 3.7219 2.7008 4.3257 3.6078 2.8192 Covariance matrix 0.0087 -0.0063 -0.0051 -0.0063 0.0081 0.0031 -0.0051 0.0031 0.0154 Current time Motion Measure Eigenvalues 0.02090.00930.0020 0.0209

16. In our system we divide video plane in disjoint blocks (8x8 blocks), and compute motion measure for each block. mm(x,y,t) for a given block location (x,y) is a function of t

17. Graph of motion measure mm(24,8,:) for Campus 1 video

18. Motion amount The feature called motion amount is defined as • The system decision on alarm situation is based on ma.

20. ma(t) as function of frame number t for Temple 1 video