Computer Vision System for Tracking Players in Sports Games

1. First Int'l Workshop on Image and Signal Processing and Analysis IWISPA 2000, June 14.-15. 2000, Pula, Croatia. Computer Vision System for Tracking Players in Sports Games Janez Perš,Stanislav Kovačič, Faculty of Electrical Engineering, University of Ljubljana, Tržaška 25, 1000 Ljubljana janez.pers@kiss.uni-lj.si, stanislav.kovacic@fe.uni-lj.si

2. Outline Introduction and motivation Image acquisition Camera calibration Player tracking Results Conclusions

3. Introduction, Motivation Human motion analysis in context of team sport - handball Objective: to obtain trajectories: ...for every player, ... ...in the whole field,... ...in every instance of time. Facts: - Up to 14 players may be present on the court at the same time. - Large playing field (20x40 meters). - Duration of the match is 60 minutes. Problems: - Large area to cover - Rapid player motion - Large amount of data to process: 25 fps * 3600 seconds = 90.000 images

4. Image acquisition Camera setup: - Two cameras, fitted with 103° lens were placed 10m above the playing field. - Wide angle lens cause significant distortion of the acquired images. Transfer to digital domain: - Motion-JPEG compression hardware. - 25 fps and 384x288 pixel resolution.Result:12 GB of compressed video data for a single match.

5. Camera calibration Linear model was used to obtain scaling factors, camera orientation and position. Radial lens distortion is the most difficult problem. Instead of polynomial approximation, we built a model of radial distortion. Results: - court plane  image plane - image plane  court plane

6. Player tracking - overview An offline process. Supervised by a human operator, who: - Defines starting player positions before tracking is started. - Corrects tracking errors. Post-processing of obtained trajectories is necessary to reduce jitter. Gaussian filter can be used. Three tracking methods were developed: - Motion detection - Template tracking - RGB (color) tracking

7. Player tracking - 1 Motion detection by subtracting images: 1. Image of the empty playing court. 2. Current frame. 3. Difference image. 4. Thresholding. 5. Filtering by median filter. 6. Blob assignment.

8. Player tracking - 1 Drawback of motion detection algorithm is high sensitivity to: - light reflections, - shadows - non-uniform illumination. There actually four players in the above scene!

9. Player tracking - 2 To exploit visible differences between the players and background “objects”, we propose template tracking. It is extremely difficult to build an accurate model of a player, especially at low resolution. We defined a set of templates , {Kj,j=1..14}, which represent basic appearances of the player: Feature vectorsG and F are derived from background image and current frame at estimated player position (ROI of 16x16 pixels).VectorH is calculated as mean average of past vectors F obtained at true player positions.

10. Player tracking - 2 • Simplified, two dimensional case: • DGF and DHF are Euclidean distances • Similarity measureS {S =[0..1]}reflects the similarity between the object in ROI and the predicted appearance in H. The position of the player in image is defined by the position of local minimum of measure S. Drawback: possibility of adaptation to the background objects.

11. Player tracking - 3 Color can be used to identify players without any need for adaptation. Players may wear black or dark-colored dresses, so we chose the RGB color representation. Problem: There are only a few pixels that correspond to the true color of player’s dress. Our ”Color tracking” algorithm simply searches for the pixel that is the most similar to operator-defined RGB value. Consequence: high amount of jitter.

12. Results • We tested three different methods: • A: Motion detection • B:RGB (color) tracking • C:Combination of color and template tracking: • - results of color tracking are used as a reliable, but inaccurate estimate. • - template tracking is used to increase accuracy and eliminate jitter. • - initial estimate for the next frame is based on the output of the color tracking algorithm to prevent adaptation to the background objects.

13. Results We measured the following: - Number of required operator interventions on a 30 second test sequence. - Processing time per frame with operator interventions disabled. - Path length of a single player. - The mean position error for a single player. We used the mean average of five paths, obtained by manually by five human operators, as the true-position estimate. Operators were performing manual tracking at 2 frames per second – computer generated results were subsampled to ensure unbiased comparison.

14. Results Legend: A: Motion detection B:RGB (color) tracking C:Combination of color and template tracking. O1, O2, O3, O4, O5 – human operators X – not available

15. Conclusions Combination of color and template tracking algorithms requires few operator interventions, and results obtained contain low amount of jitter. It is therefore the most appropriate for use in the automated player tracker. Operator supervision is still required. Results, obtained by automated tracker are less accurate than those obtained manually and have to be filtered to reduce jitter. However, obtained results can be used for further analysis by sports experts.

16. Conclusions

17. Radial distortion Pan-tilt camera assumption: Integration: Inverse formula:

18. Similarity measure S F (unknown object) DGF DFH Legend: white = 1 black = 0 G (background) H (player)

19. Manual vs. computer tracking Legend: Green - human results Black - computer results