A Deep Tracking and Segmentation Approach for Soccer Videos Visual Effects

Abstract

The applications of deep learning algorithm in sports contain enormous potential. Specifically, in soccer, tracking algorithm could record the tracks of players, which could play as an assistant to assess team performance and evaluate strategies. Moreover, through segmentation model, we could extract semantic attributes of players. This auxiliary information may contribute to the special visual effects processing in broadcasting or entertainment area. Unlike general tracking tasks, soccer videos contain much more cases of deformation, blur, and occlusion. In this paper, we propose a novel model which could combine tracking and segmentation together. A novel deformable cross-similarity correlation (DF_CORR) is adopted to estimate the deformation of players. A new soccer tracking dataset is established to evaluate the performance of top-ranked trackers in soccer videos. In soccer tracking dataset, our model outperforms the state-of-the-art trackers whose accuracy is decreased significantly compared with the general tracking tasks. Moreover, our extensive experiments show comparable segmentation performance against SiamMask, while running in a real-time speed of 36.2FPS.

Publication
Pattern Recognition and Computer Vision
Li Song
Li Song
Professor, IEEE Senior Member
Jun Ling
Jun Ling
PhD Student

I’m now a PhD student at SJTU MediaLab, supervised by Prof. Li Song. Prior to join Song’s MediaLab, I had got my bachelor degree and master degree from University of Sience and Technology of China and Shanghai Jiao Tong University, in 2018 and 2021 respectively. My research interests focus on image and video generation, deep learning and computer vision.

Related