A Deep Tracking and Segmentation Approach for Soccer Videos Visual Effects

Abstract

The applications of deep learning algorithm in sports contain enormous potential. Specifically, in soccer, tracking algorithm could record the tracks of players, which could play as an assistant to assess team performance and evaluate strategies. Moreover, through segmentation model, we could extract semantic attributes of players. This auxiliary information may contribute to the special visual effects processing in broadcasting or entertainment area. Unlike general tracking tasks, soccer videos contain much more cases of deformation, blur, and occlusion. In this paper, we propose a novel model which could combine tracking and segmentation together. A novel deformable cross-similarity correlation (DF_CORR) is adopted to estimate the deformation of players. A new soccer tracking dataset is established to evaluate the performance of top-ranked trackers in soccer videos. In soccer tracking dataset, our model outperforms the state-of-the-art trackers whose accuracy is decreased significantly compared with the general tracking tasks. Moreover, our extensive experiments show comparable segmentation performance against SiamMask, while running in a real-time speed of 36.2FPS.

Publication
Pattern Recognition and Computer Vision
Li Song
Li Song
Professor, IEEE Senior Member

Professor, Doctoral Supervisor, the Deputy Director of the Institute of Image Communication and Network Engineering of Shanghai Jiao Tong University, the Double-Appointed Professor of the Institute of Artificial Intelligence and the Collaborative Innovation Center of Future Media Network, the Deputy Secretary-General of the China Video User Experience Alliance and head of the standards group.

Jun Ling
Jun Ling
PhD Student

I’m now a PhD student at SJTU MediaLab, supervised by Prof. Li Song. Prior to join Song’s MediaLab, I had got my bachelor degree and master degree from University of Sience and Technology of China and Shanghai Jiao Tong University, in 2018 and 2021 respectively. My research interests focus on image and video generation, deep learning and computer vision.