A Generative Compression Framework For Low Bandwidth Video Conference

Abstract

Video conferences introduce a new scenario for video transmission, which focuses on keeping the fidelity of faces even in the low bandwidth network environment. In this work, we propose VSBNet, one of the frameworks to utilize face landmarks in video compression. Our method utilizes the adversarial learning to reconstruct origin frames from the landmarks. To recover more details and keep the consistency of identity, we propose the concept of visual sensitivity to separate the contour of the face from the fast-moving parts, such as eyes and mouth. Experimental results demonstrate the superiority of our framework with a low bit rate of around 1KB/s.

Publication
2021 IEEE International Conference on Multimedia Expo Workshops (ICMEW)

Demo ~1KB/s, 720p 25fps

Yan Huang
Yan Huang
PhD Student

I’m a Research PHD candidate at SJTU Media Lab. My research interest includes resource-constrained video coding and AI enhanced video coding, under the direction of Prof. Li Song.

Yiwei Zhang
Yiwei Zhang
Master Student

I’m a master student at SJTU Media Lab. I’m doing my research on multimedia system and video coding, under the direction of Prof. Li Song.

Jun Ling
Jun Ling
PhD Student

I’m now a PhD student at SJTU MediaLab, supervised by Prof. Li Song. Prior to join Song’s MediaLab, I had got my bachelor degree and master degree from University of Sience and Technology of China and Shanghai Jiao Tong University, in 2018 and 2021 respectively. My research interests focus on image and video generation, deep learning and computer vision.

Anni Tang
Anni Tang
Master Student

I’m a Research M.S. candidate at SJTU Media Lab. I’m doing my research on multimedia system and face synthesis, under the direction of Prof. Li Song and Prof. Rong Xie.

Li Song
Li Song
Professor, IEEE Senior Member

Professor, Doctoral Supervisor, the Deputy Director of the Institute of Image Communication and Network Engineering of Shanghai Jiao Tong University, the Double-Appointed Professor of the Institute of Artificial Intelligence and the Collaborative Innovation Center of Future Media Network, the Deputy Secretary-General of the China Video User Experience Alliance and head of the standards group.