DVRCNN: Dark Video Post-processing Method for VVC

Abstract

Low-light videos are usually accompanied with acquisition noise, motion blur and some other specific distortions, which makes it hard to compress by the video coding technologies and generates less satisfying compressed videos. In order to enhance both the subjective and objective quality of compressed dark videos, we propose a novel learning based post-processing scheme for the most recent VVC. Specifically, we adopt a multi-scale residue learning structure, named Dark Video Restoration Convolutional Neural Network (DVRCNN), as an additional out-loop post-processing method. To avoid the over-smooth effect by MSE metric, SSIM and texture loss are also added to the final loss function. Luma and chroma components are decomposed then fed to two corresponding models separately. Compared with VVC baseline on the six sequences given in ICME 2020 Grand Challenge, our approach significantly reduces the BD-rate by 36.08% and achieves a fair promotion on both objective and subjective quality, especially for the low bit-rate compressed sequence with severe distortion. Validation results show that the proposed model generates well to continuous scenes and variable bitrates.

Publication
MultiMedia Modeling
Donghui Feng
Donghui Feng
PhD Student

I’m a Research PHD candidate at SJTU Media Lab. I’m doing my research on Neural Network based Video Coding and End-to-End Video Coding, under the direction of Prof. Li Song and Prof. Wen Gao.

Yiwei Zhang
Yiwei Zhang
Master Student

I’m a master student at SJTU Media Lab. I’m doing my research on multimedia system and video coding, under the direction of Prof. Li Song.

Chen Zhu
Chen Zhu
PhD Student

I’m a Research PHD candidate at SJTU Media Lab. My research interest is video coding optimization, under the direction of Prof. Li Song.

Han Zhang
Han Zhang
PhD Student

I received the B.S. degree in University of Electronic Science and Technology of China, in 2014. And I am currently pursuing the Ph.D. degree in Electronic Engineering with Shanghai Jiao Tong University (SJTU).

Li Song
Li Song
Professor, IEEE Senior Member

Professor, Doctoral Supervisor, the Deputy Director of the Institute of Image Communication and Network Engineering of Shanghai Jiao Tong University, the Double-Appointed Professor of the Institute of Artificial Intelligence and the Collaborative Innovation Center of Future Media Network, the Deputy Secretary-General of the China Video User Experience Alliance and head of the standards group.