Learning An Inverse Tone Mapping Network with A Generative Adversarial Regularizer

Introduction

With the development of ultra-high-definition television techniques, the demands on high-dynamic-range (HDR) contents are increasing rapidly these years. These demands require us to transfer a large amount of existing low-dynamic-range (LDR) images and videos to HDR contents efficiently and effectively. In such a situation, manually transferring is infeasible because of the huge size of the contents, and we need to apply inverse tone mapping (iTM) techniques.

Unfortunately, traditional iTM techniques cannot meet the urgent requirements. An ideal iTM method should be multi-scale invariant and adaptive to various objects in different lighting conditions, which requires an nonlinear mapping with high complexity. However, most of existing methods simplify the mapping of luminance and that of color as segmented mapping empirically based on histogram equalization and spatial filtering. These methods either ignore the nonlinear nature of iTM or the color correlation between the iTMs across different channels. As a result, they need to adjust parameters according to the contents in images and lead to serious distortions in some situations.

To solve the challenges mentioned above, we propose a novel inverse tone mapping network (iTMN) based on generative adversarial network (GAN). We aim to train a U-Net-based HDR image generator as our inverse tone mapping network, which transfer LDR images to HDR ones. In the training phase, we learn the generator and a corresponding discriminator alternatively. In each iteration, given existing generator the discriminator is updated to distinguish the generated HDR images from the ground truth with higher accuracy. Given updated discriminator, we further update the generator by minimizing a content-based loss with the adversarial regularizer related to the discriminator. As a result, each generated HDR image is close to the ground truth and the distribution of all generated HDR images is identical to that of training HDR images. Applying this learning method, we obtain an effective inverse tone mapping network. It achieves superior performance to existing state-of-the-art methods on both subjective and objective measurements.

Demo for validation dataset

Our Method	No dMSE	No AdvReg	Huo	KO	Kuo