Efficient texture-aware multi-GAN for image inpainting

Abstract

Recent GAN-based (Generative adversarial networks) inpainting methods show remarkable improvements and generate plausible images using multi-stage networks or Contextual Attention Modules (CAM). However, these techniques increase the model complexity limiting their application in low resource environments. Furthermore, they fail in generating high-resolution images with realistic texture details due to the GAN stability problem. Motivated by these observations, we propose a multi-GAN architecture improving both the performance and rendering efficiency. Our training schema optimizes the parameters of four progressive efficient generators and discriminators in an end-to-end manner. Filling in low-resolution images is less challenging for GANs due to the small dimensional space. Meanwhile, it guides higher resolution generators to learn the global structure consistency of the image. To constrain the inpainting task and ensure fine-grained textures, we adopt an LBP-based loss function to minimize the difference between the generated and the ground truth textures. We conduct our experiments on Places2 and CelebHQ datasets. Qualitative and quantitative results show that the proposed method not only performs favorably against state-of-the-art algorithms but also speeds up the inference time. (C) 2021 Elsevier B.V. All rights reserved.