KernelGAN

KernelGAN: adversarial training of downscaling kernels for superresolution

Released in: Blind Super-Resolution Kernel Estimation using an Internal-GAN

Contributor:

Summary

Super resolution (SR) methods typically assume that the low-resolution (LR) image was downscaled from the unknown high-resolution (HR) image by a fixed ‘ideal’ downscaling kernel (e.g. Bicubic downscaling). However, this is rarely the case in real LR images, in contrast to synthetically generated SR datasets. When the assumed downscaling kernel deviates from the true one, the performance of SR methods significantly deteriorates. This gave rise to Blind-SR – namely, SR when the downscaling kernel (“SR-kernel”) is unknown. It was further shown that the true SR-kernel is the one that maximizes the recurrence of patches across scales of the LR image. This paper shows how this powerful cross-scale recurrence property can be realized using Deep Internal Learning. The authors introduce KernelGAN, an image-specific Internal-GAN, which trains solely on the LR test image at test time, and learns its internal distribution of patches. Its Generator is trained to produce a downscaled version of the LR test image, such that its Discriminator cannot distinguish between the patch distribution of the downscaled image, and the patch distribution of the original LR image. The Generator, once trained, constitutes the downscaling operation with the correct image-specific SR-kernel. KernelGAN is fully unsupervised, requires no training data other than the input image itself, and leads to state-of-the-art results in Blind-SR when plugged into existing SR algorithms.

2019

Year Released

Key Links & Stats

KernelGAN

Blind Super-Resolution Kernel Estimation using an Internal-GAN

Blind Super-Resolution Kernel Estimation using an Internal-GAN

@inproceedings{NEURIPS2019_5fd0b37c, author = {Bell-Kligler, Sefi and Shocher, Assaf and Irani, Michal}, booktitle = {Advances in Neural Information Processing Systems}, editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett}, pages = {}, publisher = {Curran Associates, Inc.}, title = {Blind Super-Resolution Kernel Estimation using an Internal-GAN}, url = {https://proceedings.neurips.cc/paper/2019/file/5fd0b37cd7dbbb00f97ba6ce92bf5add-Paper.pdf}, volume = {32}, year = {2019} }

ML Tasks

  1. Super-Resolution

ML Platform

  1. Pytorch

Modalities

  1. Still Image

Verticals

  1. General

CG Platform

  1. Not Applicable

Related organizations

The Weizmann Institute of Science, Israel