SAGAN

SAGAN: GANs with self-attention feature maps

Released in: Self-Attention Generative Adversarial Networks

Contributor:

by

Homepage

Summary

The paper proposes the Self-Attention Generative Adversarial Network (SAGAN) which allows attention-driven, long-range dependency modeling for image generation tasks. Traditional convolutional GANs generate high-resolution details as a function of only spatially local points in lower-resolution feature maps. In SAGAN, details can be generated using cues from all feature locations. Moreover, the discriminator can check that highly detailed features in distant portions of the image are consistent with each other. Furthermore, recent work has shown that generator conditioning affects GAN performance. Leveraging this insight, the authors apply spectral normalization to the GAN generator and find that this improves training dynamics. The proposed SAGAN performs better than prior work, boosting the best published Inception score from 36.8 to 52.52 and reducing Fréchet Inception distance from 27.62 to 18.65 on the challenging ImageNet dataset. Visualization of the attention layers shows that the generator leverages neighborhoods that correspond to object shapes rather than local regions of fixed shape.

2018

Year Released

Key Links & Stats

Self-Attention-GAN

SAGAN

Self-Attention Generative Adversarial Networks

@InProceedings{pmlr-v97-zhang19d, title = {Self-Attention Generative Adversarial Networks}, author = {Zhang, Han and Goodfellow, Ian and Metaxas, Dimitris and Odena, Augustus}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {7354--7363}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/zhang19d/zhang19d.pdf}, url = {https://proceedings.mlr.press/v97/zhang19d.html}, abstract = {In this paper, we propose the Self-Attention Generative Adversarial Network (SAGAN) which allows attention-driven, long-range dependency modeling for image generation tasks. Traditional convolutional GANs generate high-resolution details as a function of only spatially local points in lower-resolution feature maps. In SAGAN, details can be generated using cues from all feature locations. Moreover, the discriminator can check that highly detailed features in distant portions of the image are consistent with each other. Furthermore, recent work has shown that generator conditioning affects GAN performance. Leveraging this insight, we apply spectral normalization to the GAN generator and find that this improves training dynamics. The proposed SAGAN performs better than prior work, boosting the best published Inception score from 36.8 to 52.52 and reducing Fréchet Inception distance from 27.62 to 18.65 on the challenging ImageNet dataset. Visualization of the attention layers shows that the generator leverages neighborhoods that correspond to object shapes rather than local regions of fixed shape.} }

ML Tasks

  1. General
  2. Image Generation

ML Platform

  1. Pytorch

Modalities

  1. General

Verticals

  1. General

CG Platform

  1. Not Applicable

Related organizations

Google Research