Domain-Adversarial Training of Neural Networks

A new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions

Released in: Domain-Adversarial Training of Neural Networks

Source: Domain-Adversarial Training of Neural Networks

Contributor:

Summary

Author introduces a new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions. Our approach is directly inspired by the theory on domain adaptation suggesting that, for effective domain transfer to be  achieved, predictions must be made based on features that cannot discriminate between the training (source) and test (target) domains.

The approach implements this idea in the context of neural network architectures that are trained on labeled data from the source domain and unlabeled data from the target domain (no labeled target-domain data is necessary). As the training progresses, the approach promotes the emergence of features that are (i) discriminative for the main learning task on the source domain and (ii) indiscriminate with respect to the shift between the  domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers and a new gradient reversal layer. The resulting augmented architecture can be trained using standard backpropagation and stochastic gradient  descent, and can thus be implemented with little effort using any of the deep learning packages.

Author demonstrates the success of this approach for two distinct classification problems (document sentiment analysis and image classification), where state-of-the-art domain adaptation performance on standard benchmarks is achieved. We also validate the approach for descriptor learning task in the context of person re-identification application.

2016

Year Released

Key Links & Stats

fungtion / DANN

Domain-Adversarial Training of Neural Networks

MIT License

Domain-Adversarial Training of Neural Networks

@article{https://doi.org/10.48550/arxiv.1505.07818, doi = {10.48550/ARXIV.1505.07818}, url = {https://arxiv.org/abs/1505.07818}, author = {Ganin, Yaroslav and Ustinova, Evgeniya and Ajakan, Hana and Germain, Pascal and Larochelle, Hugo and Laviolette, François and Marchand, Mario and Lempitsky, Victor}, keywords = {Machine Learning (stat.ML), Machine Learning (cs.LG), Neural and Evolutionary Computing (cs.NE), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Domain-Adversarial Training of Neural Networks}, publisher = {arXiv}, year = {2015}, copyright = {arXiv.org perpetual, non-exclusive license} }

ML Tasks

  1. Domain Adaptation
  2. Image Classification

ML Platform

  1. Pytorch

Modalities

  1. General

Verticals

  1. General

CG Platform

  1. Not Applicable

Related organizations

Skolkovo Institute of Science and Technology (Skoltech) Skolkovo, Moscow Region, Russia

D´epartement d’informatique et de g´enie logiciel, Universit´e Laval Qu´ebec, Canada, G1V 0A6

D´epartement d’informatique, Universit´e de Sherbrooke Qu´ebec, Canada, J1K 2R1