Continuously Indexed Domain Adaptation

Abstract

Existing domain adaptation focuses on transferring knowledge between domains with categorical indices (e.g., between datasets A and B). However, many tasks involve continuously indexed domains. For example, in medical applications, one often needs to transfer disease analysis and prediction across patients of different ages, where age acts as a continuous domain index. Such tasks are challenging for prior domain adaptation methods since they ignore the underlying relation among domains.

In this paper, we propose the first method for continuously indexed domain adaptation. Our approach combines traditional adversarial adaptation with a novel discriminator that models the encoding-conditioned domain index distribution. Our theoretical analysis demonstrates the value of leveraging the domain index to generate invariant features across a continuous range of domains. Our empirical results show that our approach outperforms the state-of-the-art domain adaption methods on both synthetic and real-world medical datasets.

Video

Highlights

New Problem

Past works focus on categorical domain adaptation where having finite number of domains. While our works tackles potentially infinitely many domains that changes continuously.

New Techniques

CIDA: (1) Domain discriminator is a regressor instead of classifer. (2) Encoder takes domain indices as input.
PCIDA: Domain discriminator regress the domain index with an uncertainty estimation.

New Theory

Denoting the domain index as u and the encoding as z, we proof that (please check our paper for full theorems):

Theorem 1 (Informal). CIDA converges, if and only if, the expectation of domain index 𝔼[𝑢 | z] is identical for any embedding z.
Theorem 2 (Informal). PCIDA converges, if and only if, the expectation and the variance of domain index, 𝔼[𝑢 | z] and 𝕍[𝑢 | z] are identical for any z.
Theorem 3 (Informal). The global optimum of the two-player game between 𝐸 and 𝐷 matches the global optimum of the three-play game between 𝐸, 𝐹, and 𝐷.

Intriguing Results

CIDA is able to perform domain interpolation (target domain indices being inside source domain indices) and domain extrapolation (target domain indices being outside source domain indices).
CIDA naturally handles multi-dimensional domains.

BibTeX

@inproceedings{wang2020continuously,
  title={Continuously indexed domain adaptation},
  author={Wang, Hao and He, Hao and Katabi, Dina},
  booktitle={Proceedings of the 37th International Conference on Machine Learning},
  pages={9898--9907},
  year={2020}
}